Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

ast.literal_eval() support for set literals in Python 2.7?

In the What’s New in Python 2.7 document it says that support for set literals was back-ported from Python 3.1. However it appears that this support was not extended to the ast module's literal_eval() function, as illustrated below.

Was this intentional, an oversight, or something else — and what are the cleanest workarounds for creating a literal set from a string representation? (I assume the following works in Python 3.1+, right?)

import ast
a_set = {1,2,3,4,5}
print(a_set) 
print(ast.literal_eval('{1,2,3,4,5}'))

Output showing error message:

set([1, 2, 3, 4, 5])
Traceback (most recent call last):
  File "...\setliterals.py", line 4, in <module>
    print ast.literal_eval('{1,2,3,4,5}')
  File "...\Python\lib\ast.py", line 80, in literal_eval
    return _convert(node_or_string)
  File "...\Python\lib\ast.py", line 79, in _convert
    raise ValueError('malformed string')
ValueError: malformed string

P.S. The only workaround I can think of is to use eval().

like image 551
martineau Avatar asked May 20 '11 22:05

martineau


People also ask

What does AST literal_eval do in Python?

The ast. literal_eval method is one of the helper functions that helps traverse an abstract syntax tree. This function evaluates an expression node or a string consisting of a Python literal or container display.

How does AST work in Python?

The ast module helps Python applications to process trees of the Python abstract syntax grammar. The abstract syntax itself might change with each Python release; this module helps to find out programmatically what the current grammar looks like. An abstract syntax tree can be generated by passing ast.

How safe is AST literal_eval?

The documentation states it is safe, and there is no bug relative to security of literal_eval in the bug tracker, so you can probably assume it is safe. Also, according to the source, literal_eval parses the string to a python AST (source tree), and returns only if it is a literal.


1 Answers

I've been using this for converting columns in a pandas DataFrame (df[col] = df[col].apply(to_set). Might be useful for anyone finding this question. It may not be as fast but it avoids using eval.

def to_set(set_str):
    """
    Required to get around the lack of support for sets in ast.literal_eval. 
    It works by converting the string to a list and then to a set.

    Parameters
    ----------
    set_str : str
        A string representation of a set.

    Returns
    -------
    set

    Raises
    ------
    ValueError
        "malformed string" if the string does not start with '{' and and end 
        with '}'.

    """
    set_str = set_str.strip()
    if not (set_str.startswith('{') and set_str.endswith('}')):
        raise ValueError("malformed string")

    olds, news = ['{', '}'] , ['[',']']
    for old, new in izip(olds, news):        
        set_str = set_str.replace(old, new)

    return set(literal_eval(set_str))
like image 143
Jamie Bull Avatar answered Sep 25 '22 13:09

Jamie Bull