pax_global_header00006660000000000000000000000064141017030420014502gustar00rootroot0000000000000052 comment=e86c4413542031268832d0fa97a1b71b8f0bc4ec gast-0.5.2/000077500000000000000000000000001410170304200124445ustar00rootroot00000000000000gast-0.5.2/.github/000077500000000000000000000000001410170304200140045ustar00rootroot00000000000000gast-0.5.2/.github/workflows/000077500000000000000000000000001410170304200160415ustar00rootroot00000000000000gast-0.5.2/.github/workflows/core.yml000066400000000000000000000012331410170304200175130ustar00rootroot00000000000000name: core on: push: branches: - master pull_request: branches: - master jobs: build: runs-on: ubuntu-18.04 strategy: matrix: python-version: [3.6, 3.8, 3.9, 3.10-dev] steps: - uses: actions/checkout@v2 - name: Setup Python ${{ matrix.python-version }} uses: actions/setup-python@v2 with: python-version: ${{ matrix.python-version }} - name: Install dependencies run: | python -m pip install --upgrade pip pip install astunparse pytest - name: Setup run: | python setup.py install - name: Testing sequential run: | pytest gast-0.5.2/.gitignore000066400000000000000000000001771410170304200144410ustar00rootroot00000000000000# Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] *$py.class # Distribution / packaging build/ dist/ *.egg-info/ gast-0.5.2/LICENSE000066400000000000000000000027221410170304200134540ustar00rootroot00000000000000Copyright (c) 2016, Serge Guelton All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. Neither the name of HPCProject, Serge Guelton nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. gast-0.5.2/MANIFEST.in000066400000000000000000000000551410170304200142020ustar00rootroot00000000000000recursive-include tests *.py include LICENSE gast-0.5.2/README.rst000066400000000000000000000224021410170304200141330ustar00rootroot00000000000000GAST, daou naer! ================ A generic AST to represent Python2 and Python3's Abstract Syntax Tree(AST). GAST provides a compatibility layer between the AST of various Python versions, as produced by ``ast.parse`` from the standard ``ast`` module. Basic Usage ----------- .. code:: python >>> import ast, gast >>> code = open('file.py').read() >>> tree = ast.parse(code) >>> gtree = gast.ast_to_gast(tree) >>> ... # process gtree >>> tree = gast.gast_to_ast(gtree) >>> ... # do stuff specific to tree API --- ``gast`` provides the same API as the ``ast`` module. All functions and classes from the ``ast`` module are also available in the ``gast`` module, and work accordingly on the ``gast`` tree. Three notable exceptions: 1. ``gast.parse`` directly produces a GAST node. It's equivalent to running ``gast.ast_to_gast`` on the output of ``ast.parse``. 2. ``gast.dump`` dumps the ``gast`` common representation, not the original one. 3. ``gast.gast_to_ast`` and ``gast.ast_to_gast`` can be used to convert from one ast to the other, back and forth. Version Compatibility --------------------- GAST is tested using ``tox`` and Travis on the following Python versions: - 2.7 - 3.4 - 3.5 - 3.6 - 3.7 - 3.8 - 3.9 - 3.10 AST Changes ----------- Python3 ******* The AST used by GAST is the same as the one used in Python3.9, with the notable exception of ``ast.arg`` being replaced by an ``ast.Name`` with an ``ast.Param`` context. For minor version before 3.9, please note that ``ExtSlice`` and ``Index`` are not used. For minor version before 3.8, please note that ``Ellipsis``, ``Num``, ``Str``, ``Bytes`` and ``NamedConstant`` are represented as ``Constant``. Python2 ******* To cope with Python3 features, several nodes from the Python2 AST are extended with some new attributes/children, or represented by different nodes - ``ModuleDef`` nodes have a ``type_ignores`` attribute. - ``FunctionDef`` nodes have a ``returns`` attribute and a ``type_comment`` attribute. - ``ClassDef`` nodes have a ``keywords`` attribute. - ``With``'s ``context_expr`` and ``optional_vars`` fields are hold in a ``withitem`` object. - ``For`` nodes have a ``type_comment`` attribute. - ``Raise``'s ``type``, ``inst`` and ``tback`` fields are hold in a single ``exc`` field, using the transformation ``raise E, V, T => raise E(V).with_traceback(T)``. - ``TryExcept`` and ``TryFinally`` nodes are merged in the ``Try`` node. - ``arguments`` nodes have a ``kwonlyargs`` and ``kw_defaults`` attributes. - ``Call`` nodes loose their ``starargs`` attribute, replaced by an argument wrapped in a ``Starred`` node. They also loose their ``kwargs`` attribute, wrapped in a ``keyword`` node with the identifier set to ``None``, as done in Python3. - ``comprehension`` nodes have an ``async`` attribute (that is always set to 0). - ``Ellipsis``, ``Num`` and ``Str`` nodes are represented as ``Constant``. - ``Subscript`` which don't have any ``Slice`` node as ``slice`` attribute (and ``Ellipsis`` are not ``Slice`` nodes) no longer hold an ``ExtSlice`` but an ``Index(Tuple(...))`` instead. Pit Falls ********* - In Python3, ``None``, ``True`` and ``False`` are parsed as ``Constant`` while they are parsed as regular ``Name`` in Python2. ASDL **** .. code:: -- ASDL's six builtin types are identifier, int, string, bytes, object, singleton module Python { mod = Module(stmt* body, type_ignore *type_ignores) | Interactive(stmt* body) | Expression(expr body) | FunctionType(expr* argtypes, expr returns) -- not really an actual node but useful in Jython's typesystem. | Suite(stmt* body) stmt = FunctionDef(identifier name, arguments args, stmt* body, expr* decorator_list, expr? returns, string? type_comment) | AsyncFunctionDef(identifier name, arguments args, stmt* body, expr* decorator_list, expr? returns, string? type_comment) | ClassDef(identifier name, expr* bases, keyword* keywords, stmt* body, expr* decorator_list) | Return(expr? value) | Delete(expr* targets) | Assign(expr* targets, expr value, string? type_comment) | AugAssign(expr target, operator op, expr value) -- 'simple' indicates that we annotate simple name without parens | AnnAssign(expr target, expr annotation, expr? value, int simple) -- not sure if bool is allowed, can always use int | Print(expr? dest, expr* values, bool nl) -- use 'orelse' because else is a keyword in target languages | For(expr target, expr iter, stmt* body, stmt* orelse, string? type_comment) | AsyncFor(expr target, expr iter, stmt* body, stmt* orelse, string? type_comment) | While(expr test, stmt* body, stmt* orelse) | If(expr test, stmt* body, stmt* orelse) | With(withitem* items, stmt* body, string? type_comment) | AsyncWith(withitem* items, stmt* body, string? type_comment) | Raise(expr? exc, expr? cause) | Try(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody) | Assert(expr test, expr? msg) | Import(alias* names) | ImportFrom(identifier? module, alias* names, int? level) -- Doesn't capture requirement that locals must be -- defined if globals is -- still supports use as a function! | Exec(expr body, expr? globals, expr? locals) | Global(identifier* names) | Nonlocal(identifier* names) | Expr(expr value) | Pass | Break | Continue -- XXX Jython will be different -- col_offset is the byte offset in the utf8 string the parser uses attributes (int lineno, int col_offset) -- BoolOp() can use left & right? expr = BoolOp(boolop op, expr* values) | NamedExpr(expr target, expr value) | BinOp(expr left, operator op, expr right) | UnaryOp(unaryop op, expr operand) | Lambda(arguments args, expr body) | IfExp(expr test, expr body, expr orelse) | Dict(expr* keys, expr* values) | Set(expr* elts) | ListComp(expr elt, comprehension* generators) | SetComp(expr elt, comprehension* generators) | DictComp(expr key, expr value, comprehension* generators) | GeneratorExp(expr elt, comprehension* generators) -- the grammar constrains where yield expressions can occur | Await(expr value) | Yield(expr? value) | YieldFrom(expr value) -- need sequences for compare to distinguish between -- x < 4 < 3 and (x < 4) < 3 | Compare(expr left, cmpop* ops, expr* comparators) | Call(expr func, expr* args, keyword* keywords) | Repr(expr value) | FormattedValue(expr value, int? conversion, expr? format_spec) | JoinedStr(expr* values) | Constant(constant value, string? kind) -- the following expression can appear in assignment context | Attribute(expr value, identifier attr, expr_context ctx) | Subscript(expr value, slice slice, expr_context ctx) | Starred(expr value, expr_context ctx) | Name(identifier id, expr_context ctx, expr? annotation, string? type_comment) | List(expr* elts, expr_context ctx) | Tuple(expr* elts, expr_context ctx) -- col_offset is the byte offset in the utf8 string the parser uses attributes (int lineno, int col_offset) expr_context = Load | Store | Del | AugLoad | AugStore | Param slice = Slice(expr? lower, expr? upper, expr? step) | ExtSlice(slice* dims) | Index(expr value) boolop = And | Or operator = Add | Sub | Mult | MatMult | Div | Mod | Pow | LShift | RShift | BitOr | BitXor | BitAnd | FloorDiv unaryop = Invert | Not | UAdd | USub cmpop = Eq | NotEq | Lt | LtE | Gt | GtE | Is | IsNot | In | NotIn comprehension = (expr target, expr iter, expr* ifs, int is_async) excepthandler = ExceptHandler(expr? type, expr? name, stmt* body) attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset) arguments = (expr* args, expr* posonlyargs, expr? vararg, expr* kwonlyargs, expr* kw_defaults, expr? kwarg, expr* defaults) -- keyword arguments supplied to call (NULL identifier for **kwargs) keyword = (identifier? arg, expr value) -- import name with optional 'as' alias. alias = (identifier name, identifier? asname) attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset) withitem = (expr context_expr, expr? optional_vars) type_ignore = TypeIgnore(int lineno, string tag) } gast-0.5.2/gast/000077500000000000000000000000001410170304200134025ustar00rootroot00000000000000gast-0.5.2/gast/__init__.py000066400000000000000000000001241410170304200155100ustar00rootroot00000000000000from .gast import * from ast import NodeVisitor, NodeTransformer, iter_fields, dump gast-0.5.2/gast/ast2.py000066400000000000000000000321101410170304200146220ustar00rootroot00000000000000from astn import AstToGAst, GAstToAst import ast import gast class Ast2ToGAst(AstToGAst): # mod def visit_Module(self, node): new_node = gast.Module( self._visit(node.body), [] # type_ignores ) return new_node # stmt def visit_FunctionDef(self, node): new_node = gast.FunctionDef( self._visit(node.name), self._visit(node.args), self._visit(node.body), self._visit(node.decorator_list), None, # returns None, # type_comment ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_ClassDef(self, node): new_node = gast.ClassDef( self._visit(node.name), self._visit(node.bases), [], # keywords self._visit(node.body), self._visit(node.decorator_list), ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_Assign(self, node): new_node = gast.Assign( self._visit(node.targets), self._visit(node.value), None, # type_comment ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_For(self, node): new_node = gast.For( self._visit(node.target), self._visit(node.iter), self._visit(node.body), self._visit(node.orelse), [] # type_comment ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_With(self, node): new_node = gast.With( [gast.withitem( self._visit(node.context_expr), self._visit(node.optional_vars) )], self._visit(node.body), None, # type_comment ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_Raise(self, node): ntype = self._visit(node.type) ninst = self._visit(node.inst) ntback = self._visit(node.tback) what = ntype if ninst is not None: what = gast.Call(ntype, [ninst], []) gast.copy_location(what, node) what.end_lineno = what.end_col_offset = None if ntback is not None: attr = gast.Attribute(what, 'with_traceback', gast.Load()) gast.copy_location(attr, node) attr.end_lineno = attr.end_col_offset = None what = gast.Call( attr, [ntback], [] ) gast.copy_location(what, node) what.end_lineno = what.end_col_offset = None new_node = gast.Raise(what, None) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_TryExcept(self, node): new_node = gast.Try( self._visit(node.body), self._visit(node.handlers), self._visit(node.orelse), [] # finalbody ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_TryFinally(self, node): new_node = gast.Try( self._visit(node.body), [], # handlers [], # orelse self._visit(node.finalbody) ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node # expr def visit_Name(self, node): new_node = gast.Name( self._visit(node.id), self._visit(node.ctx), None, None, ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_Num(self, node): new_node = gast.Constant( node.n, None, ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_Subscript(self, node): new_slice = self._visit(node.slice) new_node = gast.Subscript( self._visit(node.value), new_slice, self._visit(node.ctx), ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_Ellipsis(self, node): new_node = gast.Constant( Ellipsis, None, ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_Index(self, node): return self._visit(node.value) def visit_ExtSlice(self, node): new_dims = self._visit(node.dims) new_node = gast.Tuple(new_dims, gast.Load()) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_Str(self, node): new_node = gast.Constant( node.s, None, ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_Call(self, node): if node.starargs: star = gast.Starred(self._visit(node.starargs), gast.Load()) gast.copy_location(star, node) star.end_lineno = star.end_col_offset = None starred = [star] else: starred = [] if node.kwargs: kwargs = [gast.keyword(None, self._visit(node.kwargs))] else: kwargs = [] new_node = gast.Call( self._visit(node.func), self._visit(node.args) + starred, self._visit(node.keywords) + kwargs, ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_comprehension(self, node): new_node = gast.comprehension( target=self._visit(node.target), iter=self._visit(node.iter), ifs=self._visit(node.ifs), is_async=0, ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node # arguments def visit_arguments(self, node): # missing locations for vararg and kwarg set at function level if node.vararg: vararg = ast.Name(node.vararg, ast.Param()) else: vararg = None if node.kwarg: kwarg = ast.Name(node.kwarg, ast.Param()) else: kwarg = None if node.vararg: vararg = ast.Name(node.vararg, ast.Param()) else: vararg = None new_node = gast.arguments( self._visit(node.args), [], # posonlyargs self._visit(vararg), [], # kwonlyargs [], # kw_defaults self._visit(kwarg), self._visit(node.defaults), ) return new_node def visit_alias(self, node): new_node = gast.alias( self._visit(node.name), self._visit(node.asname), ) new_node.lineno = new_node.col_offset = None new_node.end_lineno = new_node.end_col_offset = None return new_node class GAstToAst2(GAstToAst): # mod def visit_Module(self, node): new_node = ast.Module(self._visit(node.body)) return new_node # stmt def visit_FunctionDef(self, node): new_node = ast.FunctionDef( self._visit(node.name), self._visit(node.args), self._visit(node.body), self._visit(node.decorator_list), ) # because node.args doesn't have any location to copy from if node.args.vararg: ast.copy_location(node.args.vararg, node) if node.args.kwarg: ast.copy_location(node.args.kwarg, node) ast.copy_location(new_node, node) return new_node def visit_ClassDef(self, node): new_node = ast.ClassDef( self._visit(node.name), self._visit(node.bases), self._visit(node.body), self._visit(node.decorator_list), ) ast.copy_location(new_node, node) return new_node def visit_Assign(self, node): new_node = ast.Assign( self._visit(node.targets), self._visit(node.value), ) ast.copy_location(new_node, node) return new_node def visit_For(self, node): new_node = ast.For( self._visit(node.target), self._visit(node.iter), self._visit(node.body), self._visit(node.orelse), ) ast.copy_location(new_node, node) return new_node def visit_With(self, node): new_node = ast.With( self._visit(node.items[0].context_expr), self._visit(node.items[0].optional_vars), self._visit(node.body) ) ast.copy_location(new_node, node) return new_node def visit_Raise(self, node): if isinstance(node.exc, gast.Call) and \ isinstance(node.exc.func, gast.Attribute) and \ node.exc.func.attr == 'with_traceback': raised = self._visit(node.exc.func.value) traceback = self._visit(node.exc.args[0]) else: raised = self._visit(node.exc) traceback = None new_node = ast.Raise(raised, None, traceback) ast.copy_location(new_node, node) return new_node def visit_Try(self, node): if node.finalbody: new_node = ast.TryFinally( self._visit(node.body), self._visit(node.finalbody) ) else: new_node = ast.TryExcept( self._visit(node.body), self._visit(node.handlers), self._visit(node.orelse), ) ast.copy_location(new_node, node) return new_node # expr def visit_Name(self, node): new_node = ast.Name( self._visit(node.id), self._visit(node.ctx), ) ast.copy_location(new_node, node) return new_node def visit_Constant(self, node): if isinstance(node.value, (bool, int, long, float, complex)): new_node = ast.Num(node.value) elif node.value is Ellipsis: new_node = ast.Ellipsis() else: new_node = ast.Str(node.value) ast.copy_location(new_node, node) return new_node def visit_Subscript(self, node): def adjust_slice(s): if isinstance(s, (ast.Slice, ast.Ellipsis)): return s else: return ast.Index(s) if isinstance(node.slice, gast.Tuple): new_slice = ast.ExtSlice([adjust_slice(self._visit(elt)) for elt in node.slice.elts]) else: new_slice = adjust_slice(self._visit(node.slice)) ast.copy_location(new_slice, node.slice) new_node = ast.Subscript( self._visit(node.value), new_slice, self._visit(node.ctx), ) ast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_Call(self, node): if node.args and isinstance(node.args[-1], gast.Starred): args = node.args[:-1] starargs = node.args[-1].value else: args = node.args starargs = None if node.keywords and node.keywords[-1].arg is None: keywords = node.keywords[:-1] kwargs = node.keywords[-1].value else: keywords = node.keywords kwargs = None new_node = ast.Call( self._visit(node.func), self._visit(args), self._visit(keywords), self._visit(starargs), self._visit(kwargs), ) ast.copy_location(new_node, node) return new_node def visit_arg(self, node): new_node = ast.Name(node.arg, ast.Param()) ast.copy_location(new_node, node) return new_node # arguments def visit_arguments(self, node): vararg = node.vararg and node.vararg.id kwarg = node.kwarg and node.kwarg.id new_node = ast.arguments( self._visit(node.args), self._visit(vararg), self._visit(kwarg), self._visit(node.defaults), ) return new_node def visit_alias(self, node): new_node = ast.alias( self._visit(node.name), self._visit(node.asname) ) return new_node def ast_to_gast(node): return Ast2ToGAst().visit(node) def gast_to_ast(node): return GAstToAst2().visit(node) gast-0.5.2/gast/ast3.py000066400000000000000000000352011410170304200146270ustar00rootroot00000000000000from gast.astn import AstToGAst, GAstToAst import gast import ast import sys class Ast3ToGAst(AstToGAst): if sys.version_info.minor < 10: def visit_alias(self, node): new_node = gast.alias( self._visit(node.name), self._visit(node.asname), ) new_node.lineno = new_node.col_offset = None new_node.end_lineno = new_node.end_col_offset = None return new_node if sys.version_info.minor < 9: def visit_ExtSlice(self, node): new_node = gast.Tuple(self._visit(node.dims), gast.Load()) return gast.copy_location(new_node, node) def visit_Index(self, node): return self._visit(node.value) def visit_Assign(self, node): new_node = gast.Assign( self._visit(node.targets), self._visit(node.value), None, # type_comment ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node if sys.version_info.minor < 8: def visit_Module(self, node): new_node = gast.Module( self._visit(node.body), [] # type_ignores ) return new_node def visit_Num(self, node): new_node = gast.Constant( node.n, None, ) return gast.copy_location(new_node, node) def visit_Ellipsis(self, node): new_node = gast.Constant( Ellipsis, None, ) gast.copy_location(new_node, node) new_node.end_lineno = new_node.end_col_offset = None return new_node def visit_Str(self, node): new_node = gast.Constant( node.s, None, ) return gast.copy_location(new_node, node) def visit_Bytes(self, node): new_node = gast.Constant( node.s, None, ) return gast.copy_location(new_node, node) def visit_FunctionDef(self, node): new_node = gast.FunctionDef( self._visit(node.name), self._visit(node.args), self._visit(node.body), self._visit(node.decorator_list), self._visit(node.returns), None, # type_comment ) return gast.copy_location(new_node, node) def visit_AsyncFunctionDef(self, node): new_node = gast.AsyncFunctionDef( self._visit(node.name), self._visit(node.args), self._visit(node.body), self._visit(node.decorator_list), self._visit(node.returns), None, # type_comment ) return gast.copy_location(new_node, node) def visit_For(self, node): new_node = gast.For( self._visit(node.target), self._visit(node.iter), self._visit(node.body), self._visit(node.orelse), None, # type_comment ) return gast.copy_location(new_node, node) def visit_AsyncFor(self, node): new_node = gast.AsyncFor( self._visit(node.target), self._visit(node.iter), self._visit(node.body), self._visit(node.orelse), None, # type_comment ) return gast.copy_location(new_node, node) def visit_With(self, node): new_node = gast.With( self._visit(node.items), self._visit(node.body), None, # type_comment ) return gast.copy_location(new_node, node) def visit_AsyncWith(self, node): new_node = gast.AsyncWith( self._visit(node.items), self._visit(node.body), None, # type_comment ) return gast.copy_location(new_node, node) def visit_Call(self, node): if sys.version_info.minor < 5: if node.starargs: star = gast.Starred(self._visit(node.starargs), gast.Load()) gast.copy_location(star, node) starred = [star] else: starred = [] if node.kwargs: kw = gast.keyword(None, self._visit(node.kwargs)) gast.copy_location(kw, node.kwargs) kwargs = [kw] else: kwargs = [] else: starred = kwargs = [] new_node = gast.Call( self._visit(node.func), self._visit(node.args) + starred, self._visit(node.keywords) + kwargs, ) return gast.copy_location(new_node, node) def visit_NameConstant(self, node): if node.value is None: new_node = gast.Constant(None, None) elif node.value is True: new_node = gast.Constant(True, None) elif node.value is False: new_node = gast.Constant(False, None) return gast.copy_location(new_node, node) def visit_arguments(self, node): new_node = gast.arguments( self._visit(node.args), [], # posonlyargs self._visit(node.vararg), self._visit(node.kwonlyargs), self._visit(node.kw_defaults), self._visit(node.kwarg), self._visit(node.defaults), ) return gast.copy_location(new_node, node) def visit_Name(self, node): new_node = gast.Name( node.id, # micro-optimization here, don't call self._visit self._visit(node.ctx), None, None, ) return ast.copy_location(new_node, node) def visit_arg(self, node): if sys.version_info.minor < 8: extra_arg = None else: extra_arg = self._visit(node.type_comment) new_node = gast.Name( node.arg, # micro-optimization here, don't call self._visit gast.Param(), self._visit(node.annotation), extra_arg # type_comment ) return ast.copy_location(new_node, node) def visit_ExceptHandler(self, node): if node.name: new_node = gast.ExceptHandler( self._visit(node.type), gast.Name(node.name, gast.Store(), None, None), self._visit(node.body)) return ast.copy_location(new_node, node) else: return self.generic_visit(node) if sys.version_info.minor < 6: def visit_comprehension(self, node): new_node = gast.comprehension( target=self._visit(node.target), iter=self._visit(node.iter), ifs=self._visit(node.ifs), is_async=0, ) return ast.copy_location(new_node, node) class GAstToAst3(GAstToAst): if sys.version_info.minor < 10: def visit_alias(self, node): new_node = ast.alias( self._visit(node.name), self._visit(node.asname) ) return new_node if sys.version_info.minor < 9: def visit_Subscript(self, node): def adjust_slice(s): if isinstance(s, ast.Slice): return s else: return ast.Index(s) if isinstance(node.slice, gast.Tuple): if any(isinstance(elt, gast.slice) for elt in node.slice.elts): new_slice = ast.ExtSlice( [adjust_slice(x) for x in self._visit(node.slice.elts)]) else: value = ast.Tuple(self._visit(node.slice.elts), ast.Load()) ast.copy_location(value, node.slice) new_slice = ast.Index(value) else: new_slice = adjust_slice(self._visit(node.slice)) ast.copy_location(new_slice, node.slice) new_node = ast.Subscript( self._visit(node.value), new_slice, self._visit(node.ctx), ) return ast.copy_location(new_node, node) def visit_Assign(self, node): new_node = ast.Assign( self._visit(node.targets), self._visit(node.value), ) return ast.copy_location(new_node, node) if sys.version_info.minor < 8: def visit_Module(self, node): new_node = ast.Module(self._visit(node.body)) return new_node def visit_Constant(self, node): if node.value is None: new_node = ast.NameConstant(node.value) elif node.value is Ellipsis: new_node = ast.Ellipsis() elif isinstance(node.value, bool): new_node = ast.NameConstant(node.value) elif isinstance(node.value, (int, float, complex)): new_node = ast.Num(node.value) elif isinstance(node.value, str): new_node = ast.Str(node.value) else: new_node = ast.Bytes(node.value) return ast.copy_location(new_node, node) def _make_arg(self, node): if node is None: return None if sys.version_info.minor < 8: extra_args = tuple() else: extra_args = self._visit(node.type_comment), new_node = ast.arg( self._visit(node.id), self._visit(node.annotation), *extra_args ) return ast.copy_location(new_node, node) def visit_Name(self, node): new_node = ast.Name( self._visit(node.id), self._visit(node.ctx), ) return ast.copy_location(new_node, node) def visit_ExceptHandler(self, node): if node.name: new_node = ast.ExceptHandler( self._visit(node.type), node.name.id, self._visit(node.body)) return ast.copy_location(new_node, node) else: return self.generic_visit(node) if sys.version_info.minor < 5: def visit_Call(self, node): if node.args and isinstance(node.args[-1], gast.Starred): args = node.args[:-1] starargs = node.args[-1].value else: args = node.args starargs = None if node.keywords and node.keywords[-1].arg is None: keywords = node.keywords[:-1] kwargs = node.keywords[-1].value else: keywords = node.keywords kwargs = None new_node = ast.Call( self._visit(node.func), self._visit(args), self._visit(keywords), self._visit(starargs), self._visit(kwargs), ) return ast.copy_location(new_node, node) def visit_ClassDef(self, node): self.generic_visit(node) new_node = ast.ClassDef( name=self._visit(node.name), bases=self._visit(node.bases), keywords=self._visit(node.keywords), body=self._visit(node.body), decorator_list=self._visit(node.decorator_list), starargs=None, kwargs=None, ) return ast.copy_location(new_node, node) elif sys.version_info.minor < 8: def visit_FunctionDef(self, node): new_node = ast.FunctionDef( self._visit(node.name), self._visit(node.args), self._visit(node.body), self._visit(node.decorator_list), self._visit(node.returns), ) return ast.copy_location(new_node, node) def visit_AsyncFunctionDef(self, node): new_node = ast.AsyncFunctionDef( self._visit(node.name), self._visit(node.args), self._visit(node.body), self._visit(node.decorator_list), self._visit(node.returns), ) return ast.copy_location(new_node, node) def visit_For(self, node): new_node = ast.For( self._visit(node.target), self._visit(node.iter), self._visit(node.body), self._visit(node.orelse), ) return ast.copy_location(new_node, node) def visit_AsyncFor(self, node): new_node = ast.AsyncFor( self._visit(node.target), self._visit(node.iter), self._visit(node.body), self._visit(node.orelse), None, # type_comment ) return ast.copy_location(new_node, node) def visit_With(self, node): new_node = ast.With( self._visit(node.items), self._visit(node.body), ) return ast.copy_location(new_node, node) def visit_AsyncWith(self, node): new_node = ast.AsyncWith( self._visit(node.items), self._visit(node.body), ) return ast.copy_location(new_node, node) def visit_Call(self, node): new_node = ast.Call( self._visit(node.func), self._visit(node.args), self._visit(node.keywords), ) return ast.copy_location(new_node, node) def visit_arguments(self, node): extra_args = [self._make_arg(node.vararg), [self._make_arg(n) for n in node.kwonlyargs], self._visit(node.kw_defaults), self._make_arg(node.kwarg), self._visit(node.defaults), ] if sys.version_info.minor >= 8: new_node = ast.arguments( [self._make_arg(arg) for arg in node.posonlyargs], [self._make_arg(n) for n in node.args], *extra_args ) else: new_node = ast.arguments( [self._make_arg(n) for n in node.args], *extra_args ) return new_node def ast_to_gast(node): return Ast3ToGAst().visit(node) def gast_to_ast(node): return GAstToAst3().visit(node) gast-0.5.2/gast/astn.py000066400000000000000000000020411410170304200147160ustar00rootroot00000000000000import ast import gast def _generate_translators(to): class Translator(ast.NodeTransformer): def _visit(self, node): if isinstance(node, ast.AST): return self.visit(node) elif isinstance(node, list): return [self._visit(n) for n in node] else: return node def generic_visit(self, node): cls = type(node).__name__ try: new_node = getattr(to, cls)() except AttributeError: # handle nodes that are not part of the AST return for field in node._fields: setattr(new_node, field, self._visit(getattr(node, field))) for attr in node._attributes: try: setattr(new_node, attr, getattr(node, attr)) except AttributeError: pass return new_node return Translator AstToGAst = _generate_translators(gast) GAstToAst = _generate_translators(ast) gast-0.5.2/gast/gast.py000066400000000000000000000360271410170304200147220ustar00rootroot00000000000000import sys as _sys import ast as _ast from ast import boolop, cmpop, excepthandler, expr, expr_context, operator from ast import slice, stmt, unaryop, mod, AST from ast import iter_child_nodes, walk try: from ast import TypeIgnore except ImportError: class TypeIgnore(AST): pass def _make_node(Name, Fields, Attributes, Bases): NBFields = len(Fields) def create_node(self, *args, **kwargs): if args: if len(args) + len([k for k in kwargs if k in Fields]) != NBFields: raise TypeError( "{} constructor takes either 0 or {} mandatory arguments". format(Name, NBFields)) for argname, argval in zip(Fields, args): setattr(self, argname, argval) if kwargs: for argname, argval in kwargs.items(): setattr(self, argname, argval) setattr(_sys.modules[__name__], Name, type(Name, Bases, {'__init__': create_node, '_fields': Fields, '_attributes': Attributes})) _nodes = ( # mod ('Module', (('body', 'type_ignores'), (), (mod,))), ('Interactive', (('body',), (), (mod,))), ('Expression', (('body',), (), (mod,))), ('FunctionType', (('argtypes', 'returns'), (), (mod,))), ('Suite', (('body',), (), (mod,))), # stmt ('FunctionDef', (('name', 'args', 'body', 'decorator_list', 'returns', 'type_comment'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('AsyncFunctionDef', (('name', 'args', 'body', 'decorator_list', 'returns', 'type_comment'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('ClassDef', (('name', 'bases', 'keywords', 'body', 'decorator_list',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Return', (('value',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Delete', (('targets',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Assign', (('targets', 'value', 'type_comment'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('AugAssign', (('target', 'op', 'value',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('AnnAssign', (('target', 'annotation', 'value', 'simple',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Print', (('dest', 'values', 'nl',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('For', (('target', 'iter', 'body', 'orelse', 'type_comment'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('AsyncFor', (('target', 'iter', 'body', 'orelse', 'type_comment'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('While', (('test', 'body', 'orelse',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('If', (('test', 'body', 'orelse',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('With', (('items', 'body', 'type_comment'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('AsyncWith', (('items', 'body', 'type_comment'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Raise', (('exc', 'cause',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Try', (('body', 'handlers', 'orelse', 'finalbody',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Assert', (('test', 'msg',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Import', (('names',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('ImportFrom', (('module', 'names', 'level',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Exec', (('body', 'globals', 'locals',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Global', (('names',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Nonlocal', (('names',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Expr', (('value',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Pass', ((), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Break', ((), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), ('Continue', ((), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (stmt,))), # expr ('BoolOp', (('op', 'values',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('NamedExpr', (('target', 'value',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('BinOp', (('left', 'op', 'right',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('UnaryOp', (('op', 'operand',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Lambda', (('args', 'body',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('IfExp', (('test', 'body', 'orelse',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Dict', (('keys', 'values',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Set', (('elts',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('ListComp', (('elt', 'generators',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('SetComp', (('elt', 'generators',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('DictComp', (('key', 'value', 'generators',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('GeneratorExp', (('elt', 'generators',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Await', (('value',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Yield', (('value',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('YieldFrom', (('value',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Compare', (('left', 'ops', 'comparators',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Call', (('func', 'args', 'keywords',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Repr', (('value',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('FormattedValue', (('value', 'conversion', 'format_spec',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('JoinedStr', (('values',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Constant', (('value', 'kind'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Attribute', (('value', 'attr', 'ctx',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Subscript', (('value', 'slice', 'ctx',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Starred', (('value', 'ctx',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Name', (('id', 'ctx', 'annotation', 'type_comment'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('List', (('elts', 'ctx',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), ('Tuple', (('elts', 'ctx',), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (expr,))), # expr_context ('Load', ((), (), (expr_context,))), ('Store', ((), (), (expr_context,))), ('Del', ((), (), (expr_context,))), ('AugLoad', ((), (), (expr_context,))), ('AugStore', ((), (), (expr_context,))), ('Param', ((), (), (expr_context,))), # slice ('Slice', (('lower', 'upper', 'step'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), (slice,))), # boolop ('And', ((), (), (boolop,))), ('Or', ((), (), (boolop,))), # operator ('Add', ((), (), (operator,))), ('Sub', ((), (), (operator,))), ('Mult', ((), (), (operator,))), ('MatMult', ((), (), (operator,))), ('Div', ((), (), (operator,))), ('Mod', ((), (), (operator,))), ('Pow', ((), (), (operator,))), ('LShift', ((), (), (operator,))), ('RShift', ((), (), (operator,))), ('BitOr', ((), (), (operator,))), ('BitXor', ((), (), (operator,))), ('BitAnd', ((), (), (operator,))), ('FloorDiv', ((), (), (operator,))), # unaryop ('Invert', ((), (), (unaryop, AST,))), ('Not', ((), (), (unaryop, AST,))), ('UAdd', ((), (), (unaryop, AST,))), ('USub', ((), (), (unaryop, AST,))), # cmpop ('Eq', ((), (), (cmpop,))), ('NotEq', ((), (), (cmpop,))), ('Lt', ((), (), (cmpop,))), ('LtE', ((), (), (cmpop,))), ('Gt', ((), (), (cmpop,))), ('GtE', ((), (), (cmpop,))), ('Is', ((), (), (cmpop,))), ('IsNot', ((), (), (cmpop,))), ('In', ((), (), (cmpop,))), ('NotIn', ((), (), (cmpop,))), # comprehension ('comprehension', (('target', 'iter', 'ifs', 'is_async'), (), (AST,))), # excepthandler ('ExceptHandler', (('type', 'name', 'body'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset'), (excepthandler,))), # arguments ('arguments', (('args', 'posonlyargs', 'vararg', 'kwonlyargs', 'kw_defaults', 'kwarg', 'defaults'), (), (AST,))), # keyword ('keyword', (('arg', 'value'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset'), (AST,))), # alias ('alias', (('name', 'asname'), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset'), (AST,))), # withitem ('withitem', (('context_expr', 'optional_vars'), (), (AST,))), # type_ignore ('type_ignore', ((), ('lineno', 'tag'), (TypeIgnore,))), ) for name, descr in _nodes: _make_node(name, *descr) if _sys.version_info.major == 2: from .ast2 import ast_to_gast, gast_to_ast if _sys.version_info.major == 3: from .ast3 import ast_to_gast, gast_to_ast def parse(*args, **kwargs): return ast_to_gast(_ast.parse(*args, **kwargs)) def literal_eval(node_or_string): if isinstance(node_or_string, AST): node_or_string = gast_to_ast(node_or_string) return _ast.literal_eval(node_or_string) def get_docstring(node, clean=True): if not isinstance(node, (FunctionDef, ClassDef, Module)): raise TypeError("%r can't have docstrings" % node.__class__.__name__) if node.body and isinstance(node.body[0], Expr) and \ isinstance(node.body[0].value, Constant): if clean: import inspect holder = node.body[0].value return inspect.cleandoc(getattr(holder, holder._fields[0])) return node.body[0].value.s # the following are directly imported from python3.8's Lib/ast.py # def copy_location(new_node, old_node): """ Copy source location (`lineno`, `col_offset`, `end_lineno`, and `end_col_offset` attributes) from *old_node* to *new_node* if possible, and return *new_node*. """ for attr in 'lineno', 'col_offset', 'end_lineno', 'end_col_offset': if attr in old_node._attributes and attr in new_node._attributes \ and hasattr(old_node, attr): setattr(new_node, attr, getattr(old_node, attr)) return new_node def fix_missing_locations(node): """ When you compile a node tree with compile(), the compiler expects lineno and col_offset attributes for every node that supports them. This is rather tedious to fill in for generated nodes, so this helper adds these attributes recursively where not already set, by setting them to the values of the parent node. It works recursively starting at *node*. """ def _fix(node, lineno, col_offset, end_lineno, end_col_offset): if 'lineno' in node._attributes: if not hasattr(node, 'lineno'): node.lineno = lineno else: lineno = node.lineno if 'end_lineno' in node._attributes: if not hasattr(node, 'end_lineno'): node.end_lineno = end_lineno else: end_lineno = node.end_lineno if 'col_offset' in node._attributes: if not hasattr(node, 'col_offset'): node.col_offset = col_offset else: col_offset = node.col_offset if 'end_col_offset' in node._attributes: if not hasattr(node, 'end_col_offset'): node.end_col_offset = end_col_offset else: end_col_offset = node.end_col_offset for child in iter_child_nodes(node): _fix(child, lineno, col_offset, end_lineno, end_col_offset) _fix(node, 1, 0, 1, 0) return node def increment_lineno(node, n=1): """ Increment the line number and end line number of each node in the tree starting at *node* by *n*. This is useful to "move code" to a different location in a file. """ for child in walk(node): if 'lineno' in child._attributes: child.lineno = (getattr(child, 'lineno', 0) or 0) + n if 'end_lineno' in child._attributes: child.end_lineno = (getattr(child, 'end_lineno', 0) or 0) + n return node gast-0.5.2/setup.py000066400000000000000000000034201410170304200141550ustar00rootroot00000000000000# Use the newer `setuptools.setup()`, if available. try: from setuptools import setup kw = { 'test_suite': 'tests', 'tests_require': ['astunparse'], } except ImportError: from distutils.core import setup kw = {} setup(name='gast', # gast, daou naer! version='0.5.2', packages=['gast'], description='Python AST that abstracts the underlying Python version', long_description=''' A generic AST to represent Python2 and Python3's Abstract Syntax Tree(AST). GAST provides a compatibility layer between the AST of various Python versions, as produced by ``ast.parse`` from the standard ``ast`` module.''', author='serge-sans-paille', author_email='serge.guelton@telecom-bretagne.eu', url='https://github.com/serge-sans-paille/gast/', license="BSD 3-Clause", classifiers=['Development Status :: 4 - Beta', 'Environment :: Console', 'Intended Audience :: Developers', 'License :: OSI Approved :: BSD License', 'Natural Language :: English', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.8', 'Programming Language :: Python :: 3.9', 'Programming Language :: Python :: 3.10', ], python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*', **kw ) gast-0.5.2/tests/000077500000000000000000000000001410170304200136065ustar00rootroot00000000000000gast-0.5.2/tests/__init__.py000066400000000000000000000000561410170304200157200ustar00rootroot00000000000000"""Package containing unit tests for gast.""" gast-0.5.2/tests/test_api.py000066400000000000000000000103361410170304200157730ustar00rootroot00000000000000import unittest import ast import gast class APITestCase(unittest.TestCase): def test_literal_eval_string(self): code = "1, 3" self.assertEqual(ast.literal_eval(code), gast.literal_eval(code)) def test_literal_eval_code(self): code = "[1, 3]" tree = ast.parse(code, mode='eval') gtree = gast.parse(code, mode='eval') self.assertEqual(ast.literal_eval(tree), gast.literal_eval(gtree)) def test_parse(self): code = ''' def foo(x=1, *args, **kwargs): return x + y +len(args) + len(kwargs) ''' gast.parse(code) def test_dump(self): code = 'lambda x: x' tree = gast.parse(code, mode='eval') dump = gast.dump(tree) norm = ("Expression(body=Lambda(args=arguments(args=[Name(" "id='x', ctx=Param(), " "annotation=None, type_comment=None)], posonlyargs=[], " "vararg=None, kwonlyargs=[], kw_defaults=[], kwarg=None, " "defaults=[]), body=Name(id='x', ctx=Load(), " "annotation=None, type_comment=None)" "))") self.assertEqual(dump, norm) def test_walk(self): code = 'x + 1' tree = gast.parse(code, mode='eval') dump = gast.dump(tree) norm = ("Expression(body=BinOp(left=Name(id='x', ctx=Load(), " "annotation=None, type_comment=None), op=Add(), " "right=Constant(value=1, kind=None)))") self.assertEqual(dump, norm) self.assertEqual(len(list(gast.walk(tree))), 6) def test_iter_fields(self): tree = gast.Constant(value=1, kind=None) self.assertEqual({name for name, _ in gast.iter_fields(tree)}, {'value', 'kind'}) def test_iter_child_nodes(self): tree = gast.UnaryOp(gast.USub(), gast.Constant(value=1, kind=None)) self.assertEqual(len(list(gast.iter_fields(tree))), 2) def test_increment_lineno(self): tree = gast.Constant(value=1, kind=None) tree.lineno = 1 gast.increment_lineno(tree) self.assertEqual(tree.lineno, 2) def test_get_docstring(self): code = 'def foo(): "foo"' tree = gast.parse(code) func = tree.body[0] docs = gast.get_docstring(func) self.assertEqual(docs, "foo") def test_copy_location(self): tree = gast.Constant(value=1, kind=None) tree.lineno = 1 tree.col_offset = 2 node = gast.Constant(value=2, kind=None) gast.copy_location(node, tree) self.assertEqual(node.lineno, tree.lineno) self.assertEqual(node.col_offset, tree.col_offset) def test_fix_missing_locations(self): node = gast.Constant(value=6, kind=None) tree = gast.UnaryOp(gast.USub(), node) tree.lineno = 1 tree.col_offset = 2 gast.fix_missing_locations(tree) self.assertEqual(node.lineno, tree.lineno) self.assertEqual(node.col_offset, tree.col_offset) def test_NodeTransformer(self): node = gast.Constant(value=6, kind=None) tree = gast.UnaryOp(gast.USub(), node) class Trans(gast.NodeTransformer): def visit_Constant(self, node): node.value *= 2 return node tree = Trans().visit(tree) self.assertEqual(node.value, 12) def test_NodeVisitor(self): node = gast.Constant(value=6, kind=None) tree = gast.UnaryOp(gast.USub(), node) class Vis(gast.NodeTransformer): def __init__(self): self.state = [] def visit_Constant(self, node): self.state.append(node.value) vis = Vis() vis.visit(tree) self.assertEqual(vis.state, [6]) def test_NodeConstructor(self): node0 = gast.Name() load = gast.Load() node1 = gast.Name('id', load, None, None) node2 = gast.Name('id', load, None, type_comment=None) with self.assertRaises(TypeError): node1 = gast.Name('id') for field in gast.Name._fields: self.assertEqual(getattr(node1, field), getattr(node2, field)) if __name__ == '__main__': unittest.main() gast-0.5.2/tests/test_compat.py000066400000000000000000000413241410170304200165060ustar00rootroot00000000000000import unittest import gast import sys class CompatTestCase(unittest.TestCase): if sys.version_info.major == 2: def test_FunctionDef(self): code = 'def foo((x, y)): return x, y' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=" "[Tuple(elts=[Name(id='x', ctx=Store(), annotation=None, " "type_comment=None), Name(id='y', ctx=Store(), " "annotation=None, type_comment=None)], ctx=Store())], " "posonlyargs=[], vararg=None, " "kwonlyargs=[], kw_defaults=[], kwarg=None, defaults=[]), " "body=[Return(value=Tuple(elts=[Name(id='x', ctx=Load(), " "annotation=None, type_comment=None), " "Name(id='y', ctx=Load(), " "annotation=None, type_comment=None" ")], ctx=Load()))], decorator_list=" "[], returns=None, type_comment=None)], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) else: def test_ArgAnnotation(self): code = 'def foo(x:int): pass' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=" "[Name(id='x', ctx=Param(), annotation=Name" "(id='int', ctx=Load(), annotation=None, type_comment=None" "), type_comment=None)], posonlyargs=" "[], vararg=None, kwonlyargs=[], kw_defaults=[], kwarg=" "None, defaults=[]), body=[Pass()], decorator_list=[], " "returns=None, type_comment=None)], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_KeywordOnlyArgument(self): code = 'def foo(*, x=1): pass' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=" "[], posonlyargs=[], vararg=None, kwonlyargs=[Name" "(id='x', ctx=Param(), annotation=None, type_comment=None" ")], kw_defaults=[Constant(value=1, kind=None)], kwarg=" "None, defaults=[]), body=[Pass()], decorator_list=[], " "returns=None, type_comment=None)], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) if sys.version_info.minor >= 6: def test_FormattedValue(self): code = 'e = 1; f"{e}"' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[Assign(targets=[Name(id='e', ctx=Store()" ", annotation=None, type_comment=None" ")], value=Constant(value=1, kind=None), " "type_comment=None), Expr(value=" "JoinedStr(values=[FormattedValue(value=Name(id='e', " "ctx=Load(), annotation=None, type_comment=None), " "conversion=-1, format_spec=None)]))], " "type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_JoinedStr(self): code = 'e = 1; f"e = {e}"' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[Assign(targets=[Name(id='e', ctx=Store()" ", annotation=None, type_comment=None" ")], value=Constant(value=1, kind=None), " "type_comment=None), Expr(value=" "JoinedStr(values=[Constant(value='e = ', kind=None), " "FormattedValue(value=Name(id='e', ctx=Load(), " "annotation=None, type_comment=None), " "conversion=-1, format_spec=None)]))], " "type_ignores=[])") self.assertEqual(gast.dump(tree), norm) if sys.version_info.minor >= 8: def test_TypeIgnore(self): code = 'def foo(): pass # type: ignore[excuse]' tree = gast.parse(code, type_comments=True) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(" "args=[], posonlyargs=[], vararg=None, kwonlyargs=[], " "kw_defaults=[], kwarg=None, defaults=[]), body=[" "Pass()], decorator_list=[], returns=None, " "type_comment=None)], type_ignores=" "[TypeIgnore(lineno=1, tag='[excuse]')])") self.assertEqual(gast.dump(tree), norm) def test_PosonlyArgs(self): code = 'def foo(a, /, b): pass' tree = gast.parse(code, type_comments=True) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(" "args=[Name(id='b', ctx=Param(), annotation=None, " "type_comment=None)], posonlyargs=[Name(id='a', " "ctx=Param(), annotation=None, type_comment=None)], " "vararg=None, kwonlyargs=[], kw_defaults=[], " "kwarg=None, defaults=[]), body=[Pass()], " "decorator_list=[], returns=None, type_comment=None)" "], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_NamedExpr(self): code = '(x := 1) ' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[Expr(value=NamedExpr(target=Name(id='x'," " ctx=Store(), annotation=None, type_comment=None), " "value=Constant(value=1, kind=None)))], type_ignores=" "[])") self.assertEqual(gast.dump(tree), norm) else: def test_Bytes(self): code = 'b"0012"' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[Expr(value=Constant(value=b'0012', " "kind=None))], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) # common def test_TryExcept(self): code = 'try:pass\nexcept e:pass\nelse:pass' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[Try(body=[Pass()], handlers=[ExceptHandler(" "type=Name(id='e', ctx=Load(), annotation=None, " "type_comment=None), name=None, body=[Pass()])]" ", orelse=[Pass()], finalbody=[])], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_TryExceptNamed(self): code = 'try:pass\nexcept e as f:pass\nelse:pass' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[Try(body=[Pass()], handlers=[ExceptHandler(" "type=Name(id='e', ctx=Load(), annotation=None, " "type_comment=None), name=Name(id='f', ctx=" "Store(), annotation=None, type_comment=None), body=[Pass()])]" ", orelse=[Pass()], finalbody=[])], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_Raise(self): codes = ('raise Exception', 'raise "Exception"', 'raise Exception, "err"', 'raise Exception("err")', 'raise E, V, T',) norms = ("Module(body=[Raise(exc=Name(id='Exception', ctx=Load(), " "annotation=None, type_comment=None)," " cause=None)], type_ignores=[])", "Module(body=[Raise(exc=Constant(value='Exception', kind=" "None), cause=None)], type_ignores=[])", "Module(body=[Raise(exc=Call(func=Name(id='Exception', " "ctx=Load(), annotation=None, type_comment=None), " "args=[Constant(value='err', kind=None)], " "keywords=[]), cause=None)], type_ignores=[])", "Module(body=[Raise(exc=Call(func=Name(id='Exception', " "ctx=Load(), annotation=None, type_comment=None), " "args=[Constant(value='err', kind=None)], " "keywords=[]), cause=None)], type_ignores=[])", "Module(body=[Raise(exc=Call(func=Attribute(value=Call(" "func=Name(id='E', ctx=Load(), annotation=None, " "type_comment=None), args=[Name(id='V', ctx=" "Load(), annotation=None, type_comment=None)], keywords=[]), " "attr='with_traceback', ctx=Load" "()), args=[Name(id='T', ctx=Load(), annotation=None, " "type_comment=None)], keywords=[]), " "cause=None)], type_ignores=[])",) if sys.version_info.major == 3: codes = codes[0], codes[1], codes[3] norms = norms[0], norms[1], norms[3] for code, norm in zip(codes, norms): tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') self.assertEqual(gast.dump(tree), norm) def test_Call(self): self.maxDiff = None code = 'foo(x, y=1, *args, **kwargs)' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[Expr(value=Call(func=Name(id='foo', ctx=Load" "(), annotation=None, type_comment=None" "), args=[Name(id='x', ctx=Load(), " "annotation=None, type_comment=None), Starred(value=Name(" "id='args', ctx=Load(), annotation=None, type_comment=None)" ", ctx=Load())], keywords=[keyword(" "arg='y', value=Constant(value=1, kind=None)), keyword(arg" "=None, value=Name(id='kwargs', ctx=Load(), annotation=None, " "type_comment=None))]))], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_With(self): code = 'with open("any"): pass' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[With(items=[withitem(context_expr=Call(func=" "Name(id='open', ctx=Load(), annotation=None, " "type_comment=None), args=[Constant(value='any', " "kind=None)], keywords=[]), optional_vars=None)], body=[" "Pass()], type_comment=None)], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_TryFinally(self): code = 'try:pass\nfinally:pass' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[Try(body=[Pass()], handlers=[], orelse=[], " "finalbody=[Pass()])], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_star_argument(self): code = 'def foo(*a): pass' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[], " "posonlyargs=[], vararg=Name(id='a', ctx=Param(), " "annotation=None, type_comment=None), " "kwonlyargs=[], kw_defaults=[], kwarg=None, defaults=[]), " "body=[Pass()], decorator_list=[], returns=None, " "type_comment=None)], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_keyword_argument(self): code = 'def foo(**a): pass' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[], " "posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[], " "kwarg=Name(id='a', ctx=Param(), annotation=None, " "type_comment=None), defaults=[]), body=[Pass()], " "decorator_list=[], returns=None, type_comment=None)], " "type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_Index(self): code = 'def foo(a): a[1]' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[" "Name(id='a', ctx=Param(), annotation=None, type_comment=None)" "], posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[]" ", kwarg=None, defaults=[]), body=[Expr(value=Subscript(value=" "Name(id='a', ctx=Load(), annotation=None, type_comment=None)" ", slice=Constant(value=1, kind=None), ctx=Load()" "))], decorator_list=[], returns=None, type_comment=None)]" ", type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_ExtSlice(self): self.maxDiff = None code = 'def foo(a): a[:,:]' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[" "Name(id='a', ctx=Param(), annotation=None, type_comment=None)" "], posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[]" ", kwarg=None, defaults=[]), body=[Expr(value=Subscript(value=" "Name(id='a', ctx=Load(), annotation=None, type_comment=None)" ", slice=Tuple(elts=[Slice(lower=None, upper=None, step=" "None), Slice(lower=None, upper=None, step=None)], ctx=Load())" ", ctx=Load()))], decorator_list=[], returns=None, " "type_comment=None)], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_ExtSlices(self): self.maxDiff = None code = 'def foo(a): a[1,:]' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[" "Name(id='a', ctx=Param(), annotation=None, type_comment=None)" "], posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[]" ", kwarg=None, defaults=[]), body=[Expr(value=Subscript(value=" "Name(id='a', ctx=Load(), annotation=None, type_comment=None)" ", slice=Tuple(elts=[Constant(value=1, kind=" "None), Slice(lower=None, upper=None, step=None)], ctx=Load())" ", ctx=Load()))], decorator_list=[], returns=None, " "type_comment=None)], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_Ellipsis(self): self.maxDiff = None code = 'def foo(a): a[...]' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[" "Name(id='a', ctx=Param(), annotation=None, type_comment=None)" "], posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[]" ", kwarg=None, defaults=[]), body=[Expr(value=Subscript(value=" "Name(id='a', ctx=Load(), annotation=None, type_comment=None)" ", slice=Constant(value=Ellipsis, kind=None), ctx=Load()))], " "decorator_list=[], returns=None, type_comment=" "None)], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) def test_ExtSliceEllipsis(self): self.maxDiff = None code = 'def foo(a): a[1, ...]' tree = gast.parse(code) compile(gast.gast_to_ast(tree), '', 'exec') norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[" "Name(id='a', ctx=Param(), annotation=None, type_comment=None)" "], posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[]" ", kwarg=None, defaults=[]), body=[Expr(value=Subscript(value=" "Name(id='a', ctx=Load(), annotation=None, type_comment=None)" ", slice=Tuple(elts=[Constant(value=1, kind=None)" ", Constant(value=Ellipsis, kind=None)], ctx=Load()), ctx=" "Load()))], decorator_list=[], returns=None, type_comment=" "None)], type_ignores=[])") self.assertEqual(gast.dump(tree), norm) if __name__ == '__main__': unittest.main() gast-0.5.2/tests/test_self.py000066400000000000000000000015541410170304200161550ustar00rootroot00000000000000import glob import os import unittest import astunparse import gast class SelfTestCase(unittest.TestCase): def setUp(self): self.srcs = glob.glob(os.path.join(gast.__path__[0], '*.py')) def testParse(self): for src_py in self.srcs: with open(src_py) as f: content = f.read() gast.parse(content) def testCompile(self): for src_py in self.srcs: with open(src_py) as f: content = f.read() gnode = gast.parse(content) compile(gast.gast_to_ast(gnode), src_py, 'exec') def test_unparse(self): for src_py in self.srcs: with open(src_py) as f: content = f.read() gnode = gast.parse(content) astunparse.unparse(gast.gast_to_ast(gnode)) if __name__ == '__main__': unittest.main() gast-0.5.2/tox.ini000066400000000000000000000003421410170304200137560ustar00rootroot00000000000000[tox] envlist = py27,py34,py35,py36,py37,py38,py39,lint [testenv] deps = astunparse pytest commands=pytest {posargs} [testenv:lint] deps = astunparse pytest-pep8 pytest < 5 commands=pytest --pep8 -m pep8