sourcecodegen-0.6.14/0000755000175000001440000000000011565157465015077 5ustar mborchusers00000000000000sourcecodegen-0.6.14/LICENSE.txt0000644000175000001440000000337711530263622016715 0ustar mborchusers00000000000000License A copyright notice accompanies this license document that identifies the copyright holders. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions in source code must retain the accompanying copyright notice, this list of conditions, and the following disclaimer. 2. Redistributions in binary form must reproduce the accompanying copyright notice, this list of conditions, and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Names of the copyright holders must not be used to endorse or promote products derived from this software without prior written permission from the copyright holders. 4. If any files are modified, you must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. Disclaimer THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. sourcecodegen-0.6.14/setup.cfg0000644000175000001440000000007311565157465016720 0ustar mborchusers00000000000000[egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 sourcecodegen-0.6.14/CHANGES.txt0000644000175000001440000000633511565157377016721 0ustar mborchusers00000000000000Changelog ========= 0.6.14 (released 19/05/2011) - Fixed issue where an ``ast.Sliceobj`` outside the context of subscripts (this is an odd construct which is unfortunately understood by Python's legacy AST compiler, but not possible to reproduce from source code). The issue affects at least Zope's ``RestrictedPython`` package. [malthe] - Fixed issue where a function with a docstring and a function body would get incorrect indentation. [malthe] 0.6.13 (released 28/02/2011) - Added support for function docstrings. [malthe] 0.6.12 (released 12/01/2010) - Add repoze license text in LICENSE.txt with permission of author. http://lists.repoze.org/pipermail/repoze-dev/2010-January/002554.html [jinty] 0.6.11 (released 15/07/2009) - Added support for ternary operator (Python 2.5+). [malthe] 0.6.10 (released 6/07/2009) - Fixed order of node and expression for augmented assignment. [malthe] 0.6.9 (released 19/05/2009) - Fixed issue where variable keyword-arguments would sometimes not be handled correctly. [malthe] - Fixed issue where identity operator would not be handled correctly. [malthe] 0.6.8 (released 18/05/2009) - Fixed issue where the delete operator would not be treated as a statement. [malthe] 0.6.7 (released 22/04/2009) - Fixed issue where star and double-star arguments would be printed without the '*' and '**' prefix. [malthe] 0.6.6 (released 17/04/2009) - Fixed issue where multiple discarded elements in a statement would get wrong indentation. [malthe] 0.6.5 (released 19/02/2009) - Fixed issue where a module docstring would cause an undesired indentaion. [malthe] 0.6.4 (released 24/01/2009) - Added convenience method to generate code. [malthe] 0.6.3 (released 18/12/2008) - Allow proper operation under Python 2.4 (try/finally fixes). [chrism] 0.6.2 (released 14/12/2008) - Fixed issue where lambda blocks would be incorrectly indented; also, function bodies are now always properly cleared. [malthe] 0.6.1 (released 13/12/2008) - Fixed issue where generators would not be prioritized in certain cases. [malthe] 0.6 (released 13/12/2008) - Operator precedence is now taken into account such that we can avoid unnecessary use of parentheses. This is done not only for aesthetic reasons, but more importantly to avoid running into parser errors due to Python's nesting limitation. [malthe] 0.5 (released 28/11/2008) - Added `yield` and unary operators. [malthe] 0.4 (released 28/11/2008) - Make sure all binary operators get parentheses around them; we may be able to improve upon this in the future (for legibility-reasons). [malthe] 0.3 (released 27/11/2008) - Added logical operators. [malthe] - Fixed tuple unpacking issues. [malthe] - Added for-loop. [malthe] 0.2 (released 27/11/2008) - Fixed issue with tuples and parentheses. [malthe] - Fixed issue with multiple assignments. [malthe] - Added support for `not` operator. [malthe] - Fixed issue where default arguments would be written incorrectly if no non-keyword arguments were present. [malthe] - Fixed issue where statements would not be cleared properly. [malthe] - Assignments are statements and should terminate. [malthe] 0.1 (released 26/11/2008) - Initial release. sourcecodegen-0.6.14/PKG-INFO0000644000175000001440000001372111565157465016200 0ustar mborchusers00000000000000Metadata-Version: 1.0 Name: sourcecodegen Version: 0.6.14 Summary: A Python source-code generator based on the ``compiler.ast`` abstract syntax tree. Home-page: UNKNOWN Author: Malthe Borch Author-email: mborch@gmail.com License: BSD-like (http://repoze.org/license.html) Description: Overview ======== This package provides a module-level source-code generator which operates on the AST from the built-in ``compiler.ast`` module. Note that this AST is not compatible with the new ``ast`` module in Python 2.6. Usage ----- The generator works on AST parse trees. >>> from compiler import parse >>> tree = parse("""\ ... print 'Hello, world!' ... """) We can now generate Python-code equivalent to the original using the source-code generator. >>> from sourcecodegen import ModuleSourceCodeGenerator >>> generator = ModuleSourceCodeGenerator(tree) >>> print generator.getSourceCode() print 'Hello, world!' Author ------ Malthe Borch Changelog ========= 0.6.14 (released 19/05/2011) - Fixed issue where an ``ast.Sliceobj`` outside the context of subscripts (this is an odd construct which is unfortunately understood by Python's legacy AST compiler, but not possible to reproduce from source code). The issue affects at least Zope's ``RestrictedPython`` package. [malthe] - Fixed issue where a function with a docstring and a function body would get incorrect indentation. [malthe] 0.6.13 (released 28/02/2011) - Added support for function docstrings. [malthe] 0.6.12 (released 12/01/2010) - Add repoze license text in LICENSE.txt with permission of author. http://lists.repoze.org/pipermail/repoze-dev/2010-January/002554.html [jinty] 0.6.11 (released 15/07/2009) - Added support for ternary operator (Python 2.5+). [malthe] 0.6.10 (released 6/07/2009) - Fixed order of node and expression for augmented assignment. [malthe] 0.6.9 (released 19/05/2009) - Fixed issue where variable keyword-arguments would sometimes not be handled correctly. [malthe] - Fixed issue where identity operator would not be handled correctly. [malthe] 0.6.8 (released 18/05/2009) - Fixed issue where the delete operator would not be treated as a statement. [malthe] 0.6.7 (released 22/04/2009) - Fixed issue where star and double-star arguments would be printed without the '*' and '**' prefix. [malthe] 0.6.6 (released 17/04/2009) - Fixed issue where multiple discarded elements in a statement would get wrong indentation. [malthe] 0.6.5 (released 19/02/2009) - Fixed issue where a module docstring would cause an undesired indentaion. [malthe] 0.6.4 (released 24/01/2009) - Added convenience method to generate code. [malthe] 0.6.3 (released 18/12/2008) - Allow proper operation under Python 2.4 (try/finally fixes). [chrism] 0.6.2 (released 14/12/2008) - Fixed issue where lambda blocks would be incorrectly indented; also, function bodies are now always properly cleared. [malthe] 0.6.1 (released 13/12/2008) - Fixed issue where generators would not be prioritized in certain cases. [malthe] 0.6 (released 13/12/2008) - Operator precedence is now taken into account such that we can avoid unnecessary use of parentheses. This is done not only for aesthetic reasons, but more importantly to avoid running into parser errors due to Python's nesting limitation. [malthe] 0.5 (released 28/11/2008) - Added `yield` and unary operators. [malthe] 0.4 (released 28/11/2008) - Make sure all binary operators get parentheses around them; we may be able to improve upon this in the future (for legibility-reasons). [malthe] 0.3 (released 27/11/2008) - Added logical operators. [malthe] - Fixed tuple unpacking issues. [malthe] - Added for-loop. [malthe] 0.2 (released 27/11/2008) - Fixed issue with tuples and parentheses. [malthe] - Fixed issue with multiple assignments. [malthe] - Added support for `not` operator. [malthe] - Fixed issue where default arguments would be written incorrectly if no non-keyword arguments were present. [malthe] - Fixed issue where statements would not be cleared properly. [malthe] - Assignments are statements and should terminate. [malthe] 0.1 (released 26/11/2008) - Initial release. Keywords: python source-code generation ast Platform: UNKNOWN Classifier: Development Status :: 6 - Mature Classifier: Intended Audience :: Developers Classifier: Programming Language :: Python Classifier: Topic :: Internet :: WWW/HTTP Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content Classifier: Topic :: Internet :: WWW/HTTP :: WSGI Classifier: Programming Language :: Python :: 2.4 Classifier: Programming Language :: Python :: 2.5 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 sourcecodegen-0.6.14/src/0000755000175000001440000000000011565157465015666 5ustar mborchusers00000000000000sourcecodegen-0.6.14/src/sourcecodegen.egg-info/0000755000175000001440000000000011565157465022205 5ustar mborchusers00000000000000sourcecodegen-0.6.14/src/sourcecodegen.egg-info/SOURCES.txt0000644000175000001440000000076411565157457024101 0ustar mborchusers00000000000000CHANGES.txt COPYRIGHT.txt LICENSE.txt README.txt ez_setup.py setup.py src/sourcecodegen/__init__.py src/sourcecodegen/generation.py src/sourcecodegen/visitor.py src/sourcecodegen.egg-info/PKG-INFO src/sourcecodegen.egg-info/SOURCES.txt src/sourcecodegen.egg-info/dependency_links.txt src/sourcecodegen.egg-info/not-zip-safe src/sourcecodegen.egg-info/top_level.txt src/sourcecodegen/tests/__init__.py src/sourcecodegen/tests/base.py src/sourcecodegen/tests/py24.py src/sourcecodegen/tests/py25plus.pysourcecodegen-0.6.14/src/sourcecodegen.egg-info/not-zip-safe0000644000175000001440000000000111532653265024424 0ustar mborchusers00000000000000 sourcecodegen-0.6.14/src/sourcecodegen.egg-info/PKG-INFO0000644000175000001440000001372111565157457023307 0ustar mborchusers00000000000000Metadata-Version: 1.0 Name: sourcecodegen Version: 0.6.14 Summary: A Python source-code generator based on the ``compiler.ast`` abstract syntax tree. Home-page: UNKNOWN Author: Malthe Borch Author-email: mborch@gmail.com License: BSD-like (http://repoze.org/license.html) Description: Overview ======== This package provides a module-level source-code generator which operates on the AST from the built-in ``compiler.ast`` module. Note that this AST is not compatible with the new ``ast`` module in Python 2.6. Usage ----- The generator works on AST parse trees. >>> from compiler import parse >>> tree = parse("""\ ... print 'Hello, world!' ... """) We can now generate Python-code equivalent to the original using the source-code generator. >>> from sourcecodegen import ModuleSourceCodeGenerator >>> generator = ModuleSourceCodeGenerator(tree) >>> print generator.getSourceCode() print 'Hello, world!' Author ------ Malthe Borch Changelog ========= 0.6.14 (released 19/05/2011) - Fixed issue where an ``ast.Sliceobj`` outside the context of subscripts (this is an odd construct which is unfortunately understood by Python's legacy AST compiler, but not possible to reproduce from source code). The issue affects at least Zope's ``RestrictedPython`` package. [malthe] - Fixed issue where a function with a docstring and a function body would get incorrect indentation. [malthe] 0.6.13 (released 28/02/2011) - Added support for function docstrings. [malthe] 0.6.12 (released 12/01/2010) - Add repoze license text in LICENSE.txt with permission of author. http://lists.repoze.org/pipermail/repoze-dev/2010-January/002554.html [jinty] 0.6.11 (released 15/07/2009) - Added support for ternary operator (Python 2.5+). [malthe] 0.6.10 (released 6/07/2009) - Fixed order of node and expression for augmented assignment. [malthe] 0.6.9 (released 19/05/2009) - Fixed issue where variable keyword-arguments would sometimes not be handled correctly. [malthe] - Fixed issue where identity operator would not be handled correctly. [malthe] 0.6.8 (released 18/05/2009) - Fixed issue where the delete operator would not be treated as a statement. [malthe] 0.6.7 (released 22/04/2009) - Fixed issue where star and double-star arguments would be printed without the '*' and '**' prefix. [malthe] 0.6.6 (released 17/04/2009) - Fixed issue where multiple discarded elements in a statement would get wrong indentation. [malthe] 0.6.5 (released 19/02/2009) - Fixed issue where a module docstring would cause an undesired indentaion. [malthe] 0.6.4 (released 24/01/2009) - Added convenience method to generate code. [malthe] 0.6.3 (released 18/12/2008) - Allow proper operation under Python 2.4 (try/finally fixes). [chrism] 0.6.2 (released 14/12/2008) - Fixed issue where lambda blocks would be incorrectly indented; also, function bodies are now always properly cleared. [malthe] 0.6.1 (released 13/12/2008) - Fixed issue where generators would not be prioritized in certain cases. [malthe] 0.6 (released 13/12/2008) - Operator precedence is now taken into account such that we can avoid unnecessary use of parentheses. This is done not only for aesthetic reasons, but more importantly to avoid running into parser errors due to Python's nesting limitation. [malthe] 0.5 (released 28/11/2008) - Added `yield` and unary operators. [malthe] 0.4 (released 28/11/2008) - Make sure all binary operators get parentheses around them; we may be able to improve upon this in the future (for legibility-reasons). [malthe] 0.3 (released 27/11/2008) - Added logical operators. [malthe] - Fixed tuple unpacking issues. [malthe] - Added for-loop. [malthe] 0.2 (released 27/11/2008) - Fixed issue with tuples and parentheses. [malthe] - Fixed issue with multiple assignments. [malthe] - Added support for `not` operator. [malthe] - Fixed issue where default arguments would be written incorrectly if no non-keyword arguments were present. [malthe] - Fixed issue where statements would not be cleared properly. [malthe] - Assignments are statements and should terminate. [malthe] 0.1 (released 26/11/2008) - Initial release. Keywords: python source-code generation ast Platform: UNKNOWN Classifier: Development Status :: 6 - Mature Classifier: Intended Audience :: Developers Classifier: Programming Language :: Python Classifier: Topic :: Internet :: WWW/HTTP Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content Classifier: Topic :: Internet :: WWW/HTTP :: WSGI Classifier: Programming Language :: Python :: 2.4 Classifier: Programming Language :: Python :: 2.5 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 sourcecodegen-0.6.14/src/sourcecodegen.egg-info/top_level.txt0000644000175000001440000000001611565157457024735 0ustar mborchusers00000000000000sourcecodegen sourcecodegen-0.6.14/src/sourcecodegen.egg-info/dependency_links.txt0000644000175000001440000000000111565157457026254 0ustar mborchusers00000000000000 sourcecodegen-0.6.14/src/sourcecodegen/0000755000175000001440000000000011565157465020513 5ustar mborchusers00000000000000sourcecodegen-0.6.14/src/sourcecodegen/tests/0000755000175000001440000000000011565157465021655 5ustar mborchusers00000000000000sourcecodegen-0.6.14/src/sourcecodegen/tests/py24.py0000644000175000001440000000040411532653265023014 0ustar mborchusers00000000000000from sourcecodegen.tests.base import TestSourceCodeGeneration as Base from sourcecodegen.tests.base import verify class TestSourceCodeGeneration(Base): @verify def testTryFinally(self): try: pass finally: pass sourcecodegen-0.6.14/src/sourcecodegen/tests/base.py0000644000175000001440000002046211565157027023137 0ustar mborchusers00000000000000import sys import unittest import doctest import inspect import textwrap import base64 OPTIONFLAGS = (doctest.ELLIPSIS | doctest.NORMALIZE_WHITESPACE) from compiler import ast from compiler import parse from compiler import pycodegen from sourcecodegen.generation import ModuleSourceCodeGenerator version = sys.version_info[:3] def fix_tree(node): if isinstance(node, ( ast.Module, ast.Class, ast.Function, ast.GenExpr, ast.Lambda)): node.filename = '' # workaround for bug in pycodegen map(fix_tree, node.getChildNodes()) return node def verify_source(source): tree = fix_tree(parse(source, 'exec')) code = pycodegen.ModuleCodeGenerator(tree).getCode() generator = ModuleSourceCodeGenerator(tree) generated = generator.getSourceCode() try: new = fix_tree(parse(generated, 'exec')) except SyntaxError: return generated old = code.co_code new = pycodegen.ModuleCodeGenerator(new).getCode().co_code if old != new: return generated def verify(func): def test(suite): source = textwrap.dedent( "\n".join(inspect.getsource(func).split('\n')[2:]) ) result = verify_source(source) return suite.assertEqual(result, None, "%r != %r" % (source, result)) return test class TestSourceCodeGeneration(unittest.TestCase): """The ``verify`` decorator is used to create a test-case out of simple functions. The objective is to verify correct source-code generation, not actual evaluation.""" @verify def testModule(self): """Module doc-string.""" @verify def testAssignment(self): foo = bar foo = bar = moo foo, bar = bar foo, bar = foo, bar foo, (bar, moo) = foo ((foo, bar), foo) = moo @verify def testAugmentAssignment(self): foo -= bar foo += bar foo %= bar foo /= bar foo **= bar foo -= 1 bar += 1 @verify def testConditions(self): if foo and bar: pass elif boo: pass else: pass @verify def testFunctions(self): def foo(bar): pass def foo(*bar): pass def foo(**bar): pass def foo(bar, *args): pass def foo(bar, *args, **kwargs): pass def foo(foo=None): pass def foo(bar, foo=None, *args, **kwargs): pass def foo(bar, boo, foo=None, moo=42, *args, **kwargs): pass @verify def testDecorators(self): @foo @bar(boo) def bar(foo): pass @verify def testCallFunc(self): foo(bar) foo(bar=None) foo(bar, moo=None) foo(boo, *args) foo(boo, *args, **kwargs) @verify def testDel(self): del foo del foo, bar del foo.bar del foo[bar] del foo[bar:boo] @verify def testListComprehensions(self): [x for x in xs] [x for x in xs if x] [x for x in xs if x == y] @verify def testMultipleListComprehensions(self): [(x * y) for x in xs for y in ys] @verify def testGeneratorComprehensions(self): (x for x in xs) (x for x in xs if x) (x for x in xs if x == y) @verify def testMultipleGeneratorComprehensions(self): (x*y for x in xs for x in xs for y in ys) @verify def testImports(self): import foo import foo.bar from foo import bar from foo.bar import foo, bar @verify def testReturn(self): def test(): return foo return foo, bar @verify def testWhile(self): while True: # don't try this at home pass while False: pass else: pass @verify def testTryExcept(self): try: pass except Exception, e: pass except: pass @verify def testClasses(self): class foo: pass class foo(moo, boo): pass class foo(moo, boo): """this is foo.""" @verify def testLambda(self): foo = lambda: bar bar = lambda foo: bar bar = lambda foo, bar: bar bar = lambda (foo, bar): bar bar = lambda foo, **kwargs: kwargs bar = lambda foo, bar, **kwargs: kwargs bar = lambda foo, *args: args bar = lambda foo, bar, *args: args bar = lambda **kwargs: kwargs bar = lambda *args, **kwargs: (args, kwargs) @verify def testGetAttr(self): foo.bar @verify def testGetItem(self): foo['bar'] @verify def testSetAttr(self): foo.bar = moo @verify def testSetItem(self): foo['bar'] = moo @verify def testSlicing(self): foo[:] foo[1:] foo[:2] foo[1:2:3] @verify def testAssert(self): assert foo @verify def testExec(self): exec foo exec foo in bar exec foo in foo, bar @verify def testSemicolon(self): foo; bar @verify def testPrint(self): print foo print foo, bar print "Hello %s" % bar print >> foo, bar @verify def testRaise(self): raise foo @verify def testArithmetic(self): bar + foo bar - foo bar * foo bar ** foo bar % foo bar / foo bar << foo bar >> foo @verify def testTuples(self): (a, b, c) @verify def testLists(self): [a, b, c] @verify def testDicts(self): {'a': a, 'b': b, 'c': c} @verify def testComparisons(self): foo < bar foo > bar foo == bar foo != bar foo >= bar foo <= bar @verify def testLogicalOperators(self): foo | bar & bar ^ foo ~ bar ~ (foo or bar) @verify def testOperators(self): not foo bar or foo not (foo or bar) @verify def testIdentity(self): foo is bar @verify def testFormatString(self): foo % (bar, moo) foo % (bar or foo) foo % (bar and foo) @verify def testLoop(self): for foo in bar: pass else: pass for foo, (bar, moo) in boo: pass @verify def testYield(self): yield foo @verify def testUnary(self): +foo -bar @verify def testOperatorPrecenceRules(self): a + b / c % d * e - b / c + a a & c | d ^ e not abc() @verify def testOptimalOperatorPredence(self): a + b + c + d + e + f + g + h + i + j + k + l + m + n + a + b + \ a + b + c + d + e + f + g + h + i + j + k + l + m + n + a + b + \ a + b + c + d + e + f + g + h + i + j + k + l + m + n + a + b + \ a + b + c + d + e + f + g + h + i + j + k + l + m + n + a + b + \ a + b + c + d + e + f + g + h + i + j + k + l + m + n + a + b @verify def testFunctionNesting(self): def abc(): def ghi(): a = lambda jkl: mno def testStandaloneString(self): self.assertEqual(verify_source( "'0'\n0"), None) @verify def testIndentation(self): for abc in abc: ppp ppp @verify def testMethod(self): abc(*args) abc(**kwargs) abc(ghi, *args) abc(ghi, **kwargs) abc(ghi, *args, **kwargs) @verify def testBreak(self): for i in range(5): break @verify def testContinue(self): for i in range(5): continue @verify def testFunctionDocString(self): def foo(): """bar.""" def bar(): "hello" def boo(): """bar.""" return foo def baz(): "hello" return boo def testSliceobj(self): node = ast.Sliceobj([ast.Const(1), ast.Const(2), ast.Const(3)]) source = ModuleSourceCodeGenerator(node).getSourceCode() self.assertEqual(source, 'slice(1, 2, 3)') sourcecodegen-0.6.14/src/sourcecodegen/tests/py25plus.py0000644000175000001440000000060711532653265023726 0ustar mborchusers00000000000000from sourcecodegen.tests.base import TestSourceCodeGeneration as Base from sourcecodegen.tests.base import verify class TestSourceCodeGeneration(Base): @verify def testTryFinally(self): try: pass except: pass else: pass finally: pass @verify def testTernary(self): foo if bar else boo sourcecodegen-0.6.14/src/sourcecodegen/tests/__init__.py0000644000175000001440000000001411532653265023752 0ustar mborchusers00000000000000# a package sourcecodegen-0.6.14/src/sourcecodegen/visitor.py0000644000175000001440000004217611565156420022564 0ustar mborchusers00000000000000import sys from compiler import ast from cStringIO import StringIO version = sys.version_info[:3] def triple_quote(doc): return '"""%s"""' % doc.replace('"""', '\"\"\"') def format_argnames(argnames): return ", ".join( isinstance(name, tuple) and "(%s)" % format_argnames(name) or name \ for name in argnames) def format_ass(node): if isinstance(node, ast.AssTuple): return "(%s)" % ", ".join(format_ass(ass) for ass in node) return node.name class prioritized(object): def __init__(self, generator, priority): self.generator = generator self.priority = priority def __iter__(self): return self.generator def prioritize(priority): def decorator(func): def visit(self, node): return prioritized(func(self, node), priority) return visit return decorator def unary(symbol, priority): @prioritize(prioritize) def visit(self, node): yield symbol child = self.visit(node.expr) if child.priority < priority: yield '(' yield child yield ')' else: yield child return visit def binary(symbol, priority): @prioritize(prioritize) def visit(self, node): left = self.visit(node.left) right = self.visit(node.right) if left.priority < priority and right.priority < priority: yield '(' yield left yield ' %s ' % symbol yield right yield ')' else: yield left yield ' %s ' % symbol yield right return visit def n_ary(symbol, priority): @prioritize(prioritize) def visit(self, node): yield '(' for condition in tuple(node)[:-1]: yield self.visit(condition) yield ' %s ' % symbol yield self.visit(tuple(node)[-1]) yield ')' return visit class CodeStream(object): def __init__(self, indentation_string="\t"): self.indentation_string = indentation_string self.indentation = 0 self.stream = StringIO() self.clear = True def __call__(self, value): if value is None: return self.write(None) if isinstance(value, basestring): return self.out(value) if isinstance(value, tuple) and self.clear is False: self.write() self.clear = True self.indentation += 1 self(value) self.indentation -= 1 else: for part in value: self(part) def write(self, text=None): if text or self.clear == False: self.out(text) self.stream.write('\n') self.clear = True def out(self, text): if self.clear is True: indentation = self.indentation_string * self.indentation self.stream.write(indentation) self.clear = False self.stream.write(text or "") def getvalue(self): return self.stream.getvalue() class ASTVisitor(object): def __init__(self, tree): self.tree = tree def __call__(self): stream = CodeStream() stream(self.visit(self.tree)) return stream.getvalue() def visit(self, node): name = node.__class__.__name__ try: func = getattr(self, 'visit%s' % name) except AttributeError: raise NotImplementedError( "Unable to visit `%s`." % repr(node)) gen = func(node) if isinstance(gen, prioritized): return gen return prioritized(gen, 0) def visitModule(self, node): if node.doc is not None: yield triple_quote(node.doc) yield None for node in self.visit(node.node): yield node def visitStmt(self, node): yield tuple(self.visit(child) for child in node.nodes if child is not None) def visitIf(self, node): for index, test in enumerate(node.tests): if index == 0: yield "if " else: yield "elif " condition, statement = test yield self.visit(condition) yield ":" yield self.visit(statement), if node.else_: yield "else:" yield self.visit(node.else_), @prioritize(-3) def visitName(self, node): yield node.name def visitPass(self, node): yield "pass" yield None def visitDiscard(self, node): yield self.visit(node.expr) yield None def visitAssign(self, node): for index, ass in enumerate(tuple(node.nodes)): yield self.visit(ass) if index < len(tuple(node.nodes)) - 1: yield " = " yield " = " yield self.visit(node.expr) yield None def visitAssName(self, node): if node.flags == 'OP_DELETE': yield "del " yield node.name if node.flags == 'OP_DELETE': yield None def visitFunction(self, node): if node.decorators: yield self.visit(node.decorators) yield "def %s(" % node.name argnames = list(node.argnames) if argnames: if node.kwargs: kwargs = argnames.pop() if node.varargs: varargs = argnames.pop() if node.defaults: yield format_argnames(argnames[:-len(node.defaults)]) for index, default in enumerate(node.defaults): name = argnames[index-len(node.defaults)] if len(argnames) > len(node.defaults) or index > 0: yield ", %s=" % name else: yield "%s=" % name yield self.visit(default) else: yield format_argnames(argnames) if node.varargs: if len(node.argnames) > 1: yield ", " yield "*%s" % varargs if node.kwargs: if len(node.argnames) > 1 or node.varargs: yield ", " yield "**%s" % kwargs yield "):" if node.doc: yield triple_quote(node.doc), yield self.visit(node.code), @prioritize(0) def visitConst(self, node): yield repr(node.value) def visitDecorators(self, node): for decorator in tuple(node): yield '@' yield self.visit(decorator) yield None def visitCallFunc(self, node): yield self.visit(node.node) yield '(' for arg in tuple(node.args)[:-1]: yield self.visit(arg) yield ", " if node.args: yield self.visit(node.args[-1]) if node.star_args: if node.args: yield ", *" else: yield "*" yield self.visit(node.star_args) if node.dstar_args: if node.args: yield ", **" else: yield "**" yield self.visit(node.dstar_args) yield ")" def visitKeyword(self, node): yield "%s=" % node.name yield self.visit(node.expr) def visitAssTuple(self, node): first = node while isinstance(first, ast.AssTuple): first = first.nodes[0] if first.flags == 'OP_DELETE': yield "del " yield format_ass(node) if first.flags == 'OP_DELETE': yield None @prioritize(0) def visitTuple(self, node): yield "(" for index, item in enumerate(tuple(node)): yield self.visit(item) if index < len(tuple(node)) - 1: yield ", " if len(node.nodes) == 1: yield ", " yield ")" def visitGenExpr(self, node): yield "(" yield self.visit(node.code) yield ")" def visitListComp(self, node): yield "[" yield self.visitGenExprInner(node) yield "]" def visitGenExprInner(self, node): yield self.visit(node.expr) for qual in node.quals: yield self.visit(qual) def visitGenExprFor(self, node): yield " for " yield self.visit(node.assign) yield " in " yield self.visit(node.iter) for _if in node.ifs: yield self.visit(_if) def visitGenExprIf(self, node): yield " if " yield self.visit(node.test) def visitListCompFor(self, node): yield " for " yield self.visit(node.assign) yield " in " yield self.visit(node.list) for _if in node.ifs: yield self.visit(_if) def visitListCompIf(self, node): yield " if " yield self.visit(node.test) def visitCompare(self, node): yield self.visit(node.expr) for op, expr in node.ops: yield ' %s ' % op yield self.visit(expr) def visitImport(self, node): yield "import " for index, (name, alias) in enumerate(node.names): yield name if alias is not None: yield " as %s" % alias if index < len(node.names) - 1: yield ", " yield None def visitFrom(self, node): yield "from %s import " % node.modname for index, (name, alias) in enumerate(node.names): yield name if alias is not None: yield " as %s" % alias if index < len(node.names) - 1: yield ", " yield None def visitReturn(self, node): yield "return " yield self.visit(node.value) yield None def visitBreak(self, node): yield "break" yield None def visitContinue(self, node): yield "continue" yield None def visitWhile(self, node): yield "while " yield self.visit(node.test) yield ":" yield self.visit(node.body), if node.else_ is not None: yield "else:" yield self.visit(node.else_), def visitTryExcept(self, node): yield "try:" yield self.visit(node.body), for cls, var, body in node.handlers: yield "except" if cls is not None: yield " " yield self.visit(cls) if var is not None: if cls is None: yield " " else: yield ", " yield self.visit(var) yield ":" yield self.visit(body), if node.else_: yield "else:" yield self.visit(node.else_), def visitTryFinally(self, node): if version < (2,5): yield "try:" yield self.visit(node.body), yield "finally:" yield self.visit(node.final), else: yield self.visit(node.body) yield "finally:" yield self.visit(node.final), def visitClass(self, node): yield "class %s" % node.name if node.bases: yield "(" for index, base in enumerate(node.bases): yield self.visit(base) if index < len(node.bases) - 1: yield ", " yield ")" yield ":" if node.doc: yield triple_quote(node.doc), self.visit(node.code) else: yield self.visit(node.code) @prioritize(-2) def visitLambda(self, node): yield "lambda" argnames = list(node.argnames) if argnames: yield " " if node.kwargs: kwargs = argnames.pop() if node.varargs: varargs = argnames.pop() if node.defaults: yield format_argnames(argnames[:-len(node.defaults)]) for index, default in enumerate(node.defaults): name = argnames[index-len(node.defaults)] yield ", %s=" % name yield self.visit(default) else: yield format_argnames(argnames) offset = (node.varargs or 0) + (node.kwargs or 0) if node.varargs: if len(node.argnames) > offset: yield ", " yield "*%s" % varargs if node.kwargs: if node.varargs or len(node.argnames) > offset: yield ", " yield "**%s" % kwargs yield ": " yield self.visit(node.code) def visitGetattr(self, node): yield self.visit(node.expr) yield ".%s" % node.attrname def visitAssAttr(self, node): if node.flags == 'OP_DELETE': yield "del " yield self.visit(node.expr) yield ".%s" % node.attrname if node.flags == 'OP_DELETE': yield None def visitSubscript(self, node): if node.flags == 'OP_DELETE': yield "del " yield self.visit(node.expr) yield '[' for index, sub in enumerate(node.subs): if isinstance(sub, ast.Sliceobj): for i, slice in enumerate(tuple(sub)): yield self.visit(slice) if i < len(tuple(sub)) - 1: yield ":" else: yield self.visit(sub) if index < len(node.subs) - 1: yield ', ' yield ']' if node.flags == 'OP_DELETE': yield None def visitSlice(self, node): if node.flags == 'OP_DELETE': yield "del " yield self.visit(node.expr) yield '[' if node.lower: yield self.visit(node.lower) yield ':' if node.upper: yield self.visit(node.upper) yield ']' if node.flags == 'OP_DELETE': yield None def visitSliceobj(self, node): yield 'slice(' for index, item in enumerate(tuple(node)): yield self.visit(item) if index < len(tuple(node)) - 1: yield ", " yield ')' def visitExec(self, node): yield "exec " yield self.visit(node.expr) if node.locals: yield " in " yield self.visit(node.locals) if node.globals: yield ", " yield self.visit(node.globals) yield None def visitAssert(self, node): yield "assert " yield self.visit(node.test) if node.fail is not None: yield ", " yield self.visit(node.fail) yield None def visitRaise(self, node): yield "raise " yield self.visit(node.expr1) if node.expr2: yield ", " yield self.visit(node.expr2) if node.expr3: yield ", " yield self.visit(node.expr3) def visitPrintnl(self, node): yield "print " if node.dest is not None: yield ">> " yield self.visit(node.dest) yield ", " for index, expr in enumerate(tuple(node.nodes)): if expr is None: continue yield self.visit(expr) if index < len(tuple(node.nodes)) - 1 and node.nodes[index+1] is not None: yield ", " yield None def visitIfExp(self, node): yield self.visit(node.then) yield " if " yield self.visit(node.test) yield " else " yield self.visit(node.else_) def visitWith(self, node): raise NotImplementedError( "The `with` keyword is not supported.") def visitAugAssign(self, node): yield self.visit(node.node) yield " %s " % node.op yield self.visit(node.expr) yield None def visitList(self, node): yield '[' for index, item in enumerate(node.nodes): yield self.visit(item) if index < len(node.nodes) - 1: yield ", " yield ']' def visitDict(self, node): yield '{' for index, (expr, value) in enumerate(node.items): yield self.visit(expr) yield ': ' yield self.visit(value) if index < len(node.items) - 1: yield ", " yield '}' def visitFor(self, node): yield "for %s in " % format_ass(node.assign) yield self.visit(node.list) yield ":" yield self.visit(node.body) if node.else_ is not None: yield "else:" yield self.visit(node.else_) def visitYield(self, node): yield "yield " yield self.visit(node.value) yield None visitPower = binary('**', 10) visitInvert = unary('~', 9) visitUnaryAdd = unary('+', 8) visitUnarySub = unary('-', 8) visitMul = binary('*', 7) visitMod = binary('%', 7) visitDiv = binary('/', 7) visitAdd = binary('+', 6) visitSub = binary('-', 6) visitLeftShift = binary('<<', 5) visitRightShift = binary('>>', 5) visitBitand = n_ary('&', 4) visitBitxor = n_ary('^', 3) visitBitor = n_ary('|', 2) visitNot = unary('not ', 1) visitAnd = n_ary('and', 0) visitOr = n_ary('or', -1) sourcecodegen-0.6.14/src/sourcecodegen/__init__.py0000644000175000001440000000016411532653265022616 0ustar mborchusers00000000000000from sourcecodegen.generation import ModuleSourceCodeGenerator from sourcecodegen.generation import generate_code sourcecodegen-0.6.14/src/sourcecodegen/generation.py0000644000175000001440000000063711532653265023217 0ustar mborchusers00000000000000from visitor import ASTVisitor class ModuleSourceCodeGenerator(object): """Generates Python source code from an AST tree (as parsed by the ``compiler.parse`` method).""" def __init__(self, tree): self.tree = tree def getSourceCode(self): visitor = ASTVisitor(self.tree) return visitor() def generate_code(tree): return ModuleSourceCodeGenerator(tree).getSourceCode() sourcecodegen-0.6.14/COPYRIGHT.txt0000644000175000001440000000007211530263622017170 0ustar mborchusers00000000000000Copyright (c) 2008-2010 Malthe Borch, All Rights Reserved sourcecodegen-0.6.14/README.txt0000644000175000001440000000127111530263622016557 0ustar mborchusers00000000000000Overview ======== This package provides a module-level source-code generator which operates on the AST from the built-in ``compiler.ast`` module. Note that this AST is not compatible with the new ``ast`` module in Python 2.6. Usage ----- The generator works on AST parse trees. >>> from compiler import parse >>> tree = parse("""\ ... print 'Hello, world!' ... """) We can now generate Python-code equivalent to the original using the source-code generator. >>> from sourcecodegen import ModuleSourceCodeGenerator >>> generator = ModuleSourceCodeGenerator(tree) >>> print generator.getSourceCode() print 'Hello, world!' Author ------ Malthe Borch sourcecodegen-0.6.14/ez_setup.py0000644000175000001440000002231311530263622017271 0ustar mborchusers00000000000000#!python """Bootstrap setuptools installation If you want to use setuptools in your package's setup.py, just include this file in the same directory with it, and add this to the top of your setup.py:: from ez_setup import use_setuptools use_setuptools() If you want to require a specific version of setuptools, set a download mirror, or use an alternate download directory, you can do so by supplying the appropriate options to ``use_setuptools()``. This file can also be run as a script to install or upgrade setuptools. """ import sys DEFAULT_VERSION = "0.6c8" DEFAULT_URL = "http://pypi.python.org/packages/%s/s/setuptools/" % sys.version[:3] md5_data = { 'setuptools-0.6b1-py2.3.egg': '8822caf901250d848b996b7f25c6e6ca', 'setuptools-0.6b1-py2.4.egg': 'b79a8a403e4502fbb85ee3f1941735cb', 'setuptools-0.6b2-py2.3.egg': '5657759d8a6d8fc44070a9d07272d99b', 'setuptools-0.6b2-py2.4.egg': '4996a8d169d2be661fa32a6e52e4f82a', 'setuptools-0.6b3-py2.3.egg': 'bb31c0fc7399a63579975cad9f5a0618', 'setuptools-0.6b3-py2.4.egg': '38a8c6b3d6ecd22247f179f7da669fac', 'setuptools-0.6b4-py2.3.egg': '62045a24ed4e1ebc77fe039aa4e6f7e5', 'setuptools-0.6b4-py2.4.egg': '4cb2a185d228dacffb2d17f103b3b1c4', 'setuptools-0.6c1-py2.3.egg': 'b3f2b5539d65cb7f74ad79127f1a908c', 'setuptools-0.6c1-py2.4.egg': 'b45adeda0667d2d2ffe14009364f2a4b', 'setuptools-0.6c2-py2.3.egg': 'f0064bf6aa2b7d0f3ba0b43f20817c27', 'setuptools-0.6c2-py2.4.egg': '616192eec35f47e8ea16cd6a122b7277', 'setuptools-0.6c3-py2.3.egg': 'f181fa125dfe85a259c9cd6f1d7b78fa', 'setuptools-0.6c3-py2.4.egg': 'e0ed74682c998bfb73bf803a50e7b71e', 'setuptools-0.6c3-py2.5.egg': 'abef16fdd61955514841c7c6bd98965e', 'setuptools-0.6c4-py2.3.egg': 'b0b9131acab32022bfac7f44c5d7971f', 'setuptools-0.6c4-py2.4.egg': '2a1f9656d4fbf3c97bf946c0a124e6e2', 'setuptools-0.6c4-py2.5.egg': '8f5a052e32cdb9c72bcf4b5526f28afc', 'setuptools-0.6c5-py2.3.egg': 'ee9fd80965da04f2f3e6b3576e9d8167', 'setuptools-0.6c5-py2.4.egg': 'afe2adf1c01701ee841761f5bcd8aa64', 'setuptools-0.6c5-py2.5.egg': 'a8d3f61494ccaa8714dfed37bccd3d5d', 'setuptools-0.6c6-py2.3.egg': '35686b78116a668847237b69d549ec20', 'setuptools-0.6c6-py2.4.egg': '3c56af57be3225019260a644430065ab', 'setuptools-0.6c6-py2.5.egg': 'b2f8a7520709a5b34f80946de5f02f53', 'setuptools-0.6c7-py2.3.egg': '209fdf9adc3a615e5115b725658e13e2', 'setuptools-0.6c7-py2.4.egg': '5a8f954807d46a0fb67cf1f26c55a82e', 'setuptools-0.6c7-py2.5.egg': '45d2ad28f9750e7434111fde831e8372', 'setuptools-0.6c8-py2.3.egg': '50759d29b349db8cfd807ba8303f1902', 'setuptools-0.6c8-py2.4.egg': 'cba38d74f7d483c06e9daa6070cce6de', 'setuptools-0.6c8-py2.5.egg': '1721747ee329dc150590a58b3e1ac95b', } import sys, os def _validate_md5(egg_name, data): if egg_name in md5_data: from md5 import md5 digest = md5(data).hexdigest() if digest != md5_data[egg_name]: print >>sys.stderr, ( "md5 validation of %s failed! (Possible download problem?)" % egg_name ) sys.exit(2) return data def use_setuptools( version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir, download_delay=15 ): """Automatically find/download setuptools and make it available on sys.path `version` should be a valid setuptools version number that is available as an egg for download under the `download_base` URL (which should end with a '/'). `to_dir` is the directory where setuptools will be downloaded, if it is not already available. If `download_delay` is specified, it should be the number of seconds that will be paused before initiating a download, should one be required. If an older version of setuptools is installed, this routine will print a message to ``sys.stderr`` and raise SystemExit in an attempt to abort the calling script. """ was_imported = 'pkg_resources' in sys.modules or 'setuptools' in sys.modules def do_download(): egg = download_setuptools(version, download_base, to_dir, download_delay) sys.path.insert(0, egg) import setuptools; setuptools.bootstrap_install_from = egg try: import pkg_resources except ImportError: return do_download() try: pkg_resources.require("setuptools>="+version); return except pkg_resources.VersionConflict, e: if was_imported: print >>sys.stderr, ( "The required version of setuptools (>=%s) is not available, and\n" "can't be installed while this script is running. Please install\n" " a more recent version first, using 'easy_install -U setuptools'." "\n\n(Currently using %r)" ) % (version, e.args[0]) sys.exit(2) else: del pkg_resources, sys.modules['pkg_resources'] # reload ok return do_download() except pkg_resources.DistributionNotFound: return do_download() def download_setuptools( version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir, delay = 15 ): """Download setuptools from a specified location and return its filename `version` should be a valid setuptools version number that is available as an egg for download under the `download_base` URL (which should end with a '/'). `to_dir` is the directory where the egg will be downloaded. `delay` is the number of seconds to pause before an actual download attempt. """ import urllib2, shutil egg_name = "setuptools-%s-py%s.egg" % (version,sys.version[:3]) url = download_base + egg_name saveto = os.path.join(to_dir, egg_name) src = dst = None if not os.path.exists(saveto): # Avoid repeated downloads try: from distutils import log if delay: log.warn(""" --------------------------------------------------------------------------- This script requires setuptools version %s to run (even to display help). I will attempt to download it for you (from %s), but you may need to enable firewall access for this script first. I will start the download in %d seconds. (Note: if this machine does not have network access, please obtain the file %s and place it in this directory before rerunning this script.) ---------------------------------------------------------------------------""", version, download_base, delay, url ); from time import sleep; sleep(delay) log.warn("Downloading %s", url) src = urllib2.urlopen(url) # Read/write all in one block, so we don't create a corrupt file # if the download is interrupted. data = _validate_md5(egg_name, src.read()) dst = open(saveto,"wb"); dst.write(data) finally: if src: src.close() if dst: dst.close() return os.path.realpath(saveto) def main(argv, version=DEFAULT_VERSION): """Install or upgrade setuptools and EasyInstall""" try: import setuptools except ImportError: egg = None try: egg = download_setuptools(version, delay=0) sys.path.insert(0,egg) from setuptools.command.easy_install import main return main(list(argv)+[egg]) # we're done here finally: if egg and os.path.exists(egg): os.unlink(egg) else: if setuptools.__version__ == '0.0.1': print >>sys.stderr, ( "You have an obsolete version of setuptools installed. Please\n" "remove it from your system entirely before rerunning this script." ) sys.exit(2) req = "setuptools>="+version import pkg_resources try: pkg_resources.require(req) except pkg_resources.VersionConflict: try: from setuptools.command.easy_install import main except ImportError: from easy_install import main main(list(argv)+[download_setuptools(delay=0)]) sys.exit(0) # try to force an exit else: if argv: from setuptools.command.easy_install import main main(argv) else: print "Setuptools version",version,"or greater has been installed." print '(Run "ez_setup.py -U setuptools" to reinstall or upgrade.)' def update_md5(filenames): """Update our built-in md5 registry""" import re from md5 import md5 for name in filenames: base = os.path.basename(name) f = open(name,'rb') md5_data[base] = md5(f.read()).hexdigest() f.close() data = [" %r: %r,\n" % it for it in md5_data.items()] data.sort() repl = "".join(data) import inspect srcfile = inspect.getsourcefile(sys.modules[__name__]) f = open(srcfile, 'rb'); src = f.read(); f.close() match = re.search("\nmd5_data = {\n([^}]+)}", src) if not match: print >>sys.stderr, "Internal error!" sys.exit(2) src = src[:match.start(1)] + repl + src[match.end(1):] f = open(srcfile,'w') f.write(src) f.close() if __name__=='__main__': if len(sys.argv)>2 and sys.argv[1]=='--md5update': update_md5(sys.argv[2:]) else: main(sys.argv[1:]) sourcecodegen-0.6.14/setup.py0000644000175000001440000000274411565157440016611 0ustar mborchusers00000000000000__version__ = '0.6.14' import os import sys from ez_setup import use_setuptools use_setuptools() from setuptools import setup, find_packages here = os.path.abspath(os.path.dirname(__file__)) README = open(os.path.join(here, 'README.txt')).read() CHANGES = open(os.path.join(here, 'CHANGES.txt')).read() version = sys.version_info[:3] if version < (2,5): test_suite = 'sourcecodegen.tests.py24' else: test_suite = 'sourcecodegen.tests.py25plus' setup( name='sourcecodegen', version=__version__, description='A Python source-code generator based on the ``compiler.ast`` ' + 'abstract syntax tree.', long_description="\n\n".join((README, CHANGES)), classifiers=[ "Development Status :: 6 - Mature", "Intended Audience :: Developers", "Programming Language :: Python", "Topic :: Internet :: WWW/HTTP", "Topic :: Internet :: WWW/HTTP :: Dynamic Content", "Topic :: Internet :: WWW/HTTP :: WSGI", "Programming Language :: Python :: 2.4", "Programming Language :: Python :: 2.5", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", ], keywords='python source-code generation ast', author="Malthe Borch", author_email="mborch@gmail.com", license='BSD-like (http://repoze.org/license.html)', packages=find_packages('src'), package_dir = {'': 'src'}, include_package_data=True, zip_safe=False, test_suite=test_suite, )