Nuitka-0.5.0.1/ 0000755 0001750 0001750 00000000000 12265271051 013341 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/LICENSE.txt 0000644 0001750 0001750 00000026136 12265044767 015210 0 ustar hayen hayen 0000000 0000000
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Nuitka-0.5.0.1/misc/ 0000755 0001750 0001750 00000000000 12265271051 014274 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/misc/print-all-sources.sh 0000755 0001750 0001750 00000002075 12265264105 020224 0 ustar hayen hayen 0000000 0000000 #!/bin/sh
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
cd `dirname $0`/..
find nuitka -name \*.py -a \! -path *inline_copy*
find bin -name \*.py
find nuitka/build/static_src -name \*.cpp
find nuitka/build/include -name \*.hpp
find nuitka/build/ -name \*.scons
find misc -name \*.sh
find bin -name \*.sh
echo Developer_Manual.rst
echo Changelog.rst
Nuitka-0.5.0.1/misc/check-release 0000755 0001750 0001750 00000035103 12265264105 016721 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import print_function
import os, sys, tempfile, subprocess
if "nocheck" in os.environ.get( "DEB_BUILD_OPTIONS", "" ).split():
print( "Skiped all tests as per DEB_BUILD_OPTIONS environment." )
sys.exit( 0 )
from optparse import OptionParser
parser = OptionParser()
parser.add_option(
"--skip-basic-tests",
action = "store_false",
dest = "basic_tests",
default = True,
help = """\
The basic tests, execute these to check if Nuitka is healthy. Default
is %default."""
)
parser.add_option(
"--skip-syntax-tests",
action = "store_false",
dest = "syntax_tests",
default = True,
help = """\
The syntax tests, execute these to check if Nuitka handles Syntax errors fine.
Default is %default."""
)
parser.add_option(
"--skip-program-tests",
action = "store_false",
dest = "program_tests",
default = True,
help = """\
The programs tests, execute these to check if Nuitka handles programs, e.g.
import recursions, etc. fine. Default is %default."""
)
parser.add_option(
"--skip-package-tests",
action = "store_false",
dest = "package_tests",
default = True,
help = """\
The packages tests, execute these to check if Nuitka handles packages, e.g.
import recursions, etc. fine. Default is %default."""
)
parser.add_option(
"--skip-standalone-tests",
action = "store_false",
dest = "standalone_tests",
default = os.name == "posix" and os.uname()[0] != "NetBSD",
help = """\
The packages tests, execute these to check if Nuitka handles packages, e.g.
import recursions, etc. fine. Default is %default."""
)
parser.add_option(
"--skip-reflection-test",
action = "store_false",
dest = "reflection_test",
default = True,
help = """\
The reflection test compiles Nuitka with Nuitka, and then Nuitka with the
compile Nuitka and compares the outputs. Default is %default."""
)
parser.add_option(
"--skip-cpython26-tests",
action = "store_false",
dest = "cpython26",
default = True,
help = """\
The standard CPython2.6 test suite. Execute this for all corner cases to be
covered. With Python 2.7 this covers exception behavior quite well. Default
is %default."""
)
parser.add_option(
"--skip-cpython27-tests",
action = "store_false",
dest = "cpython27",
default = True,
help = """\
The standard CPython2.7 test suite. Execute this for all corner cases to be
covered. With Python 2.6 these are not run. Default is %default."""
)
parser.add_option(
"--skip-cpython32-tests",
action = "store_false",
dest = "cpython32",
default = True,
help = """\
The standard CPython3.2 test suite. Execute this for all corner cases to be
covered. With Python 2.6 these are not run. Default is %default."""
)
parser.add_option(
"--skip-cpython33-tests",
action = "store_false",
dest = "cpython33",
default = True,
help = """\
The standard CPython3.3 test suite. Execute this for all corner cases to be
covered. With Python 2.x these are not run. Default is %default."""
)
parser.add_option(
"--no-python2.6",
action = "store_true",
dest = "no26",
default = False,
help = """\
Do not use Python2.6 even if available on the system. Default is %default."""
)
parser.add_option(
"--no-python2.7",
action = "store_true",
dest = "no27",
default = False,
help = """\
Do not use Python2.7 even if available on the system. Default is %default."""
)
parser.add_option(
"--no-python3.2",
action = "store_true",
dest = "no32",
default = False,
help = """\
Do not use Python3.2 even if available on the system. Default is %default."""
)
parser.add_option(
"--no-python3.3",
action = "store_true",
dest = "no33",
default = False,
help = """\
Do not use Python3.3 even if available on the system. Default is %default."""
)
parser.add_option(
"--no-wine",
action = "store_true",
dest = "no_wine",
default = False,
help = """\
Do not use wine to cross compile. Default is %default."""
)
options, positional_args = parser.parse_args()
if positional_args:
parser.print_help()
sys.exit( "\nError, no positional argument allowed." )
# Go its own directory, to have it easy with path knowledge.
os.chdir( os.path.dirname( os.path.abspath( __file__ ) ) )
os.chdir( ".." )
path_sep = ";" if os.name == "nt" else ":"
# Add the local bin directory to search path start.
os.environ[ "PATH" ] = \
os.path.join(
os.getcwd(),
"bin" ) + \
path_sep + \
os.environ[ "PATH" ]
def checkExecutableCommand( command ):
""" Check if a command is executable. """
# Do respect given options to disable specific Python versions
if command == "python2.6" and options.no26:
return False
if command == "python2.7" and options.no27:
return False
if command == "python3.2" and options.no32:
return False
if command == "python3.3" and options.no33:
return False
if command == "wine" and options.no_wine:
return False
# Shortcuts for python versions, also needed for Windows as it won't have
# the version number in the Python binaries at all.
if command == "python2.6" and sys.version_info[0:2] == (2,6):
return True
if command == "python2.7" and sys.version_info[0:2] == (2,7):
return True
if command == "python3.2" and sys.version_info[0:2] == (3,2):
return True
if command == "python3.3" and sys.version_info[0:2] == (3,3):
return True
path = os.environ[ "PATH" ]
suffixes = ( ".exe", ) if os.name == "nt" else ( "", )
for part in path.split( path_sep ):
if not part:
continue
for suffix in suffixes:
if os.path.exists( os.path.join( part, command + suffix ) ):
return True
else:
return False
def setExtraFlags( where, name, flags ):
if where is not None:
tmp_dir = tempfile.gettempdir()
# Try to avoid RAM disk /tmp and use the disk one instead.
if tmp_dir == "/tmp" and os.path.exists( "/var/tmp" ):
tmp_dir = "/var/tmp"
where = os.path.join( tmp_dir, name, where )
if not os.path.exists( where ):
os.makedirs( where )
os.environ[ "NUITKA_EXTRA_OPTIONS" ] = flags + " --output-dir=" + where
else:
os.environ[ "NUITKA_EXTRA_OPTIONS" ] = flags
def executeSubTest( command, hide_stderror = False ):
parts = command.split()
parts[0] = parts[0].replace( "/", os.path.sep )
# On Windows, we need to specify Python ourselves.
if os.name == "nt":
parts.insert( 0, sys.executable )
print( "Run '%s' in '%s'." % ( " ".join( parts ), os.getcwd() ) )
sys.stdout.flush()
result = subprocess.call( parts )
if result != 0:
sys.exit( result )
def execute_tests( where, use_python, flags ):
print(
"Executing test case called %s with CPython %s and extra flags '%s'." % (
where,
use_python,
flags
)
)
if os.name == "nt":
if use_python == "python2.6":
if sys.version.startswith( "2.6"):
os.environ[ "PYTHON" ] = sys.executable
else:
os.environ[ "PYTHON" ] = r"C:\Python26\python.exe"
elif use_python == "python2.7":
if sys.version.startswith( "2.7"):
os.environ[ "PYTHON" ] = sys.executable
else:
os.environ[ "PYTHON" ] = r"C:\Python27\python.exe"
elif use_python == "python3.2":
if sys.version.startswith( "3.2"):
os.environ[ "PYTHON" ] = sys.executable
else:
os.environ[ "PYTHON" ] = r"C:\Python32\python.exe"
elif use_python == "python3.3":
if sys.version.startswith( "3.3"):
os.environ[ "PYTHON" ] = sys.executable
else:
os.environ[ "PYTHON" ] = r"C:\Python33\python.exe"
else:
assert False, use_python
else:
os.environ[ "PYTHON" ] = use_python
if options.basic_tests:
print( "Running the basic tests with options '%s' with %s:" % ( flags, use_python ) )
setExtraFlags( where, "basics", flags )
executeSubTest( "./tests/basics/run_all.py search" )
if options.syntax_tests:
print( "Running the syntax tests with options '%s' with %s:" % ( flags, use_python ) )
setExtraFlags( where, "syntax", flags )
executeSubTest( "./tests/syntax/run_all.py search" )
if options.program_tests:
print( "Running the program tests with options '%s' with %s:" % ( flags, use_python ) )
setExtraFlags( where, "programs", flags )
executeSubTest( "./tests/programs/run_all.py search" )
if options.package_tests:
print( "Running the package tests with options '%s' with %s:" % ( flags, use_python ) )
setExtraFlags( where, "packages", flags )
executeSubTest( "./tests/packages/run_all.py search" )
if options.standalone_tests:
print( "Running the standalone tests with options '%s' with %s:" % ( flags, use_python ) )
setExtraFlags( None, "standalone", flags )
executeSubTest( "./tests/standalone/run_all.py search" )
if options.reflection_test:
print( "Running the reflection test with options '%s' with %s:" % ( flags, use_python ) )
setExtraFlags( None, "reflected", flags )
executeSubTest( "./tests/reflected/compile_itself.py search" )
if not use_python.startswith( "python3" ):
if os.path.exists( "./tests/CPython26/run_all.py" ):
if options.cpython26:
print( "Running the CPython 2.6 tests with options '%s' with %s:" % ( flags, use_python ) )
setExtraFlags( where, "26tests", flags )
executeSubTest( "./tests/CPython26/run_all.py search" )
else:
print( "The CPython2.6 tests are not present, not run." )
# Running the Python 2.7 test suite with CPython 2.6 gives little
# insight, because "importlib" will not be there and that's it.
if use_python != "python2.6":
if os.path.exists( "./tests/CPython27/run_all.py" ):
if options.cpython27:
print( "Running the CPython 2.7 tests with options '%s' with %s:" % ( flags, use_python ) )
setExtraFlags( where, "27tests", flags )
executeSubTest( "./tests/CPython27/run_all.py search" )
else:
print( "The CPython2.7 tests are not present, not run." )
if "--debug" not in flags:
# Not running the Python 3.2 test suite with CPython2.6, as that's about
# the same as CPython2.7 and won't have any new insights.
if use_python != "python2.6":
if os.path.exists( "./tests/CPython32/run_all.py" ):
if options.cpython32:
setExtraFlags( where, "32tests", flags )
executeSubTest( "./tests/CPython32/run_all.py search" )
else:
print( "The CPython3.2 tests are not present, not run." )
# Running the Python 3.3 test suite only with CPython2.x.
if not use_python.startswith( "python2" ):
if os.path.exists( "./tests/CPython33/run_all.py" ):
if options.cpython33:
setExtraFlags( where, "33tests", flags )
executeSubTest( "./tests/CPython33/run_all.py search" )
else:
print( "The CPython3.3 tests are not present, not run." )
if "NUITKA_EXTRA_OPTIONS" in os.environ:
del os.environ[ "NUITKA_EXTRA_OPTIONS" ]
assert checkExecutableCommand( "python2.6" ) or checkExecutableCommand( "python2.7" ) or checkExecutableCommand( "python3.2" ) or checkExecutableCommand( "python3.3" )
# Just the quick syntax test, full tests are run later.
if checkExecutableCommand( "python3.2" ):
executeSubTest(
"bin/nuitka --python-version=3.2 --version",
hide_stderror = True
)
if checkExecutableCommand( "python2.6" ):
execute_tests( "python2.6-debug", "python2.6", "--debug" )
else:
print( "Cannot execute tests with Python 2.6, disabled or not installed." )
if checkExecutableCommand( "python2.7" ):
execute_tests( "python2.7-debug", "python2.7", "--debug" )
else:
print( "Cannot execute tests with Python 2.7, disabled or not installed." )
# Temporary measure, the nodebug tests pass, debug doesn't for the CPython3.2
# test suite without unused closure variable removal.
if False and checkExecutableCommand( "python3.2" ):
execute_tests( "python3.2-debug", "python3.2", "--debug" )
else:
print( "Cannot execute tests with Python 3.2, disabled or not installed." )
if checkExecutableCommand( "python2.6" ):
execute_tests( "python2.6-nodebug", "python2.6", "" )
else:
print( "Cannot execute tests with Python 2.6, disabled or not installed." )
if checkExecutableCommand( "python2.7" ):
execute_tests( "python2.7-nodebug", "python2.7", "" )
else:
print( "Cannot execute tests with Python 2.7, disabled or not installed." )
if checkExecutableCommand( "python3.2" ):
execute_tests( "python3.2-nodebug", "python3.2", "" )
else:
print( "Cannot execute tests with Python 3.2, disabled or not installed." )
if checkExecutableCommand( "python3.3" ):
execute_tests( "python3.2-nodebug", "python3.3", "" )
else:
print( "Cannot execute tests with Python 3.3, disabled or not installed." )
if checkExecutableCommand( "cppcheck" ):
command = [
"cppcheck",
"-q",
"--error-exitcode=1",
"--enable=all",
"--check-config", "nuitka/build/",
"-I", "nuitka/build/include/",
"-I", "/usr/include/python2.7"
]
sys.stdout.flush()
result = subprocess.call( command )
if result != 0:
sys.exit( result )
print( "OK." )
Nuitka-0.5.0.1/misc/nuitka.bat 0000644 0001750 0001750 00000001607 12265264105 016265 0 ustar hayen hayen 0000000 0000000 @echo off
rem Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
rem
rem Part of "Nuitka", an optimizing Python compiler that is compatible and
rem integrates with CPython, but also works on its own.
rem
rem Licensed under the Apache License, Version 2.0 (the "License");
rem you may not use this file except in compliance with the License.
rem You may obtain a copy of the License at
rem
rem http://www.apache.org/licenses/LICENSE-2.0
rem
rem Unless required by applicable law or agreed to in writing, software
rem distributed under the License is distributed on an "AS IS" BASIS,
rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
rem See the License for the specific language governing permissions and
rem limitations under the License.
rem
setlocal
"%~dp0..\python" "%~dp0nuitka" %*
endlocal
Nuitka-0.5.0.1/misc/check-with-pylint 0000755 0001750 0001750 00000010177 12265270763 017604 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Disabled globally:
#
# W0232: Class has no __init__ method
# Who cares, I am using overrides that don't need to change object init a lot
# and I rarely ever made a mistake with forgetting to call __init__ of the
# parent.
#
# I0011: Locally disabling W....
# Strange one anyway, I want to locally disable stuff. And that just makes it
# a different warning. Amazing. Luckily I can decide to ignore that globally
# then.
import sys, os, subprocess
# Go its own directory, to have it easy with path knowledge.
os.chdir( os.path.dirname( os.path.abspath( __file__ ) ) )
os.chdir( ".." )
pylint_options = """
--rcfile=/dev/null
--include-ids=y
--disable=I0011,W0232
--reports=no
--persistent=no
--method-rgx=[a-z_][a-zA-Z0-9_]{2,40}$
--module-rgx=.*
--function-rgx=.*
--variable-rgx=.*
--argument-rgx=.*
--const-rgx=.*
--max-line-length=120
--no-docstring-rgx=.*
--max-module-lines=5000
--min-public-methods=0
--max-public-methods=100
--max-args=10
--max-parents=9
""".split()
from optparse import OptionParser
parser = OptionParser()
parser.add_option(
"--hide-todos", "--no-todos",
action = "store_true",
dest = "no_todos",
default = False,
help = """\
Default is %default."""
)
parser.add_option(
"--emacs",
action = "store_true",
dest = "emacs",
default = False,
help = """\
Default is %default."""
)
options, positional_args = parser.parse_args()
if os.environ.get( "TODO", 0 ) or options.no_todos:
pylint_options.append( "--notes=" )
blacklist = (
"oset.py",
"odict.py",
"SyntaxHighlighting.py",
"TreeDisplay.py"
)
def executePyLint( filename ):
command = "pylint %s %s %s" % (
" ".join( pylint_options ),
os.environ.get( "PYLINT_EXTRA_OPTIONS", "" ),
filename
)
process = subprocess.Popen(
args = command,
stdout = subprocess.PIPE,
stderr = subprocess.STDOUT,
shell = True
)
stdout, _stderr = process.communicate()
exit_code = process.returncode
assert not _stderr
if stdout:
for line in stdout.split( b"\n" ):
output = line.decode()
if options.emacs and ":" in output:
parts = output.split( ":" )
output = filename + ":" + parts[1].split(",")[0].strip() + ": " + \
parts[0] + " " + ":".join( parts[2:] )
sys.stderr.write( output + "\n" )
else:
print( output )
sys.stdout.flush()
if "PYTHONPATH" not in os.environ:
os.environ[ "PYTHONPATH" ] = "."
if positional_args:
for positional_arg in positional_args:
executePyLint( positional_arg )
else:
executePyLint( "bin/nuitka" )
for dirpath, dirnames, filenames in os.walk( "nuitka" ):
dirnames.sort()
if "inline_copy" in dirnames:
dirnames.remove( "inline_copy" )
filenames.sort()
for filename in filenames:
if not filename.endswith( ".py" ):
continue
# Skip temporary files from flymake mode of Emacs.
if filename.endswith( "_flymake.py" ):
continue
# Skip temporary files from unsaved files of Emacs.
if filename.startswith( ".#" ):
continue
if filename not in blacklist:
executePyLint( os.path.join( dirpath, filename ) )
Nuitka-0.5.0.1/misc/Logo/ 0000755 0001750 0001750 00000000000 12265271051 015174 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/misc/Logo/Nuitka-Logo-Symbol.svg 0000644 0001750 0001750 00000016351 12265044767 021333 0 ustar hayen hayen 0000000 0000000
Nuitka-0.5.0.1/misc/Logo/Nuitka-Logo-Vertical.svg 0000644 0001750 0001750 00000042775 12265044767 021650 0 ustar hayen hayen 0000000 0000000
Nuitka-0.5.0.1/misc/Logo/Nuitka-Logo-Horizontal.svg 0000644 0001750 0001750 00000016231 12265044767 022214 0 ustar hayen hayen 0000000 0000000
Nuitka-0.5.0.1/misc/nuitka-run.bat 0000644 0001750 0001750 00000001613 12265264105 017064 0 ustar hayen hayen 0000000 0000000 @echo off
rem Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
rem
rem Part of "Nuitka", an optimizing Python compiler that is compatible and
rem integrates with CPython, but also works on its own.
rem
rem Licensed under the Apache License, Version 2.0 (the "License");
rem you may not use this file except in compliance with the License.
rem You may obtain a copy of the License at
rem
rem http://www.apache.org/licenses/LICENSE-2.0
rem
rem Unless required by applicable law or agreed to in writing, software
rem distributed under the License is distributed on an "AS IS" BASIS,
rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
rem See the License for the specific language governing permissions and
rem limitations under the License.
rem
setlocal
"%~dp0..\python" "%~dp0nuitka-run" %*
endlocal
Nuitka-0.5.0.1/Developer_Manual.rst 0000644 0001750 0001750 00000303216 12265264105 017324 0 ustar hayen hayen 0000000 0000000 Nuitka Developer Manual
~~~~~~~~~~~~~~~~~~~~~~~
.. image:: images/Nuitka-Logo-Symbol.png
.. contents::
.. raw:: pdf
PageBreak oneColumn
SetPageCounter 1
The purpose of this developer manual is to present the current design of Nuitka,
the coding rules, and the motivations for choices made. It is intended to be a
guide to the source code, and to give explanations that don't fit into the
source code in comments form.
It should be used as a reference for the process of planning and documenting
decisions we made. Therefore we are e.g. presenting here the type inference
plans before implementing them. And we update them as we proceed.
It grows out of discussions and presentations made at conferences as well as
private conversations or discussions on the mailing list or bug tracker.
Milestones
==========
1. Feature parity with CPython, understand all the language construct and behave
absolutely compatible.
Feature parity has been reached for CPython 2.6 and 2.7. We do not target any
older CPython release. For CPython 3.2 it also has been reached. We do not
target older CPython 3.1 and 3.0 releases.
This milestone was reached.
2. Create the most efficient native code from this. This means to be fast with
the basic Python object handling.
This milestone was reached.
3. Then do constant propagation, determine as many values and useful constraints
as possible at compile time and create more efficient code.
This milestone is considered almost reached.
4. Type inference, detect and special case the handling of strings, integers,
lists in the program.
This milestone is considered in progress
5. Add interfacing to C code, so Nuitka can turn a ``ctypes`` binding into an
efficient binding as written with C.
This milestone is planned only.
6. Add hints module with a useful Python implementation that the compiler can
use to learn about types from the programmer.
This milestone is planned only.
Version Numbers
===============
For Nuitka we use a defensive version numbering system to indicate that it is
not yet ready for everything. We have defined milestones and the version numbers
should express which of these, we consider done.
- So far:
Before milestone 1, we used "0.1.x" version numbers. After reaching it, we
used "0.2.x" version numbers.
Before milestone 2 and 3, we used "0.3.x" version numbers. After almost
reaching 3, and beginning with 4, we use "0.4.x" version numbers.
- Future:
When we start to have sufficient amount of type inference in a stable release,
that will be "0.5.x" version numbers. With ``ctypes`` bindings in a sufficient
state it will be "0.6.x".
- Final:
We will then round it up and call it "Nuitka 1.0" when this works as expected
for a bunch of people. The plan is to reach this goal during 2014. This is
based on lots of assumptions that may not hold up though.
Of course, this may be subject to change.
Current State
=============
Nuitka top level works like this:
- ``nuitka.tree.Building`` outputs node tree
- ``nuitka.optimization`` enhances it as best as it can
- ``nuitka.finalization`` marks the tree for code generation
- ``nuitka.codegen.CodeGeneration`` creates identifier objects and code snippets
- ``nuitka.codegen.Generator`` knows how identifiers and code is constructed
- ``nuitka.MainControl`` keeps it all together
This design is intended to last.
Regarding Types, the state is:
- Types are always ``PyObject *``, implicitly.
- The only more specific use of type is "compile time constant", which can be
used to predict some operations, conditions, etc.
- Every operation is expected to have ``PyObject *`` as result, if it is not a constant,
then we know nothing about it.
The limitation to only ``PyObject *`` will go away.
Coding Rules
============
These rules should generally be adhered when working on Nuitka code. It's not
library code and it's optimized for readability, and avoids all performance
optimization for itself.
Line Length
-----------
No more than 120 characters. Screens are wider these days, but most of the code
aims at keeping the lines below 100.
Indentation
-----------
No tabs, 4 spaces, no trailing white space.
Identifiers
-----------
Classes are camel case with leading upper case. Methods are with leading verb in
lower case, but also camel case. Around braces, and after comma, there is spaces
for better readability. Variables and parameters are lower case with "_" as a
separator.
.. code-block:: python
class SomeClass:
def doSomething(some_parameter):
some_var = ( "foo", "bar" )
Base classes that are abstract end in ``Base``, so that a meta class can use
that convention.
Function calls use keyword argument preferably. These are slower in CPython, but
more readable:
.. code-block:: python
return Generator.getSequenceCreationCode(
sequence_kind = sequence_kind,
element_identifiers = identifiers,
context = context
)
The ``=`` are all aligned to the longest parameter names without extra spaces
for it.
When the names don't add much value, sequential calls should be done, but
ideally with one value per line:
.. code-block:: python
return Identifier(
"TO_BOOL( %s )" % identifier.getCodeTemporaryRef(),
0
)
Here, ``Identifier`` will be so well known that the reader is expected to know
the argument names and their meaning, but it would be still better to add them.
Contractions should span across multiple lines for increased readability:
.. code-block:: python
result = [
"PyObject *decorator_%d" % ( d + 1 )
for d in
range( decorator_count )
]
Module/Package Names
--------------------
Normal modules are named in camel case with leading upper case, because their of
role as singleton classes. The difference between a module and a class is small
enough and in the source code they are also used similarly.
For the packages, no real code is allowed in them and they must be lower case,
like e.g. ``nuitka`` or ``codegen``. This is to distinguish them from the
modules.
Packages shall only be used to group packages. In ``nuitka.codegen`` the code
generation packages are located, while the main interface is
``nuitka.codegen.CodeGeneration`` and may then use most of the entries as local
imports.
The use of a global package ``nuitka``, originally introduced by Nicolas, makes
the packaging of Nuitka with ``distutils`` etc. easier and lowers the
requirements on changes to the ``sys.path`` if necessary.
.. note::
There are not yet enough packages inside Nuitka, feel free to propose changes
as you see fit.
Names of modules should be plurals if they contain classes. Example is ``Nodes``
contains ``Node`` classes.
Prefer list contractions over built-ins
---------------------------------------
This concerns ``map``, ``filter``, and ``apply``. Usage of these built-ins is
highly discouraged within Nuitka source code. Using them is considered worth a
warning by "PyLint" e.g. "Used builtin function 'map'". We should use list
comprehensions instead, because they are more readable.
List contractions are a generalization for all of them. We love readable and
with Nuitka as a compiler will there won't be any performance difference at all.
I can imagine that there are cases where list comprehensions are faster because
you can avoid to make a function call. And there may be cases, where map is
faster, if a function must be called. These calls can be very expensive, and if
you introduce a function, just for ``map``, then it might be slower.
But of course, Nuitka is the project to free us from what is faster and to allow
us to use what is more readable, so whatever is faster, we don't care. We make
all options equally fast and let people choose.
For Nuitka the choice is list contractions as these are more easily changed and
readable.
Look at this code examples from Python:
.. code-block:: python
class A:
def getX(self):
return 1
x = property( getX )
class B(A):
def getX(self):
return 2
A().x == 1 # True
B().x == 1 # True (!)
This pretty much is what makes properties bad. One would hope ``B().x`` to be
``2``, but instead it's not changed. Because of the way properties take the
functions and not members, and because they then are not part of the class, they
cannot be overloaded without re-declaring them.
Overloading is then not at all obvious anymore. Now imagine having a setter and
only overloading the getter. How to you easily update the property?
So, that's not likable about them. And then we are also for clarity in these
internal APIs too. Properties try and hide the fact that code needs to run and
may do things. So lets not use them.
For an external API you may exactly want to hide things, but internally that has
no use, and in Nuitka, every API is internal API. One exception may be the
``hints`` module, which will gladly use such tricks for easier write syntax.
The "git flow" model
====================
* The flow was is used for releases and occasionally subsequent hot fixes.
A few feature branches were used so far. It allows for quick delivery of fixes
to both the stable and the development version, supported by a git plugin,
that can be installed via "apt-get install git-flow" on latest Debian Testing
at least.
* Stable (master branch)
The stable version, is expected to pass all the tests at all times and is
fully supported. As soon as bugs are discovered, they are fixed as hot fixes,
and then merged to develop by the "git flow" automatically.
* Development (develop branch)
The future release, supposedly in almost ready for release state at nearly all
times, but this is as strict. It is not officially supported, and may have
problems and at times inconsistencies.
* Feature Branches
On these long lived developments that extend for multiple release cycles or
contain changes that break Nuitka temporarily. They need not be functional at
all.
Current Feature branches:
- ``feature/ctypes_annotation``: Achieve the inlining of ctypes calls, so they
become executed at no speed penalty compared to direct calls via extension
modules. This being fully CPython compatible and pure Python, is considered
the "Nuitka" way of creating extension modules that provide bindings.
Checking the Source
===================
The checking for errors is currently done with "PyLint". In the future, Nuitka
will gain the ability to present its findings in a similar way, but this is not
a priority, and not there yet.
So, we currently use "PyLint" with options defined in a script.
.. code-block:: sh
./misc/check-with-pylint --hide-todos
Ideally the above command gives no warnings. This has not yet been reached. The
existing warnings often still serve as a kind of "TODO" items. We are not white
listing them, because they indicate a problem that should be solved.
If you submit a patch, it would be good if you checked that it doesn't introduce
new warnings, but that is not strictly required. it will happen before release,
and that is considered enough. You probably are already aware of the beneficial
effects.
Running the Tests
=================
This section describes how to run Nuitka tests.
Running all Tests
-----------------
The top level access to the tests is as simple as this:
.. code-block:: bash
./misc/check-release
For fine grained control, it has the following options::
-h, --help show this help message and exit
--skip-basic-tests The basic tests, execute these to check if Nuitka is
healthy. Default is True.
--skip-syntax-tests The syntax tests, execute these to check if Nuitka
handles Syntax errors fine. Default is True.
--skip-program-tests The programs tests, execute these to check if Nuitka
handles programs, e.g. import recursions, etc. fine.
Default is True.
--skip-reflection-test
The reflection test compiles Nuitka with Nuitka, and
then Nuitka with the compile Nuitka and compares the
outputs. Default is True.
--skip-cpython26 The standard CPython2.6 test suite. Execute this for
all corner cases to be covered. With Python 2.7 this
covers exception behavior quite well. Default is True.
--skip-cpython27 The standard CPython2.7 test suite. Execute this for
all corner cases to be covered. With Python 2.6 these
are not run. Default is True.
--skip-cpython32 The standard CPython3.2 test suite. Execute this for all
corner cases to be covered. With Python 2.x these are not
run. Default is True.
You will only run the CPython test suites, if you have the submodules of the
Nuitka git repository checked out. Otherwise, these will be skipped
automatically with a warning that they are not available.
The policy is generally, that ``./misc/check-release`` running and passing all
tests shall be considered sufficient for a release.
Basic Tests
-----------
You can run the "basic" tests like this:
.. code-block:: bash
./tests/basics/run_all.py search
These tests normally give sufficient coverage to assume that a change is
correct, if these tests pass. To control the Python version used for testing,
you can set the ``PYTHON`` environment variable to e.g. "python3.2", or execute
the "run_all.py" with the intended version, it is portable across all supported
Python versions.
Syntax Tests
------------
Then there are "syntax" tests, i.e. language constructs that need to give a
syntax error.
It sometimes happens that Nuitka must do this itself, because the ``ast.parse``
don't see the problem. Using ``global`` on a function argument is an example of
this. These tests make sure that the errors of Nuitka and CPython are totally
the same for this:
.. code-block:: bash
./tests/syntax/run_all.py search
Program Tests
-------------
Then there are small programs tests, that exercise all kinds of import tricks
and problems with inter-module behavior. These can be run like this:
.. code-block:: bash
./tests/programs/run_all.py search
Compile Nuitka with Nuitka
--------------------------
And there is the "compile itself" or "reflected" test. This test makes Nuitka
compile itself and compare the resulting C++, which helps to find
in-determinism. The test compiles every module of Nuitka into an extension
module and all of Nuitka into a single binary.
That test case also gives good coverage of the ``import`` mechanisms, because
Nuitka uses a lot of packages.
.. code-block:: bash
./tests/reflected/compile_itself.py
Design Descriptions
===================
These should be a lot more and contain graphics from presentations given. It
will be filled in, but not now.
Nuitka Logo
-----------
The logo was submitted by "dr. Equivalent". It's source is contained in
``misc/Logo`` where 3 variants of the logo in SVG are placed.
* Symbol only (symbol)
.. image:: images/Nuitka-Logo-Symbol.png
* Text next to symbol (horizontal)
.. image:: images/Nuitka-Logo-Horizontal.png
* Text beneath symbol (vertical)
.. image:: images/Nuitka-Logo-Vertical.png
From these logos, PNG images, "favicons", and a BMP file for the Windows
installer are derived.
The exact ImageMagick commands are in ``misc/make-doc.py``, but are now executed
each time, the commands are also replicated here:
.. code-block:: bash
convert -background none misc/Logo/Nuitka-Logo-Symbol.svg images/Nuitka-Logo-Symbol.png
convert -background none misc/Logo/Nuitka-Logo-Vertical.svg images/Nuitka-Logo-Vertical.png
convert -background none misc/Logo/Nuitka-Logo-Horizontal.svg images/Nuitka-Logo-Horizontal.png
optipng -o2 images/Nuitka-Logo-Symbol.png
optipng -o2 images/Nuitka-Logo-Vertical.png
optipng -o2 images/Nuitka-Logo-Horizontal.png
convert -background grey -resize 152x261 misc/Logo/Nuitka-Logo-Vertical.svg -alpha background images/Nuitka-Logo-WinInstaller.bmp
Choice of the Target Language
-----------------------------
* Choosing the target language, is an important decision
* The portability of Nuitka is decided here
* Other factors:
* How difficult is it to generate the code?
* Does the Python C-API have bindings?
* Is that language known?
* Does the language aid to find bugs?
* These candidates were considered
* C++03, C++11, Ada
.. table:: Requirement to Language matrix:
===================== ====== ========= =========
Requirement\\Language C++03 C++11 Ada
===================== ====== ========= =========
Portable Yes No [1]_ Yes
--------------------- ------ --------- ---------
Knowledge Yes No [2]_ Yes
--------------------- ------ --------- ---------
Python C-API Yes Yes No [3]_
--------------------- ------ --------- ---------
Runtime checks No No Yes [4]_
--------------------- ------ --------- ---------
Code Generation Hard Easy Harder
===================== ====== ========= =========
_`1`:: C++11 is not fully supported from any compiler
(temporary problem)
_`2`:: Not a whole lot of people have C++11 knowledge. My *only* C++11 code was
that in Nuitka.
_`3`:: The Python C-API for Ada would have to be created by us, possible just
big project by itself.
_`4`:: Runtime checks exist only for Ada in that quality. I miss automatic
``CONSTRAINT_ERROR`` exceptions, for data structures with validity indicators,
where in other languages, I need to check myself.
The *decision for C++03* is ultimately:
* for portability
* for language knowledge
All of these are important advantages.
For C++11 initially spoke easy code generation:
* variadic templates
* raw strings
Yet, as it turns out, variadic templates do not help at all with evaluation
order, so that code that used it, needed to be changed to generating instances
of their code. And raw strings turned out to be not as perfect as one wants to
be, and solving the problem with C++03 is feasible too, even if not pretty.
For Ada would have spoken the time savings through run time checks, which would
have shortened some debugging sessions quite some. But building the Python C-API
bindings on our own, and potentially incorrectly, would have eaten that up.
Use of Scons internally
-----------------------
Nuitka does not make its users interface with Scons at all, it's purely used
internally. Nuitka itself, being pure Python, will run without any build process
just fine.
For interfacing, there is the module ``nuitka.build.SconsInterface`` that will
support calling scons - potentially from an inline copy on Windows or when using
source releases - and passing arguments to it. These arguments are passed as
``key=value``, and decoded in the scons file of Nuitka.
The scons file is named ``SingleExe.scons`` for lack of better name. It's really
wrong now, but we have yet to find a better name. It once expressed the
intention to be used to create executables, but the same works for modules too.
It supports operation in multiple modes, and modules is just one of them. It
runs outside of Nuitka process scope, even with a different Python version
potentially, so all the information must be passed on the command line.
What follows is the (lengthy) list of arguments that the scons file processes:
* ``source_dir``
Where is the generated C++ source code. Scons will just compile everything it
finds there. No list of files is passed.
* ``nuitka_src``
Where do the include files and static C++ parts of Nuitka live. These provide
e.g. the implementation of compiled function, generators, and other helper
codes, this will point to where ``nuitka.build`` package lives normally.
* ``result_base``
This is not a full name, merely the basename for the result to be produced,
but with path included, and the suffix comes from module or executable mode.
* ``module_mode``
Build a module instead of a program.
* ``debug_mode``
Enable debug mode, which is a mode, where Nuitka tries to help identify errors
in itself, and will generate less optimal code. This also asks for warnings,
and makes the build fail if there are any.
* ``python_debug``
Compile and link against Python debug mode, which does assertions and extra
checks, to identify errors, mostly related to reference counting. May make the
build fail, if no debug build library of CPython is available. On Windows it
typically is not installed.
* ``optimize_mode``
Optimization mode, enable as much as currently possible. This refers to
building the binary.
* ``full_compat_mode``
Full compatibility, even where it's stupid, i.e. do not provide information,
even if available, in order to assert maximum compatibility. Intended to
control level of compatability to absurd.
* ``experimental_mode``
Do things that are not yet accepted to be safe.
* ``lto_mode``
Make use of link time optimization of g++ compiler if available and known good
with the compiler in question. So far, this was not found to make major
differences.
* ``win_disable_console``
Windows subsystem mode: Disable console for windows builds.
* ``unstriped_mode``
Unstriped mode: Do not remove debug symbols.
* ``clang_mode``
Clang compiler mode, default on MacOS X and FreeBSD, optional on Linux.
* ``mingw_mode``
MinGW compiler mode, optional and interesting to Windows only.
* ``standalone_mode``
Building a standalone distribution for the binary.
* ``show_scons``
Show scons mode, output information about Scons operation. This will e.g. also
output the actual compiler used, output from compilation process, and
generally debug information relating to be build process.
* ``python_prefix``
Home of Python to be compiled against, used to locate headers and libraries.
* ``target_arch``
Target architecture to build.
* ``icon_path``
The icon to use for Windows programs if given.
Locating Modules and Packages
------------------------------
The search for of modules used is driven by ``nuitka.Importing`` module.
* From the module documentation
The actual import of a module may already execute code that changes
things. Imagine a module that does ``os.system()``, it will be done. People
often connect to databases, and these kind of things, at import time. Not a
good style, but it's being done.
Therefore CPython exhibits the interfaces in an ``imp`` module in standard
library, which one can use those to know ahead of time, what file import would
load. For us unfortunately there is nothing in CPython that is easily
accessible and gives us this functionality for packages and search paths
exactly like CPython does, so we implement here a multi step search process
that is compatible.
This approach is much safer of course and there is no loss. To determine if
it's from the standard library, one can abuse the attribute ``__file__`` of
the ``os`` module like it's done in ``isStandardLibraryPath`` of this module.
* Role
This module serves the recursion into modules and analysis if a module is a
known one. It will give warnings for modules attempted to be located, but not
found. These warnings are controlled by a while list inside the module.
Hooking for module ``import`` process
-------------------------------------
Currently, in created code, for every ``import`` variable a normal
``__import__()`` call is executed. The "ModuleUnfreezer.cpp" (located in
"nuitka/build/static_src") provides the implementation of a ``sys.meta_path``
hook.
This one allows us to have the Nuitka provided module imported even when
imported by non-compiled code.
.. note::
Of course it would make sense to compile time detect which module it is that
is being imported and then to make it directly. At this time, we don't have
this inter-module optimization yet, mid-term it should become easy to add.
Supporting ``__class__`` of Python3
-----------------------------------
In Python3 the handling of ``__class__`` and ``super`` is different from
Python2. It used to be a normal variable, and now the following things have
changed.
* The use of the ``super`` variable name triggers the addition of a closure
variable ``__class__``, as can be witnessed by the following code:
.. code-block:: python
class X:
def f1(self):
print( locals() )
def f2(self):
print( locals() )
super
x = X()
x.f1()
x.f2()
.. code-block:: python
{'self': <__main__.X object at 0x7f1773762390>}
{'self': <__main__.X object at 0x7f1773762390>, '__class__': }
* This value of ``__class__`` is also available in the child functions.
* The parser marks up code objects usage of "super". It doesn't have to be a
call, it can also be a local variable. If the ``super`` builtin is assigned to
another name and that is used without arguments, it won't work unless
``__class__`` is taken as a closure variable.
* As can be seen in the CPython3.2 code, the closure value is added after the
class creation is performed.
* It appears, that only functions locally defined to the class are affected and
take the closure.
This left Nuitka with the strange problem, of how to emulate that.
The solution is this:
* Under Python3, usage of ``__class__`` as a reference in a function body that
is not a class dictionary creation, marks it up via
``markAsClassClosureTaker``.
* Functions that are marked up, will be forced to reference variable to
``__class__``.
.. note::
This one should be optimized away later if not used. Currently we have "no
unused closure variable" detection, but it would cover it.
* When recognizing calls to ``super`` without arguments, make the arguments into
variable reference to ``__class__`` and potentially ``self`` (actually first
argument name).
* Class dictionary definitions are added.
These are special direct function calls, ready to propagate also "bases" and
"metaclass" values, which need to be calculated outside.
The function bodies used for classes will automatically store ``__class__`` as
a shared local variable, if anything uses it. And if it's not assigned by user
code, it doesn't show up in the "locals()" used for dictionary creation.
Existing ``__class__`` local variable values are in fact provided as closure,
and overridden with the built class , but they should be used for the closure
giving, before the class is finished.
So ``__class__`` will be local variable of the class body, until the class is
built, then it will be the ``__class__`` itself.
Frame Stack
-----------
In Python, every function, class, and module has a frame. It creates created
when the scope it entered, and there is a stack of these at run time, which
becomes visible in tracebacks in case of exceptions.
The choice of Nuitka is to make this non-static elements of the node tree, that
are as such subject to optimization. In cases, where they are not needed, they
may be removed.
Consider the following code.
.. code-block:: python
def f():
if someNotRaisingCall():
return somePotentiallyRaisingCall()
else:
return None
In this example, the frame is not needed for all the code, because the condition
checked wouldn't possibly raise at all. The idea is the make the frame guard
explicit and then to move it downwards in the tree, whenever possible.
So we start out with code like this one:
.. code-block:: python
def f():
with frame_guard( "f" ):
if someNotRaisingCall():
return somePotentiallyRaisingCall()
else:
return None
This is to be optimized into:
.. code-block:: python
def f():
if someNotRaisingCall():
with frame_guard( "f" ):
return somePotentiallyRaisingCall()
else:
return None
Notice how the frame guard taking is limited and may be avoided, or in best
cases, it might be removed completely. Also this will play a role when in-lining
function, it will not be lost or need any extra care.
Parameter Parsing
-----------------
The parsing of parameters is very convoluted in Python, and doing it in an
compatible way is not that easy. This is a description of the required process,
for easier overview.
Input
+++++
The input is an argument tuple (type is fixed), which contains the positional
arguments, and potentially an argument dictionary (type is fixed, but could also
be ``NULL``, indicating no keyword arguments.
Keyword dictionary
++++++++++++++++++
The keyword argument dictionary is checked first. Anything in there, that cannot
be associated, either raises an error, or is added to a potentially given star
dict argument. So there are two major cases.
* No star dict argument: Iterate over dictionary, and assign or raise errors.
This check covers extra arguments given.
* With star dict argument: Iterate over dictionary, and assign or raise errors.
Interesting case for optimization are no positional arguments, then no check
is needed, and the keyword argument dictionary could be used as the star
argument. Should it change, a copy is needed though.
What's noteworthy here, is that in comparison of the keywords, we can hope that
they are the same value as we use. The interning of strings increases chances
for non-compiled code to do that, esp. for short names.
We then can do a simple ``==`` comparison and only fall back to real string
comparisons, after all of these failed. That means more code, but also a lot
faster code in the positive case.
Argument tuple
++++++++++++++
After this completed, the argument tuple is up for processing. The first thing
it needs to do is to check if it's too many of them, and then to complain.
For arguments in Python2, there is the possibility of them being nested, in
which case they cannot be provided in the keyword dictionary, and merely should
get picked from the argument tuple.
Otherwise, the length of the argument tuple should be checked against its
position and if possible, values should be taken from there. If it's already set
(from the keyword dictionary), raise an error instead.
Language Conversions to make things simpler
-------------------------------------------
There are some cases, where the Python language has things that can in fact be
expressed in a simpler or more general way, and where we choose to do that at
either tree building or optimization time.
The ``assert`` statement
++++++++++++++++++++++++
The ``assert`` statement is a special statement in Python, allowed by the
syntax. It has two forms, with and without a second argument. The later is
probably less known, as is the fact that raise statements can have multiple
arguments too.
The handling in Nuitka is:
.. code-block:: python
assert value
# Absolutely the same as:
if not value:
raise AssertionError
.. code-block:: python
assert value, raise_arg
# Absolutely the same as:
if not value:
raise AssertionError, raise_arg
This makes assertions absolutely the same as a raise exception in a conditional
statement.
This transformation is performed at tree building already, so Nuitka never knows
about ``assert`` as an element and standard optimizations apply. If e.g. the
truth value of the assertion can be predicted, the conditional statement will
have the branch statically executed or removed.
The "comparison chain" expressions
++++++++++++++++++++++++++++++++++
.. code-block:: python
a < b > c < d
# With "temp variables" and "assignment expressions", absolutely
# the same as:
a < ( tmp_b = b ) and tmp_b > ( tmp_c = c ) and ( tmp_c < d )
This transformation is performed at tree building already. The temporary
variables keep the value for the potential read in the same expression. The
syntax is not Python, and only pseudo language to expression the internal
structure of the node tree after the transformation.
This useful "keeper" variables that enable this transformation and allow to
express the short circuit nature of comparison chains by using ``and``
operations.
The ``execfile`` built-in
+++++++++++++++++++++++++
Handling is:
.. code-block:: python
execfile( filename )
# Basically the same as:
exec( compile( open( filename ).read() ), filename, "exec" )
.. note::
This allows optimizations to discover the file opening nature easily and
apply file embedding or whatever we will have there one day.
This transformation is performed when the ``execfile`` builtin is detected as
such during optimization.
Generator expressions with ``yield``
++++++++++++++++++++++++++++++++++++
These are converted at tree building time into a generator function body that
yields the iterator given, which is the put into a for loop to iterate, created
a lambda function of and then called with the first iterator.
That eliminates the generator expression for this case. It's a bizarre construct
and with this trick needs no special code generation.
Function Decorators
+++++++++++++++++++
When one learns about decorators, you see that:
.. code-block:: python
@decorator
def function():
pass
# Is basically the same as:
def function():
pass
function = decorator( function )
The only difference is the assignment to function. In the ``@decorator`` case,
if the decorator fails with an exception, the name ``function`` is not assigned.
Therefore in Nuitka this assignment is from a "function body expression" and
only the last decorator returned value is assigned to the function name.
This removes the need for optimization and code generation to support decorators
at all. And it should make the two variants optimize equally well.
In-place Assignments
++++++++++++++++++++
In-place assignments are re-formulated to an expression using temporary
variables.
These are not as much a reformulation of ``+=`` to ``+``, but instead one which
makes it explicit that the assign target may change its value.
.. code-block:: python
a += b
.. code-block:: python
_tmp = a.__iadd__( b )
if a is not _tmp:
a = _tmp
Using ``__iadd__`` here to express that not the ``+``, but the in-place variant
``iadd`` is used instead. The ``is`` check may be optimized away depending on
type and value knowledge later on.
Complex Assignments
+++++++++++++++++++
Complex assignments are defined as those with multiple targets to assign from a
single source and are re-formulated to such using a temporary variable and
multiple simple assignments instead.
.. code-block:: python
a = b = c
.. code-block:: python
_tmp = c
b = _tmp
a = _tmp
del _tmp
This is possible, because in Python, if one assignment fails, it can just be
interrupted, so in fact, they are sequential, and all that is required is to not
calculate ``c`` twice, which the temporary variable takes care of.
Unpacking Assignments
+++++++++++++++++++++
Unpacking assignments are re-formulated to use temporary variables as well.
.. code-block:: python
a, b.attr, c[ind] = d = e, f, g = h()
Becomes this:
.. code-block:: python
_tmp = h()
_iter1 = iter( _tmp )
_tmp1 = unpack( _iter1, 3 )
_tmp2 = unpack( _iter1, 3 )
_tmp3 = unpack( _iter1, 3 )
unpack_check( _iter1 )
a = _tmp1
b.attr = _tmp2
c[ind] = _tmp3
d = _tmp
_iter2 = iter( _tmp )
_tmp4 = unpack( _iter2, 3 )
_tmp5 = unpack( _iter2, 3 )
_tmp6 = unpack( _iter2, 3 )
unpack_check( _iter1 )
e = _tmp4
f = _tmp5
g = _tmp6
That way, the unpacking is decomposed into multiple simple statementy. It will
be the job of optimizations to try and remove unnecessary unpacking, in case
e.g. the source is a known tuple or list creation.
.. note::
The ``unpack`` is a special node which is a form of ``next`` that will raise
a ``ValueError`` when it cannot get the next value, rather than a
``StopIteration``. The message text contains the number of values to unpack,
therefore the integer argument.
.. note::
The ``unpack_check`` is a special node that raises a ``ValueError`` exception
if the iterator is not finished, i.e. there are more values to unpack.
With Statements
+++++++++++++++
The ``with`` statements are re-formulated to use temporary variables as
well. The taking and calling of ``__enter__`` and ``__exit__`` with arguments,
is presented with standard operations instead. The promise to call ``__exit__``
is fulfilled by ``try``/``except`` clause instead.
.. code-block:: python
with some_context as x:
something( x )
.. code-block:: python
tmp_source = some_context
# Actually it needs to be "special look-up" for Python2.7, so attribute
# look-up won't be exactly what is there.
tmp_exit = tmp_source.__exit__
# This one must be held for the whole with statement, it may be assigned
# or not, in our example it is. If an exception occurs when calling
# ``__enter__``, the ``__exit__`` should not be called.
tmp_enter_result = tmp_source.__enter__()
# Indicator variable to know if "tmp_exit" has been called.
tmp_indicator = False
try:
# Now the assignment is to be done, if there is any name for the
# manager given, this may become multiple assignment statements and
# even unpacking ones.
x = tmp_enter_result
# Then the code of the "with" block.
something( x )
except Exception:
# Note: This part of the code must not set line numbers, which we
# indicate with special source code references, which we call "internal".
# Otherwise the line of the frame would get corrupted.
tmp_indicator = True
if not tmp_exit( *sys.exc_info() ):
raise
finally:
if not tmp_indicator
# Call the exit if no exception occurred with all arguments
# as "None".
tmp_exit( None, None, None )
.. note::
We don't refer really to ``sys.exc_info()`` at all, instead, we have
references to the current exception type, value and trace, taken directory
from the caught exception object on the C++ level.
If we had the ability to optimize ``sys.exc_info()`` to do that, we could use
the same transformation, but right now we don't have it.
For Loops
+++++++++
The for loops use normal assignments and handle the iterator that is implicit in
the code explicitly.
.. code-block:: python
for x,y in iterable:
if something( x ):
break
else:
otherwise()
This is roughly equivalent to the following code:
.. code-block:: python
_iter = iter( iterable )
_no_break_indicator = False
while True:
try:
_tmp_value = next( _iter )
except StopIteration:
# Set the indicator that the else branch may be executed.
_no_break_indicator = True
# Optimization should be able to tell that the else branch is run
# only once.
break
# Normal assignment re-formulation applies to this assignment of course.
x, y = _tmp_value
del _tmp_value
if something( x ):
break
if _no_break_indicator:
otherwise()
.. note::
The ``_iter`` temporary variable is of course in a temp block and the ``x,
y`` assignment is the normal is of course re-formulation of an assignment
that cannot fail.
The ``try``/``except`` is detected to allow to use a variant of ``next`` that
throws no C++ exception, but instead to use ``ITERATOR_NEXT`` and which
returns NULL in that case, so that the code doesn't really have any Python
level exception handling going on.
While Loops
+++++++++++
Loops in Nuitka have no condition attached anymore, so while loops are
re-formulated like this:
.. code-block:: python
while condition:
something()
.. code-block:: python
while True:
if not condition:
break
something()
This is to totally remove the specialization of loops, with the condition moved
to the loop body in a conditional statement, which contains a break statement.
That makes it clear, that only break statements exit the loop, and allow for
optimization to remove always true loop conditions, without concerning code
generation about it, and to detect such a situation, consider e.g. endless
loops.
.. note::
Loop analysis can therefore work on a reduced problem (which ``break``
statements are executed under which conditions) and be very general, but it
cannot take advantage of the knowledge encoded directly anymore. The fact
that the loop body may not be entered at all, if the condition is not met, is
something harder to discover.
Exception Handler Values
++++++++++++++++++++++++
Exception handlers in Python may assign the caught exception value to a variable
in the handler definition.
.. code-block:: python
try:
something()
except Exception as e:
handle_it()
That is equivalent to the following:
.. code-block:: python
try:
something()
except Exception:
e = sys.exc_info()[1]
handle_it()
Of course, the value of the current exception, use special references for
assignments, that access the C++ and don't go via ``sys.exc_info`` at all, these
are called ``CaughtExceptionValueRef``.
Statement ``try``/``except`` with ``else``
++++++++++++++++++++++++++++++++++++++++++
Much like ``else`` branches of loops, an indicator variable is used to indicate
the entry into any of the exception handlers.
Therefore, the ``else`` becomes a real conditional statement in the node tree,
checking the indicator variable and guarding the execution of the ``else``
branch.
Class Creation (Python2)
++++++++++++++++++++++++
Classes in Python2 have a body that only serves to build the class dictionary
and is a normal function otherwise. This is expressed with the following
re-formulation:
.. code-block:: python
# in module "SomeModule"
# ...
class SomeClass(SomeBase, AnotherBase)
""" This is the class documentation. """
some_member = 3
.. code-block:: python
def _makeSomeClass:
# The module name becomes a normal local variable too.
__module__ = "SomeModule"
# The doc string becomes a normal local variable.
__doc__ = """ This is the class documentation. """
some_member = 3
return locals()
# force locals to be a writable dictionary, will be optimized away, but
# that property will stick. This is only to express, that locals(), where
# used will be writable to.
exec ""
SomeClass = make_class("SomeClass", (SomeBase, AnotherBase), _makeSomeClass())
That is roughly the same, except that ``_makeSomeClass`` is *not* visible to its
child functions when it comes to closure taking, which we cannot express in
Python language at all.
Therefore, class bodies are just special function bodies that create a
dictionary for use in class creation. They don't really appear after the tree
building stage anymore. The type inference will of course have to become able to
understand ``make_class`` quite well, so it can recognize the created class
again.
Class Creation (Python3)
++++++++++++++++++++++++
In Python3, classes are a complicated way to write a function call, that can
interact with its body. The body starts with a dictionary provided by the
metaclass, so that is different, because it can ``__prepare__`` a non-empty
locals for it, which is hidden away in "prepare_class_dict" below.
What's noteworthy, is that this dictionary, could e.g. be a ``OrderDict``. I am
not sure, what ``__prepare__`` is allowed to return.
.. code-block:: python
# in module "SomeModule"
# ...
class SomeClass(SomeBase, AnotherBase, metaclass = SomeMetaClass)
""" This is the class documentation. """
some_member = 3
.. code-block:: python
# Non-keyword arguments, need to be evaluated first.
tmp_bases = ( SomeBase, AnotherBase )
# Keyword arguments go next, __metaclass__ is just one of them. In principle
# we need to forward the others as well, but this is ignored for the sake of
# brevity.
tmp_metaclass = select_metaclass(tmp_bases, SomeMetaClass )
tmp_prepared = tmp_metaclass.__prepare__("SomeClass", tmp_bases)
# The function that creates the class dictionary. Receives temporary variables
# to work with.
def _makeSomeClass:
# This has effect, currently I don't know how to force that in Python3
# syntax, but we will use something that ensures it.
locals() = tmp_prepared
# The module name becomes a normal local variable too.
__module__ = "SomeModule"
# The doc string becomes a normal local variable.
__doc__ = """ This is the class documentation. """
some_member = 3
# Create the class, share the potential closure variable "__class__"
# with others.
__class__ = tmp_metaclass("SomeClass", tmp_bases, locals())
return __class__
# Build and assign the class.
SomeClass = _makeSomeClass()
Generator Expressions
+++++++++++++++++++++
There are re-formulated as functions.
Generally they are turned into calls of function bodies with (potentially
nested) for loops:
.. code-block:: python
gen = ( x*2 for x in range(8) if cond() )
.. code-block:: python
def _gen_helper(__iterator):
for x in __iterator:
if cond():
yield x*2
gen = _gen_helper( range(8 ) )
List Contractions
+++++++++++++++++
The list contractions of Python2 are different from those of Python3, in that
they don't actually do any closure variable taking, and that no function object
ever exists.
.. code-block:: python
list_value = [ x*2 for x in range(8) if cond() ]
.. code-block:: python
def _listcontr_helper(__iterator):
result = []
for x in __iterator:
if cond():
result.append( x*2 )
return result
list_value = listcontr_helper( range(8) )
The difference is that with Python3, the function "_listcontr_helper" is real
and named ````, whereas with Python2 the function must be considered
in-lined.
This in-inlining in case of Python2 causes difficulties, because it's statements
that occur inside an expression, which means a lot of side effects, that may or
may not be possible to unroll to outside.
Set Contractions
++++++++++++++++
TODO.
Dict Contractions
+++++++++++++++++
TODO.
Boolean expressions ``and`` and ``or``
++++++++++++++++++++++++++++++++++++++
The short circuit operators ``or`` and ``and`` tend to be only less general that
the ``if``/``else`` expressions and are therefore re-formulated as such:
.. code-block:: python
expr1() or expr2()
.. code-block:: python
_tmp if ( _tmp = expr1() ) else expr2()
.. code-block:: python
expr1() and expr2()
.. code-block:: python
expr2() if ( _tmp = expr1() ) else _tmp
In this form, the differences between these two operators becomes very apparent,
the operands are simply switching sides.
With this the branch that the "short-circuit" expresses, becomes obvious, at the
expense of having the assignment expression to the temporary variable, that one
needs to create anyway.
Simple Calls
++++++++++++
As seen below, even complex calls are simple calls. In simple calls of Python
there is still some hidden semantic going on, that we expose.
.. code-block:: python
func( arg1, arg2, named1 = arg3, named2 = arg4 )
On the C-API level there is a tuple and dictionary built. This one is exposed:
.. code-block:: python
func( *( arg1, arg2 ), **{ "named1" : arg3, "named2" : arg4 } )
A called function will access this tuple and the dictionary to parse the
arguments, once that is also re-formulated (argument parsing), it can then lead
to simple inlining. This way calls only have 2 arguments with constant
semantics, that fits perfectly with the C-API where it is the same, so it is
actually easier for code generation.
Although the above looks like a complex call, it actually is not. No checks are
needed for the types of the star arguments and it's directly translated to
``PyObject_Call``.
Complex Calls
+++++++++++++
The call operator in Python allows to provide arguments in 4 forms.
* Positional (or normal) arguments
* Named (or keyword) arguments
* Star list arguments
* Star dictionary arguments
The evaluation order is precisely that. An example would be:
.. code-block:: python
something( pos1, pos2, name1 = named1, name2 = named2, *star_list, **star_dict )
The task here is that first all the arguments are evaluated, left to right, and
then they are merged into only two, that is positional and named arguments
only. for this, the star list argument and the star dict arguments, are merged
with the positional and named arguments.
What's peculiar, is that if both the star list and dict arguments are present,
the merging is first done for star dict, and only after that for the star list
argument. This makes a difference, because in case of an error, the star
argument raises first.
.. code-block:: python
something( *1, **2 )
This raises "TypeError: something() argument after ** must be a mapping, not
int" as opposed to a possibly more expected "TypeError: something() argument
after * must be a sequence, not int."
That doesn't matter much though, because the value is to be evaluated first
anyway, and the check is only performed afterwards. If the star list argument
calculation gives an error, this one is raised before checking the star dict
argument.
So, what we do, is we convert complex calls by the way of special functions,
which handle the dirty work for us. The optimization is then tasked to do the
difficult stuff. Our example becomes this:
.. code-block:: python
def _complex_call(called, pos, kw, star_list_arg, star_dict_arg):
# Raises errors in case of duplicate arguments or tmp_star_dict not
# being a mapping.
tmp_merged_dict = merge_star_dict_arguments( called, tmp_named, mapping_check( called, tmp_star_dict ) )
# Raises an error if tmp_star_list is not a sequence.
tmp_pos_merged = merge_pos_arguments( called, tmp_pos, tmp_star_list )
# On the C-API level, this is what it looks like.
return called( *tmp_pos_merged, **tmp_merged_dict )
returned = _complex_call(
called = something,
pos = (pos1, pos2),
named = {
"name1" : named1,
"name2" = named2
},
star_list_arg = star_list,
star_list_arg = star_dict
)
The call to ``_complex_call`` is be a direct function call with no parameter
parsing overhead. And the call in its end, is a special call operation, which
relates to the "PyObject_Call" C-API.
Print statements
++++++++++++++++
The ``print`` statement exists only in Python2. It implicitly coverts its
arguments to strings before printing them. In order to make this accessible and
compile time optimized, this is made visible in the node tree.
.. code-block:: python
print arg1, "1", 1
.. code-block:: python
print str(arg1), "1", str(1)
Only string objects are spared from the ``str`` built-in wrapper, because that
would only cause noise in optimization stage.
Nodes that serve special purposes
---------------------------------
Side Effects
++++++++++++
When an exception is bound to occur, and this can be determined at compile time,
Nuitka will not generate the code the leads to the exception, but directly just
raise it. But not in all cases, this is the full thing.
Consider this code:
.. code-block:: python
f( a(), 1 / 0 )
The second argument will create a ``ZeroDivisionError`` exception, but before
that ``a()`` must be executed, but the call to ``f`` will never happen and no
code is needed for that, but the name look-up must still succeed. This then
leads to code that is internally like this:
.. code-block:: python
f( a(), raise ZeroDivisionError )
which is then modeled as:
.. code-block:: python
side_effect( a(), f, raise ZeroDivisionError )
where you can consider side_effect a function that returns the last
expression. Of course, if this is not part of another expression, but close to
statement level, side effects, can be converted to multiple statements simply.
Another use case, is that the value of an expression can be predicted, but that
the language still requires things to happen, consider this:
.. code-block:: python
a = len( ( f(), g() ) )
We can tell that ``a`` will be 2, but the call to ``f`` and ``g`` must still be
performed, so it becomes:
.. code-block:: python
a = side_effects( f(), g(), 2 )
Modelling side effects explicitely has the advantage of recognizing them easily
and allowing to drop the call to the tuple building and checking its length,
only to release it.
Caught Exception Type/Value References
++++++++++++++++++++++++++++++++++++++
When catching an exception, in C++, an exception object is used. Exception
handler code is being re-formulated to assign the caught exception to a name, to
check its type for values, etc.
For these, not ``sys.exc_info()`` is used, instead there are special nodes
dedicated to these values: ``CaughtExceptionTypeRef`` and
``CaughtExceptionValueRef``.
Plan to replace "python-qt" for the GUI
=======================================
Porting the tree inspector available with ``--dump-gui`` to "wxWindows" is very
much welcome as the "python-qt4" bindings are severely under documented.
Plan to add "ctypes" support
============================
Add interfacing to C code, so Nuitka can turn a ``ctypes`` binding into an
efficient binding as if it were written manually with Python C-API or better.
Goals/Allowances to the task
----------------------------
1. Goal: Must not use any pre-existing C/C++ language file headers, only
generate declarations in generated C++ code ourselves. We would rather write
a C header to ``ctypes`` declarations convert if it needs to be, but not mix
and use declarations from existing header code.
2. Allowance: May use ``ctypes`` module at compile time to ask things about
``ctypes`` and its types.
3. Goal: Should make use of ``ctypes``, to e.g. not hard code what
``ctypes.c_int()`` gives on the current platform, unless there is a specific
benefit.
4. Allowance: Not all ``ctypes`` usages must be supported immediately.
5. Goal: Try and be as general as possible. For the compiler, ``ctypes`` support
should be hidden behind a generic interface of some sort. Supporting ``math``
module should be the same thing.
Type Inference - The Discussion
-------------------------------
Main goal is to forward value knowledge. When you have ``a = b``, that means
that a and b now "alias". And if you know the value of ``b`` you can assume to
know the value of ``a``. This is called "Aliasing".
When assigning ``a`` to something new, that won't change ``b`` at all. But when
an attribute is set, a method called of it, that impacts both, or actually the
value. We need to understand mutable vs. immutable though.
.. code-block:: python
a = 3
b = 3
b += 4 # a is not changed
a = [ 3 ]
b = [ 3 ]
b += [ 4 ] # a is changed
If we cannot tell, we must assume that ``a`` might be changed. It's either ``b``
or what ``a`` was before. If the type is not mutable, we can assume the aliasing
to be broken up, and if it is, we can assume both to be the same value still.
When that value is a compile time constant, we will want to push it forward,
because storing such a constant under a variable name has a cost and loading it
back from the variable as well. So, you want to be able collapse such code:
.. code-block:: python
a = 3
b = 7
c = a / b
to:
.. code-block:: python
c = 3 / 7
and that obviously to:
.. code-block:: python
c = 0
This may be called "(Constant) Value Propagation". But we are aiming for even
more. We want to forward propagate abstract properties of the values.
.. note::
Built-in exceptions, and built-in names are also compile time constants.
In order to fully benefit from type knowledge, the new type system must be able
to be fully friends with existing built-in types. The behavior of a type
``long``, ``str``, etc. ought to be implemented as far as possible with the
builtin ``long``, ``str`` as well.
.. note::
This "use the real thing" concept extends beyond builtin types,
e.g. ``ctypes.c_int()`` should also be used, but we must be aware of platform
dependencies. The maximum size of ``ctypes.c_int`` values would be an example
of that. Of course that may not be possible for everything.
This approach has well proven itself with built-in functions already, where
we use real built-ins where possible to make computations. We have the
problem though that built-ins may have problems to execute everything with
reasonable compile time cost.
Another example, consider the following code:
.. code-block:: python
len( "a" * 1000000000000 )
To predict this code, calculating it at compile time using constant operations,
while feasible, puts an unacceptable burden on the compilation.
Esp. we wouldn't want to produce such a huge constant and stream it, the C++
code would become too huge. So, we need to stop the ``*`` operator from being
used at compile time and cope with reduced knowledge, already here:
.. code-block:: python
"a" * 10000000000000
Instead, we would probably say that for this expression:
- The result is a ``str`` or ``PyStringObject``.
- We know its length exactly, it's ``10000000000000``.
- Can predict every of its elements when sub-scripted, sliced, etc., if need
be, with a function we may create.
Similar is true for this horrible thing:
.. code-block:: python
range( 10000000000000 )
So it's a rather general problem, this time we know:
- The result is a ``list`` or ``PyListObject``
- We know its length exactly, ``10000000000000``
- Can predict every of its elements when index, sliced, etc., if need be,
with a function.
Again, we wouldn't want to create the list. Therefore Nuitka avoids executing
these calculation, when they result in constants larger than a threshold of
e.g. 256. This concept has to be also applied to integers and more CPU and
memory traps.
Now lets look at a more common use case:
.. code-block:: python
for x in range( 10000000000000 ):
doSomething()
Looking at this example, one traditional way to look at it, would be to turn
``range`` into ``xrange``, and to note that ``x`` is unused. That would already
perform better. But really better is to notice that ``range()`` generated values
are not used at all, but only the length of the expression matters.
And even if ``x`` were used, only the ability to predict the value from a
function would be interesting, so we would use that computation function instead
of having an iteration source. Being able to predict from a function could mean
to have Python code to do it, as well as C++ code to do it. Then code for the
loop can be generated without any CPython library usage at all.
.. note::
Of course, it would only make sense where such calculations are "O(1)"
complexity, i.e. do not require recursion like "n!" does.
The other thing is that CPython appears to at - run time - take length hints
from objects for some operations, and there it would help too, to track length
of objects, and provide it, to outside code.
Back to the original example:
.. code-block:: python
len( "a" * 1000000000000 )
The theme here, is that when we can't compute all intermediate expressions, and
we sure can't do it in the general case. But we can still, predict some of
properties of an expression result, more or less.
Here we have ``len`` to look at an argument that we know the size of. Great. We
need to ask if there are any side effects, and if there are, we need to maintain
them of course. This is already done by existing optimization if an operation
generates an exception.
.. note::
The optimization of ``len`` has been implemented and works for all kinds of
container creation and ranges.
Applying this to "ctypes"
-------------------------
The not so specific problem to be solved to understand ``ctypes`` declarations
is maybe as follows:
.. code-block:: python
import ctypes
This leads to Nuitka in its tree to have an assignment from a ``__import__``
expression to the variable ``ctypes``. It can be predicted by default to be a
module object, and even better, it can be known as ``ctypes`` from standard
library with more or less certainty. See the section about "Importing".
So that part is "easy", and it's what will happen. During optimization, when the
module ``__import__`` expression is examined, it should say:
- ``ctypes`` is a module
- ``ctypes`` is from standard library (if it is, may not be true)
- ``ctypes`` has a ``ModuleFriend`` that knows things about it attributes,
that should be asked.
The later is the generic interface, and the optimization should connect the two,
of course via package and module full names. It will need a
``ModuleFriendRegistry``, from which it can be pulled. It would be nice if we
can avoid ``ctypes`` to be loaded into Nuitka unless necessary, so these need to
be more like a plug-in, loaded only if necessary, i.e. the user code actually
uses ``ctypes``.
Coming back to the original expression, it also contains an assignment
expression, because it is more like this:
.. code-block:: python
ctypes = __import__( "ctypes" )
The assigned to object, simply gets the type inferred propagated as part of an
SSA form. Ideally, we could be sure that nothing in the program changes the
variable, and therefore have only one version of that variable.
For module variables, when the execution leaves the module to unknown code, or
unclear code, it might change the variable. Therefore, likely we will often only
assume that it could still be ctypes, or something else.
Depending on how well we control module variable assignment, we can decide this
more of less quickly. With "compiled modules" types, the expectation is that
it's merely a quick C++ `==` comparison check. The module friend should offer
code to allow a check if it applies, for uncertain cases.
Then when we come to uses of it:
.. code-block:: python
ctypes.c_int()
At this point, using SSA, we are more of less sure, that ``ctypes`` is at that
point the module, and that we know what it's ``c_int`` attribute is, at comile
time, and what it's call result is. We will use the module friend to help with
that. It will attach knowledge about the result of that expression during the
SSA collection process.
This is more like a value forward propagation than anything else. In fact,
constant propagation should only be the special case of it, and one design goal
of Nuitka was always to cover these two cases with the same code.
Excursion to Functions
----------------------
In order to decide what this means to functions and their call boundaries, if we
propagate forward, how to handle this:
.. code-block:: python
def my_append(a, b):
a.append( b )
return a
We would notate that ``a`` is first a "unknown but defined parameter object",
then later on something that definitely has an ``append`` attribute, when
returned. Otherwise an exception occurs.
The type of ``a`` changes to that after ``a.append`` look-up succeeds. It might
be many kinds of an object, but e.g. it could have a higher probability of being
a ``PyListObject``. And we would know it cannot be a ``PyStringObject``, as that
one has no "append".
.. note::
If classes, i.e. other types in the program, have an ``append`` attribute, it
should play a role too, there needs to be a way to plug-in to this decisions.
.. note::
On the other hand, types without ``append`` attribute could be eliminated.
It would be great, if functions provided some sort of analysis on their return
type, or a quick way to predict return value properties, based on input value
knowledge.
So this could work:
.. code-block:: python
b = my_append( [], 3 )
assert b == [3] # Could be decided now
Goal: The structure we use makes it easy to tell what ``my_append`` may be. So,
there should be a means to ask it about call results with given type/value
information. We need to be able to tell, if evaluating ``my_append`` makes sense
with given parameters or not, if it does impact the return value.
We should e.g. be able to make ``my_append`` tell, one or more of these:
- Returns the first parameter value as return value (unless it raises an
exception).
- The return value has the same type as ``a`` (unless it raises an
exception).
- The return value has an ``append`` attribute.
- The return value might be a ``list`` object.
- The return value may not be a ``str`` object.
- The function will raise if first argument has no ``append`` attribute.
The exactness of statements may vary. But some things may be more
interesting. If e.g. the aliasing of a parameter value to the return value is
known exactly, then information about it need to all be given up, but some can
survive.
It would be nice, if ``my_append`` had sufficient information, so we could
specialize with ``list`` and ``int`` from the parameters, and then e.g. know at
least some things that it does in that case. Such specialization would have to
be decided if it makes sense. In the alternative, it could be done for each
variant anyway, as there won't be that many of them.
Doing this "forward" analysis appears to be best suited for functions and
therefore long term. We will try it that way.
Excursion to Loops
------------------
.. code-block:: python
a = 1
while 1: # think loop: here
b = a + 1
a = b
if cond():
break
print a
The handling of loops (both "for" and "while" are re-formulated to loops with
breaks) has its own problem. The loop start and may have an assumption from
before it started, that "a" is constant, but that is only true for the first
iteration. So, we can't pass knowledge from outside loop forward directly into
the for loop body.
So the collection for loops needs to be two pass. First, to collect assignments,
and merge these into the start state, before entering the loop body. The need to
make two passes is special to loops.
For a start, it could be done like this though: At loop entry, all knowledge is
removed about everything, and so is at loop exit. That way, only the loop inner
working is optimized, and before and after the loop are separate things. The
optimal handling of "a" in the example code will take a while.
For a general solution, it would be sweet to trace different exit paths
differently. One loop exit may be good enough, as it will be the common case.
Excursion to Conditions
-----------------------
.. code-block:: python
if cond:
x = 1
else:
x = 2
b = x < 3
The above code contains a condition, and these have the problem, that when
exiting the conditional block, a merge must be done, of the "x" versions. It
could be either one. The merge may trace the condition under which a choice is
taken. That way, we could decide pairs of traces under the same condition.
These merges of SSA variable versions, represent alternatives. They pose
difficulties, and might have to be reduced to commonality. In the above example,
the "<" operator will have to check for each version, and then to decide that
both indeed give the same result.
The constraint collection tracks variable changes in conditional branches, and
then merges the existing state at conditional statement exits.
.. note::
A branch is considered "exiting" if it is not abortive. Should it end in a
``raise``, ``break``, ``continue``, or ``return``, there is no need to merge
that branch, as execution of that branch is terminated.
Should both branches be abortive, that makes things really simple, as there
is no need to even continue.
Should only one branch exist, but be abortive, then no merge is needed, and
the collection can assume after the conditional statement, that the branch
was not taken, and continue.
When exiting both the branches, these branches must both be merged, with their
new information.
In the above case:
- The "yes" branch knows variable ``x`` is an ``int`` of constant value ``1``
- The "no" branch knows variable ``x`` is an ``int`` of constant value ``2``
That might be collapsed to:
- The variable ``x`` is an integer of value in ``(1,2)``
Given this, we then should be able to precompute the value of this:
.. code-block:: python
b = x < 3
The comparison operator can therefore decide and tell:
- The variable ``b`` is a boolean of constant value ``True``.
Were it unable to decide, it would still be able to say:
- The variable ``b`` is a boolean.
For conditional statements optimization, it's also noteworthy, that the
condition is known to pass or not pass the truth check, inside branches, and in
the case of non-exiting single branches, after the statement it's not true.
We may want to take advantage of it. Consider e.g.
.. code-block:: python
if type( a ) is list:
a.append( x )
else:
a += ( x, )
In this case, the knowledge that ``a`` is a list, could be used to generate
better code and with the definite knowledge that ``a`` is of type list. With
that knowledge the ``append`` attribute call will become the ``list`` built-in
type operation.
Excursion to ``return`` statements
----------------------------------
The ``return`` statement (like ``break``, ``continue``, ``raise``) is "aborting"
to control flow. It is always the last statement of inspected block. Were there
statements to follow it, optimization will remove it as dead code.
If all branches of a conditional statement are "aborting", the statement is
decided "aborting" too. If a loop doesn't break, it should be considered
"aborting" too.
.. note::
The removal of statements following "aborting" statements is implemented, and
so is the discovery of abortive conditional statements. It's not yet done for
loops, temp blocks, etc. though.
So, ``return`` statements are easy for local optimization. In the general
picture, it would be sweet to collect all return statements, and analyze the
commonality of them. The goal to predict function results, might be solvable by
looking at their traces.
Excursion to ``yield`` expressions
----------------------------------
The ``yield`` expression can be treated like a normal function call, and as such
invalidates some known constraints just as much as they do. It executes outside
code for an unknown amount of time, and then returns, with little about the
outside world known anymore.
Mixed Types
-----------
Consider the following inside a function or module:
.. code-block:: python
if cond is not None:
a = [ x for x in something() if cond(x) ]
else:
a = ()
A programmer will often not make a difference between ``list`` and ``tuple``. In
fact, using a ``tuple`` is a good way to express that something won't be changed
later, as these are mutable.
.. note::
Better programming style, would be to use this:
.. code-block:: python
if cond is not None:
a = tuple( x for x in something() if cond(x) )
else:
a = ()
People don't do it, because they dislike the performance hit encountered by
the generator expression being used to initialize the tuple. But it would be
more consistent, and so Nuitka is using it, and of course one day Nuitka
ought to be able to make no difference in performance for it.
To Nuitka though this means, that if ``cond`` is not predictable, after the
conditional statement we may either have a ``tuple`` or a ``list`` type object
in ``a``. In order to represent that without resorting to "I know nothing about
it", we need a kind of ``min``/``max`` operating mechanism that is capable of
say what is common with multiple alternative values.
.. note::
At this time, we don't really have that mechanism to find the commonality
between values.
Back to "ctypes"
----------------
.. code-block:: python
v = ctypes.c_int()
Coming back to this example, we needed to propagate ``ctypes``, then we can
propagate "something" from ``ctypes.int`` and then known what this gives with a
call and no arguments, so the walk of the nodes, and diverse operations should
be addressed by a module friend.
In case a module friend doesn't know what to do, it needs to say so by
default. This should be enforced by a base class and give a warning or note.
Now to the interface
--------------------
The following is the intended interface:
- Iteration with node methods ``computeStatement`` and ``computeNode``.
These traverse modules and functions (i.e. scopes) and visit everything in the
order that Python executes it. The visiting object is ``ConstraintCollection``
and pass forward. Some node types, e.g. ``StatementConditional`` new create
child constraint collections and handle the SSA merging at exit.
- Replacing nodes during the visit.
Both ``computeStatement`` and ``computeNode`` are tasked to return potential
replacements of themselves, together with "tags" (meaningless now), and a
"message", used for verbose tracing.
The replacement node of "+" operator, may e.g. the pre-computed result,
wrapped in side effects of the node.
- Assignments and references affect SSA.
The SSA tree is initialized every time a scope is visited. Then during
traversal, traces are built up. Every assignment and merge starts a new trace
for that matter. References to a given variable version are traced that way.
- Value escapes are traced too.
When an operation hands over a value to outside code, it indicates so to the
constraint collection. This is for it to know, when e.g. a constant value,
might be mutated meanwhile.
- Nodes can be queried about their properties.
The node base classes offers methods that allow to check if certain operations
are supported or not. These can always return ``True`` (yes), ``False`` (no),
and ``None`` (cannot decide). In the case of the later, optimizations may not
be able do much about it. Lets call these values "tri-state".
The default implementation will be very pessimistic. Specific node types may
then declare, that they e.g. have no side effects, do no raise, have a know
truth value, have a known iteration length, can predict their iteration
values, etc.
- Nodes are linked to certain states.
During the collect, a variable reference, is linked to a certain trace state,
and that can be used by parent operations.
.. code-block:: python
a = 1
b = a + a
In this example, the references to "a", can look-up the "1" in the trace, and
base their responses to "+" on it. It will ask "isCompileTimeConstant()" and
both nodes will respond "True", then "getCompileTimeConstant()" will return
"1", which will be computed. Then "extractSideEffects()" will return "()" and
therefore, the result "2" will not be wrapped.
- Class for module import expression ``ExpressionImportModule``.
This one just knows that something is imported, but not how or what it is
assigned to. It will be able in a recursive compile, to provide the module as
an assignment source, or the module variables or submodules as an attribute
source when referenced from a variable trace or in an expression.
- Base class for module friend ``ModuleFriendBase``.
This is intended to provide something to overload, which e.g. can handle
``math`` in a better way.
- Module ``ModuleFriendRegistry``
Provides a register function with ``name`` and instances of
``ValueFriendModuleBase`` to be registered. Recursed to modules should
integrate with that too. The registry could well be done with a metaclass
approach.
- The module friends should each live in a module of their own.
With a naming policy to be determined. These modules should add themselves via
above mechanism to ``ModuleFriendRegistry`` and all shall be imported and
register. Importing of e.g. ``ctypes`` should be delayed to when the friend is
actually used. A meta class should aid this task.
The delay will avoid unnecessary blot of the compiler at run time, if no such
module is used. For "qt" and other complex stuff, this will be a must.
- The walk should initially be single pass, and not maintain history.
Instead optimization that needs to look at multiple things, e.g. "unused
assignment", will look at the whole SSA collection afterwards.
Discussing with examples
------------------------
The following examples:
.. code-block:: python
# Assignment, the source decides the type of the assigned expression
a = b
# Operator "attribute look-up", the looked up expression "ctypes" decides
# via its trace.
ctypes.c_int
# Call operator, the called expressions decides with help of arguments,
# which have been walked, before the call itself.
called_expression_of_any_complexity()
# import gives a module any case, and the "ModuleRegistry" may say more.
import ctypes
# From import need not give module, "x" decides what it is.
from x import y
# Operations are decided by arguments, and CPython operator rules between
# argument states.
a + b
The optimization is mostly performed by walking of the tree and performing
constraint collection. When it encounters assignments and references to them, it
considers current state of traces and uses it for ``computeExpression``.
.. note::
Assignments to attributes, indexes, slices, etc. will also need to follow the
flow of ``append``, so it cannot escape attention that a list may be
modified. Usages of ``append`` that we cannot be sure about, must be traced
to exist, and disallow the list to be considered known value again.
Code Generation Impact
----------------------
Right now, code generation assumes that everything is a ``PyObject *``, i.e. a
Python object, and does not take knowledge of ``int`` or other types into
consideration at all, and it should remain like that for some time to come.
Instead, ``ctypes`` value friend will be asked give ``Identifiers``, like other
codes do too. And these need to be able to convert themselves to objects to work
with the other things.
But Code Generation should no longer require that operations must be performed
on that level. Imagine e.g. the following calls:
.. code-block:: python
c_call( other_c_call() )
Value returned by "other_c_call()" of say ``c_int`` type, should be possible to
be fed directly into another call. That should be easy by having a ``asIntC()``
in the identifier classes, which the ``ctypes`` Identifiers handle without
conversions.
Code Generation should one day also become able to tell that all uses of a
variable have only ``c_int`` value, and use ``int`` instead of
``PyObjectLocalVariable`` more or less directly. We could consider
``PyIntLocalVariable`` of similar complexity as ``int`` after the C++ compiler
performed its in-lining.
Such decisions would be prepared by finalization, which then would track the
history of values throughout a function or part of it.
Initial Implementation
----------------------
The basic interface will be added to *all* expressions and a node may override
it, potentially using constraint collection state, as attached during
"computeExpression".
Goal 1
++++++
Initially most things will only be able to give up on about anything. And it
will be little more than a tool to do simple look-ups in a general form. It will
then be the first goal to turn the following code into better performing one:
.. code-block:: python
a = 3
b = 7
c = a / b
return c
to:
.. code-block:: python
a = 3
b = 7
c = 3 / 7
return c
and then:
.. code-block:: python
a = 3
b = 7
c = 0
return c
and then:
.. code-block:: python
a = 3
b = 7
c = 0
return 0
This depends on SSA form to be able to tell us the values of ``a``, ``b``, and
``c`` to be written to by constants, which can be forward propagated at no cost.
Goal 2
++++++
The assignments to ``a``, ``b``, and ``c`` shall all become prey to "unused"
assignment analysis in the next step. They are all only assigned to, and the
assignment source has no effect, so they can be simply dropped.
.. code-block:: python
return 0
In the SSA form, these are then assignments without references. These
assignments, can be removed if the assignment source has no side effect. Or at
least they could be made "anonymous", i.e. use a temporary variable instead of
the named one. That would have to take into account though, that the old version
still needs a release.
The most general form would first merely remove assignments that have no impact,
and leave the value as a side effect, so we arrive at this first:
.. code-block:: python
3
7
0
return 0
When applying the removal of expression only statements without effect, this
gives us:
.. code-block:: python
return 0
which is the perfect result. Doing it in one step would only be an optimization.
In order to be able to manipulate nodes related to a variable trace, we need to
attach the nodes that did it. Consider this:
.. code-block:: python
if cond():
x = 1
elif other():
x = 3
# Not using "x".
return 0
In the above case, the merge of the value friends, should say that ``x`` may be
undefined, or one of ``1`` or ``3``, but since ``x`` is not used, apply the
"dead value" trick to each branch.
The removal of the "merge" of the 3 ``x`` versions, should exhibit that the
other versions are also only assigned to, and can be removed. These merges of
course appear as usages of the ``x`` versions.
Goal 3
++++++
Then third goal is to understand all of this:
.. code-block:: python
def f():
a = []
print a
for i in range(1000):
print a
a.append( i )
return len( a )
.. note::
There are many operations in this, and all of them should be properly
handled, or at least ignored in safe way.
The first goal code gave us that the ``list`` has an annotation from the
assignment of ``[]`` and that it will be copied to ``a`` until the for loop in
encountered. Then it must be removed, because the ``for`` loop somehow says so.
The ``a`` may change its value, due to the unknown attribute look-up of it
already, not even the call. The for loop must be able to say "may change value"
due to that, of course also due to the call of that attribute too.
The code should therefore become equivalent to:
.. code-block:: python
def f():
a = []
print []
for i in range(1000):
print a
a.append( i )
return len( a )
But no other changes must occur, especially not to the "return" statement, it
must not assume "a" to be constant "[]" but an unknown "a" instead.
With that, we would handle this code correctly and have some form constant value
propagation in place, handle loops at least correctly, and while it is not much,
it is important demonstration of the concept.
Goal 4
++++++
The fourth goal is to understand the following:
.. code-block:: python
def f(cond):
y = 3
if cond:
x = 1
else:
x = 2
return x < y
In this we have a branch, and we will be required to keep track of both the
branches separately, and then to merge with the original knowledge. After the
conditional statement we will know that "x" is an "int" with possible values in
"(1,2)", which can be used to predict that the return value is always "True".
The forth goal will therefore be that the "ValueFriendConstantList" knows that
append changes "a" value, but it remains a list, and that the size increases by
one. It should provide an other value friend "ValueFriendList" for "a" due to
that.
In order to do that, such code must be considered:
.. code-block:: python
a = []
a.append( 1 )
a.append( 2 )
print len( a )
It will be good, if "len" still knows that "a" is a list, but not the constant
list anymore.
From here, work should be done to demonstrate the correctness of it with the
basic tests applied to discover undetected issues.
Fifth and optional goal: Extra bonus points for being able to track and predict
"append" to update the constant list in a known way. Using "list.append" that
should be done and lead to a constant result of "len" being used.
The sixth and challenging goal will be to make the code generation be impacted
by the value friends types. It should have a knowledge that "PyList_Append" does
the job of append and use "PyList_Size" for "len". The "ValueFriends" should aid
the code generation too.
Last and right now optional goal will be to make "range" have a value friend,
that can interact with iteration of the for loop, and "append" of the "list"
value friend, so it knows it's possible to iterate 5000 times, and that "a" has
then after the "loop" this size, so "len( a )" could be predicted. For during
the loop, about a the range of its length should be known to be less
than 5000. That would make the code of goal 2 completely analyzed at compile
time.
Limitations for now
-------------------
- Aim only for limited examples. For ``ctypes`` that means to compile time
evaluate:
.. code-block:: python
print ctypes.c_int( 17 ) + ctypes.c_long( 19 )
Later then call to "libc" or something else universally available,
e.g. "strlen()" or "strcmp()" from full blown declarations of the callable.
- We won't have the ability to test that optimization are actually performed, we
will check the generated code by hand.
With time, we will add XML based checks with "xpath" queries, expressed as
hints, but that is some work that will be based on this work here. The "hints"
fits into the "ValueFriends" concept nicely or so the hope is.
- No inter-function optimization functions yet
Of course, once in place, it will make the ``ctypes`` annotation even more
usable. Using ``ctypes`` objects inside functions, while creating them on the
module level, is therefore not immediately going to work.
- No loops yet
Loops break value propagation. For the ``ctypes`` use case, this won't be much
of a difficulty. Due to the strangeness of the task, it should be tackled
later on at a higher priority.
- Not too much.
Try and get simple things to work now. We shall see, what kinds of constraints
really make the most sense. Understanding ``list`` subscript/slice values
e.g. is not strictly useful for much code and should not block us.
.. note::
This design is not likely to be the final one.
.. raw:: pdf
PageBreak
Idea Bin
========
This an area where to drop random ideas on our minds, to later sort it out, and
out it into action, which could be code changes, plan changes, issues created,
etc.
* Make "SELECT_METACLASS" meta class selection transparent.
Looking at the "SELECT_METACLASS" it should become an anonymous helper
function. In that way, the optimization process can remove choices at compile
time, and e.g. inline the effect of a meta class, if it is known.
This of course makes most sense, if we have the optimizations in place that
will allow this to actually happen.
* Keeping track of iterations
The constraint collection trace should become the place, where variables or
values track their use state. The iterator should keep track of the "next()"
calls made to it, so it can tell which value to given in that case.
That would solve the "iteration of constants" as a side effect and it would
allow to tell that they can be removed.
That would mean to go back in the tree and modify it long after.
.. code-block:: python
a = iter( ( 2, 3 ) )
b = next( a )
c = next( a )
del a
It would be sweet if we could recognize that:
.. code-block:: python
a = iter( ( 2, 3 ) )
b = side_effect( next( a ), 2 )
c = side_effect( next( a ), 3 )
del a
That trivially becomes:
.. code-block:: python
a = iter( ( 2, 3 ) )
next( a )
b = 2
next( a )
c = 3
del a
When the "del a" is examined at the end of scope, or due to another assignment
to the same variable, ending the trace, we would have to consider of the
"next" uses, and retrofit the information that they had no effect.
.. code-block:: python
a = iter( ( 2, 3 ) )
b = 2
b = 3
del a
* Aliasing
Each time an assignment is made, an alias is created. A value may have
different names.
.. code-block:: python
a = iter( range(9 ))
b = a
c = next(b)
d = next(a)
If we fail to detect the aliasing nature, we will calculate "d" wrongly. We
may incref and decref values to trace it.
Aliasing is automatically traced already in SSA form. The "b" is assigned to
version of "a". So, that should allow to replace it with this:
.. code-block:: python
a = iter( range(9 ))
c = next(a)
d = next(a)
Which then will be properly handled.
* Shelve for caching
If we ever came to the conclusion to want and cache complex results of
analysis, we could do so with the shelve module. We would have to implement
``__deepcopy__`` and then could store in there optimized node structures from
start values after parsing.
* Tail recursion optimization.
Functions that return the results of calls, can be optimized. The Stackless
Python does it already.
* Integrate with "upx" compression.
Calling "upx" on the created binaries, would be easy.
* The timing of ``__del__`` calls.
When you do a(b(c())) in Python, it deletes the argument value, i.e. return
value of c() immediately after calling b().
Currently we translate that to C++ roughly like this: a(b(c())) as well. Only
that in C++, b returns an object, that has a scope. It appears, the d-tor is
executed at the end of the statement. In C++ the ";" is a sequence point,
i.e. things must be done by then.
Unfortunately C++ loves temporaries so much, it won't immediately delete them
after use, but only after full expression, which means ")" or ";", and
attempts with fake sequence points all failed.
But, there may be another way. Right now, ``PyObject *`` is the interface for
about everything passed around. And "PyObjectTemporary" releases values that
are needed by the interface to have a reference, and deleted afterwards.
But it could, and should be different. All helper functions should be template
functions that accept ``PyObjectRef1`` and ``PyObjectRef0``, and know about
the reference, and then manage ``PyObjectRef1`` instances to release their
reference as soon as they are not needed. With ``PyObjectRef0`` that would be
a no-op.
This is a lot of work. The good news, is that it's work that will be needed,
to support types other than ``PyObject *`` efficiently. Them being converted
to ``PyObject *`` and releasing that reference, it would be transparent to all
code.
* In-lining constant "exec" and "eval".
It should be possible to re-formulate at least cases without "locals" or
"globals" given.
.. code-block:: python
def f():
a = 1
b = 2
exec( """a+=b;c=1""" )
return a, c
Should become this here:
.. code-block:: python
def f():
a = 1
b = 2
a+=b #
c=1 # MaybeLocalVariables for everything except known local ones.
return a, c
If this holds up, inlining ``exec`` should be relatively easy.
* Original and overloaded built-ins
This is about making things visible in the node tree. In Nuitka things that
are not visible in the node tree tend to be wrong. We already pushed around
information to the node tree a lot.
Later versions, Nuitka will become able to determine it has to be the original
built-in at compilt time, then a condition that checks will be optimized away,
together with the slow path. Or the other path, if it won't be. Then it will
be optimized away, or if doubt exists, it will be correct. That is the goal.
Right now, the change would mean to effectively disable all built-in call
optimization, which is why we don't immediately do it.
Making the compatible version, will also require a full listing of all
built-ins, which is typing work merely, but not needed now. And a way to stop
built-in optimization from optimizing builtin calls that it used in a
wrap. Probably just some flag to indicate it when it visits it to skip
it. That's for later.
But should we have that both, I figure, we could not raise a ``RuntimeError``
error, but just do the correct thing, in all cases. An earlier step may raise
``RuntimeError`` error, when built-in module values are written to, that we
don't support.
.. raw:: pdf
PageBreak
* SSA form for Nuitka nodes
* Assignments collect a counter from the variable, which becomes the variable
version. This happens during tree building phase.
* References need to back track to the last assignment on their path, which
may be a merge. Constraint collection can do that.
* Data structures
Every constraint collection has these:
* variable_actives
Dictionary, where per "variable" the currently used version is. Used to
track situations changes in branches. This is the main input for merge
process.
* variable_traces
Dictionary, where "variable" and "version" form the key. The values are
objects with or without an assignment, and a list of usages, which starts
out empty.
These objects have usages appended to them. In "onVariableSet", a new
version is allocated, which gives a new object for the dictionary, with an
empty usages list, because each write starts a new version. In
"onVariableUsage" the version is detected from the current version. It may
be not set yet, which means, it's a read of an undefined value (local
variable, not a parameter name), or unknown in case of global variable.
These objects may be told that their value has escaped. This should
influence the value friend they attached to the initial assignment. Each
usage may have a current value friend state that is different.
* When merging branches of conditional statements, the merge shall apply as
follows.
* Branches have their own collection, with deviating sets of
"variable_actives". These are children of an outer collections
* Case a) One branch only.
For that branch a collection is performed. As usual new assignments
generate a new version making it "active", references then related to
these "active" versions.
Then, when the branch is merged, for all "active" variables, it is
considered, if that is a change related to before the branch. If it's not
the same, a merge trace with the branch condition is created with the one
active in the collection before that statement.
* Case b) Two branches.
When there are two branches, they both as are treated as above, except for
the merge.
When merging, a difference in active variables between the two branches
creates the merge trace.
.. note::
For conditional expressions, there are always only two branches. Even if
you think you have more than one branch, you do not. It's always nested
branches, already when it comes out of the parser.
* Trace structure
* Initial write of the version
There may be a initial write for each version. It can only occur at the
start of it, but not later, and there is only one. The "value friend" of
it.
* Merge of other one or two other versions
One could be empty, i.e. the variable would not be assigned. This is kind
of the initial write, and the merge references one or multiple "value
friends", which are optional.
* Bunch of read usages. They may allow escape of the value or not. When they
do, it's a change. The value friend must be informed of it. If it's a real
escape, usage is not known. If it's merely an alias, e.g. the value is now
in another variable trace, they could be linked. Otherwise the "value
friend" must be demoted immediately to one that gives more vague
information.
This should be reflected in a class "VariableTrace".
* Recursion checks are expensive.
If the "caller" or the "called" can declare that it cannot be called by
itself, we could leave it out.
TODO: Are they really that expensive? Unnecessary yes, but expensive may not
be true.
* References
Currently Nuitka has "Variable" objects. Every variable reference node type
refers to a "VariableReference" node and there are multiple of them. Every
variable traces the reference objects created.
The idea of references started out with closure references and has expanded
from there. It's now used to decide that a variable is shared. You can ask it,
and because it knows its references, it can tell.
The thing is, this is not updated, so should a closure variable reference go
away, it's still shared, as the reference remains. The thing with replaced and
removed nodes, is that currently they do not remove themselves, there is no
``__del__`` being called. I consider this too unreliable.
That makes the detection of "shared" unreliable and with false positives, that
so far do not harm much. There is an issue with Python3 not compiling with
debug mode that might be a cause of it.
Anyway, the problem is increased by the scope of code in use in each
optimization pass is only ever increasing, but starts out small. That a
variable is shared or merely used elsewhere, might be discovered late. By
starting from scratch again, over and over, we might discover this only later.
That may mean, we should do trace based optimization only after it's all
complete, and not before. During the collection, information about the sharing
should be reset at the start, and the built up and judged at the end.
The task to maintain this would be near ModuleRegistry.
* Outline functions
The list contractions of Python2, and potentially other contractions or
in-lined functions too, in case they don't need any closure from it, could be
considered part of the surrounding function.
These would have function bodies, with proper return, and generate code as a
function would, but with the closure and local variables shared from arguments
in what is considered a direct call.
The outline functions would not be considered closure takers, nor closure
givers. They should be visited when they are used, almost like a statement
sequences, and returns would define their value.
.. raw:: pdf
PageBreak
Updates for this Manual
=======================
This document is written in REST. That is an ASCII format which is readable as
ASCII, but used to generate PDF or HTML documents.
You will find the current source under:
http://nuitka.net/gitweb/?p=Nuitka.git;a=blob_plain;f=Developer_Manual.rst
And the current PDF under:
http://nuitka.net/doc/Developer_Manual.pdf
Nuitka-0.5.0.1/PKG-INFO 0000644 0001750 0001750 00000000500 12265271051 014431 0 ustar hayen hayen 0000000 0000000 Metadata-Version: 1.0
Name: Nuitka
Version: 0.5.0.1
Summary: Python compiler with full language support and CPython compatibility
Home-page: http://nuitka.net
Author: Kay Hayen
Author-email: Kay.Hayen@gmail.com
License: Apache License, Version 2.0
Description: UNKNOWN
Keywords: compiler,python,nuitka
Platform: UNKNOWN
Nuitka-0.5.0.1/MANIFEST.in 0000644 0001750 0001750 00000002040 12265044767 015107 0 ustar hayen hayen 0000000 0000000 include LICENSE.txt
include MANIFEST.in
include README.txt README.pdf
include Changelog.rst Changelog.pdf
include Developer_Manual.rst Developer_Manual.pdf
include doc/*.1
include bin/compare_with_cpython
include bin/compare_with_xml
include misc/*.sh
include misc/*.bat
include misc/check-with-pylint
include misc/check-release
# Logo with source
include misc/Logo/Nuitka-Logo-Symbol.svg
include misc/Logo/Nuitka-Logo-Vertical.svg
include misc/Logo/Nuitka-Logo-Horizontal.svg
include images/Nuitka-Logo-Symbol.png
include images/Nuitka-Logo-Vertical.png
include images/Nuitka-Logo-Horizontal.png
recursive-include lib *.py
# Core tests are included along with Nuitka itself.
recursive-include tests/basics *.py
recursive-include tests/syntax *.py
recursive-include tests/packages *.py
recursive-include tests/programs *.py
recursive-include tests/standalone *.py
recursive-include tests/reflected *.py
include tests/test_common.py
# Bnechmarks are included too, but will be removed for Debian.
recursive-include tests/benchmarks *.py LICENSE README
Nuitka-0.5.0.1/Changelog.rst 0000644 0001750 0001750 00000731145 12265270763 016006 0 ustar hayen hayen 0000000 0000000 Nuitka Release 0.5.0
====================
This release breaks interface compatibility, therefore the major version number
change. Also "standalone mode" has seen significant improvements on both
Windows, and Linux. Should work much better now.
But consider that this part of Nuitka is still in its infancy. As it is not the
top priority of mine for Nuitka, which primarily is intended as an super
compatible accelerator of Python, it will continue to evolve nearby.
There is also many new optimization based on structural improvements in the
direction of actual SSA.
Bug Fixes
---------
- The "standalone mode" was not working on all Redhat, Fedora, and openSUSE
platforms and gave warnings with older compilers. Fixed in 0.4.7.1 already.
- The "standalone mode" was not including all useful encodings. `Issue#116
`__. Fixed in 0.4.7.2 already.
- The "standalone mode" was defaulting to ``--python-flag=-S`` which disables
the parsing of "site" module. That unfortunately made it necessary to reach
some modules without modifying ``PYTHONPATH`` which conflicts with the
"out-of-the-box" experience.
- The "standalone mode" is now handling packages properly and generally working
on Windows as well.
- The syntax error of having an all catching except clause and then a more
specific one wasn't causing a ``SyntaxError`` with Nuitka.
.. code-block:: python
try:
something()
except:
somehandling():
except TypeError:
notallowed()
- A corruption bug was identified, when re-raising exceptions, the top entry of
the traceback was modified after usage. Depending on ``malloc`` this was
potentially causing an endless loop when using it for output.
New Features
------------
- Windows: The "standalone" mode now properly detects used DLLs using
`Dependency Walker `__ which it
offers to download and extra for you.
It is used as a replacement to ``ldd`` on Linux when building the binary, and
as a replacement of ``strace`` on Linux when running the tests to check that
nothing is loaded from the outside.
New Optimization
----------------
- When iterating over ``list``, ``set``, this is now automatically lowered to
``tuples`` avoiding the mutable container types.
So the following code is now equivalent:
.. code-block:: python
for x in [ a, b, c ]:
...
# same as
for x in ( a, b, c ):
...
For constants, this is even more effective, because for mutable constants, no
more is it necessary to make a copy.
- Python2: The iteration of large ``range`` is now automatically lowered to
``xrange`` which is faster to loop over, and more memory efficient.
- Added support for the ``xrange`` built-in.
- The statement only expression optimization got generalized and now is capable
of removing useless parts of operations, not only the whole thing when it has
not side effects.
.. code-block:: python
[a,b]
# same as
a
b
This works for all container types.
Another example is ``type`` built-in operation with single argument. When the
result is not used, it need not be called.
.. code-block:: python
type(a)
# same as
a
And another example ``is`` and ``is not`` have no effect of their own as well,
therefore:
.. code-block:: python
a is b
# same as
a
b
- Added proper handling of conditional expression branches in SSA based
optimization. So far these branches were ignored, which only acceptable for
temporary variables as created by tree building, but not other variable
types. This is preparatory for introducing SSA for local variables.
Organizational
--------------
- The option ``--exe`` is now ignored and creating an executable is the default
behavior of ``nuitka``, a new option ``--module`` allows to produce extension
modules.
- The binary ``nuitka-python`` was removed, and is replaced by ``nuitka-run``
with now only implies ``--execute`` on top of what ``nuitka`` is.
- Using dedicated `Buildbot `__ for continuous integration
testing and release creation as well.
- The `Downloads `__ now offers MSI files
for Win64 as well.
- Discontinued the support for cross compilation to Win32. That was too limited
and the design choice is to have a running CPython instance of matching
architecture at Nuitka compile time.
New Tests
---------
- Expanded test coverage for "standalone mode" demonstrating usage of "hex"
encoding, and PySide package.
Summary
-------
The "executable by default" interface change improves on the already high ease
of use. The new optimization do not give all that much in terms of numbers, but
are all signs of structural improvements, and it is steadily approaching the
point, where the really interesting stuff will happen.
The progress for standalone mode is of course significant. It is still not quite
there yet, but it is making quick progress now. This will attract a lot of
attention hopefully.
As for optimization, the focus for it has shifted to making exception handlers
work optimal by default (publish the exception to sys.exc_info() and create
traceback only when necessary) and be based on standard branches. Removing
special handling of exception handlers, will be the next big step. This release
includes some correctness fixes stemming from that work already.
Nuitka Release 0.4.7
====================
This release includes important new features, lots of polishing cleanups, and
some important performance improvements as well.
Bug Fixes
---------
- The RPM packages didn't build due to missing inline copy of Scons. Fixed in
0.4.6.1 already.
- The recursion into modules and unfreezing them was not working for packages
and modules anymore. Fixed in 0.4.6.2 already.
- The Windows installer was not including Scons. Fixed in 0.4.6.3 already.
- Windows: The immediate execution as performed by ``nuitka --execute`` was not
preserving the exit code. `Issue#26 `__.
- Python3.3: Packages without ``__init.py__`` were not properly embedding the
name-space package as well.
- Python3: Fix, modules and packages didn't add themselves to ``sys.modules``
which they should, happened only for programs.
- Python3.3: Packages should set ``__package`` to their own name, not the one of
their parents.
- Python3.3: The ``__qualname__`` of nested classes was corrected.
- For modules that recursed to other modules, an infinite loop could be
triggered when comparing types with rich comparisons. `Issue#115
`__.
New Features
------------
- The "standalone" mode allows to compile standalone binaries for programs and
run them without Python installation. The DLLs loaded by extension modules on
Windows need to be added manually, on Linux these are determined
automatically already.
To achieve running without Python installation, Nuitka learned to freeze
bytecode as an alternative to compiling modules, as some modules need to be
present when the CPython library is initialized.
- New option ``--python-flag`` allows to specify flags to the compiler that the
"python" binary normally would. So far ``-S`` and ``-v`` are supported, with
sane aliases ``no_site`` and ``trace_imports``.
The recommended use of ``--python-flag=-S`` is to avoid dependency creep in
standalone mode compilations, because the ``site`` module often imports many
useless things that often don't apply to target systems.
New Optimization
----------------
- Faster frame stack handling for functions without ``try``/``except`` (or
``try``/``finally`` in Python3). This gives a speed boost to "PyStone" of
ca. 2.5% overall.
- Python2: Faster attribute getting and setting, handling special cases at
compile time. This gives a minor speed boost to "PyStone" of ca. 0.5% overall.
- Python2: Much quicker calls of ``__getattr__`` and ``__setattr__`` as this is
now using the quicker call method avoiding temporary tuples.
- Don't treat variables usages used in functions called directly by their owner
as shared. This leads to more efficient code generation for contractions and
class bodies.
- Create ``unicode`` constants directly from their UTF-8 string representation
for Python2 as well instead of un-streaming. So far this was only done for
Python3. Affects only program start-up.
- Directly create ``int`` and ``long`` constants outside of ``2**31`` and
``2**32-1``, but only limited according to actual platform values. Affects
only program start-up.
- When creating ``set`` values, no longer use a temporary ``tuple`` value, but
use a properly generated helper functions instead. This makes creating sets
much faster.
- Directly create ``set`` constants instead of un-streaming them. Affects only
program start-up.
- For correct line numbers in traceback, the current frame line number must be
updated during execution. This was done more often than necessary, e.g. loops
set the line number before loop entry, and at first statement.
- Module variables are now accessed even faster, the gain for "PyStone" is only
0.1% and mostly the result of leaner code.
Organizational
--------------
- The "standalone mode" code (formerly known as "portable mode" has been redone
and activated. This is a feature that a lot of people expect from a compiler
naturally. And although the overall goal for Nuitka is of course acceleration,
this kind of packaging is one of the areas where CPython needs improvement.
- Added package for Ubuntu 13.10 for download, removed packages for Ubuntu 11.04
and 11.10, no more supported.
- Added package for openSUSE 13.1 for download.
- Nuitka is now part of Arch and can be installed with ``pacman -S nuitka``.
- Using dedicated `Buildbot `__ for continuous integration
testing. Not yet public.
- Windows: In order to speed up repeated compilation on a platform without
``ccache``, added Scons level caching in the build directory.
- Disabled hash randomization for inside Nuitka (but not in ultimately created
binaries) for a more stable output, because dictionary constants will not
change around. This makes the build results possible to cache for ``ccache``
and Scons as well.
Tests
-----
- The ``programs`` tests cases now fail if module or directory recursion is not
working, being executed in another directory.
- Added test runner for packages, with initial test case for package with
recursion and sub-packages.
- Made some test cases more strict by reducing ``PYTHONPATH`` provision.
- Detect use of extra flags in tests that don't get consumed avoiding
ineffective flags.
- Use ``--execute`` on Windows as well, the issue that prevented it has been
solved after all.
Cleanups
--------
- The generated code uses ``const_``, ``var_``, ``par_`` prefixes in the
generated code and centralized the decision about these into single place.
- Module variables no longer use C++ classes for their access, but instead
accessor functions, leading to much less code generated per module variable
and removing the need to trace their usage during code generation.
- The test runners now share common code in a dedicated module, previously they
replicated it all, but that turned out to be too tedious.
- Massive general cleanups, many of which came from new contributor Juan Carlos
Paco.
- Moved standalone and freezer related codes to dedicated package
``nuitka.freezer`` to not pollute the ``nuitka`` package name space.
- The code generation use variable identifiers and their accesses was cleaned
up.
- Removed several not-so-special case identifier classes because they now behave
more identical and all work the same way, so a parameters can be used to
distinguish them.
- Moved main program, function object, set related code generation to dedicated
modules.
Summary
-------
This release marks major technological progress with the introduction of the
much sought standalone mode and performance improvements from improved code
generation.
The major break through for SSA optimization was not yet achieved, but this is
again making progress in the direction of it. Harmonizing variables of different
kinds was an important step ahead.
Also very nice is the packaging progress, Nuitka was accepted into Arch after
being in Debian Testing for a while already. Hope is to see more of this kind of
integration in the future.
Nuitka Release 0.4.6
====================
This release includes progress on all fronts. The primary focus was to advance
SSA optimization over older optimization code that was already in place. In this
domain, there are mostly cleanups.
Another focus has been to enhance Scons with MSVC on Windows. Nuitka now finds
an installed MSVC compiler automatically, properly handles architecture of
Python and Windows. This improves usability a lot.
Then this is also very much about bug fixes. There have been several hot fixes
for the last release, but a complicated and major issue forced a new release,
and many other small issues.
And then there is performance. As can be seen in the `performance graph
`__, this release is the fastest so
far. This came mainly from examining the need for comparison slots for compiled
types.
And last, but not least, this also expands the base of supported platforms,
adding Gentoo, and self compiled Python to the mix.
Bug Fixes
---------
- Support Nuitka being installed to a path that contains spaces and handle main
programs with spaces in their paths. `Issue#106
`__. Fixed in 0.4.5.1 already.
- Support Python being installed to a path that contains spaces. `Issue#106
`__. Fixed in 0.4.5.2 already.
- Windows: User provided constants larger than 65k didn't work with
MSVC. `Issue#108 `__. Fixed in 0.4.5.3
already.
- Windows: The option ``--windows-disable-console`` was not effective with
MSVC. `Issue#107 `__. Fixed in 0.4.5.3
already.
- Windows: For some users, Scons was detecting their MSVC installation properly
already from registry, but it didn't honor the target architecture. `Issue#99
`__. Fixed in 0.4.5.3 already.
- When creating Python modules, these were marked as executable ("x" bit), which
they are of course not. Fixed in 0.4.5.3 already.
- Python3.3: On architectures where ``Py_ssize_t`` is not the same as ``long``
this could lead to errors. Fixed in 0.4.5.3 already.
- Code that was using nested mutable constants and changed the nested ones was
not executing correctly. `Issue#112 `__.
- Python2: Due to list contractions being re-formulated as functions, ``del``
was rejected for the variables assigned in the contraction. `Issue#111
`__.
.. code-block:: python
[ expr(x) for x in iterable() ]
del x # Should work, was gave an unjustified SyntaxError.
New Features
------------
- Compiled types when used in Python comparison now work. Code like this will
work:
.. code-block:: python
def f():
pass
assert type( f ) == types.FunctionType
This of course also works for ``in`` operator, and is another step ahead in
compatibility, and surprising too. And best of all, this works even if the
checking code is not compiled with Nuitka.
- Windows: Detecting MSVC installation from registry, if no compiler is already
present in PATH.
- Windows: Now options ``--mingw`` to force compilation with MinGW.
New Optimization
----------------
- Rich comparisons (``==``, ``<``, and the like) are now faster than ever before
due to a full implementation of its own in Nuitka that eliminates a bit of the
overhead. In the future, we will aim at giving it type hints to make it even
faster. This gives a minor speed boost to PyStone of ca. 0.7% overall.
- Integer comparisons are now treated preferably, as they are in CPython, which
gives 1.3% speed boost to CPython.
- The SSA based analysis is now used to provide variable scopes for temporary
variables as well as reference count needs.
Cleanups
--------
- Replaced "value friend" based optimization code with SSA based optimization,
which allowed to remove complicated and old code that was still used mainly in
optimization of ``or`` and ``and`` expressions.
- Delayed declaration of temp variables and their reference type is now
performed based on information from SSA, which may given more accurate
results. Not using "variable usage" profiles for this anymore.
- The Scons interface and related code got a massive overhaul, making it more
consistent and better documented. Also updated the internal copy to 2.3.0 for
the platforms that use it, mostly Windows.
- Stop using ``os.system`` and ``subprocess.call( ..., shell = True )`` as it is
not really portable at all, use ``subprocess.call( ..., shell = False )``
instead.
- As usual lots of cleanups related to line length issues and PyLint.
Organizational
--------------
- Added support for Gentoo Linux.
- Added support for self compiled Python versions with and without debug
enabled. `Issue#110 `__
- Added use of Nuitka fonts for headers in manuals.
- Does not install inline copy of Scons only on systems where it is not going to
be used, that is mostly non-Windows, and Linux where it is not already
present. This makes for cleaner RPM packages.
Summary
-------
While the SSA stuff is not yet bearing performance fruits, it starts to carry
weight. Taking over the temporary variable handling now also means we can apply
the same stuff to local variables later.
To make up for the delay in SSA driven performance improvements, there is more
traditional code acceleration for rich comparisons, making it significant, and
the bug fixes make Nuitka more compatible than ever.
So give this a roll, it's worth it. And feel free to `join the mailing list
`_ or `make a donation
`__ to support Nuitka.
Nuitka Release 0.4.5
====================
This release incorporates very many bug fixes, most of which were already part
of hot fixes, usability improvements, documentation improvements, new logo,
simpler Python3 on Windows, warnings for recursion options, and so on. So it's
mostly a consolidation release.
Bug Fixes
---------
- When targeting Python 3.x, Nuitka was using "python" to run Scons to run it
under Python 2.x, which is not good enough on systems, where that is already
Python3. Improved to only do the guessing where necessary (i.e. when using the
inline copy of Scons) and then to prefer "python2". `Issue#95
`__. Fixed in 0.4.4.1 already.
- When using Nuitka created binaries inside a "virtualenv", created programs
would instantly crash. The attempt to load and patch ``inspect`` module was
not making sure that ``site`` module was already imported, but inside the
"virtualenv", it cannot be found unless. `Issue#96
`__. Fixed in 0.4.4.1 already.
- The option ``--recurse-directory`` to include plugin directories was
broken. `Issue#97 `__. Fixed in 0.4.4.2
already.
- Python3: Files with "BOM" marker causes the compiler to crash. `Issue#98
`__. Fixed in 0.4.4.2 already.
- Windows: The generated code for ``try``/``return``/``finally`` was working
with gcc (and therefore MinGW), but not with MSVC, causing crashes. `Issue#102
`__. Fixed in 0.4.4.2 already.
- The option ``--recurse-all`` did not recurse to package ``__init__.py`` files
in case ``from x.y import z`` syntax was used. `Issue#100
`__. Fixed in 0.4.4.2 already.
- Python3 on MacOS: Corrected link time error. Fixed in 0.4.4.2 already.
- Python3.3 on Windows: Fixed crash with too many arguments to a kwonly argument
using function. Fixed in 0.4.4.2 already.
- Python3.3 on Windows: Using "yield from" resulted in a link time error. Fixed
in 0.4.4.2 already.
- Windows: Added back XML manifest, found a case where it is needed to prevent
clashes with binary modules.
- Windows: Generators only worked in the main Python threads. Some unusual
threading modules therefore failed.
- Using ``sys.prefix`` to find the Python installation instead of hard coded
paths. `Issue#103 `__.
New Features
------------
- Windows: Python3 finds Python2 installation to run Scons automatically now.
Nuitka itself runs under Python3 just fine, but in order to build the
generated C++ code into binaries, it uses Scons which still needs Python2.
Nuitka will now find the Python2 installation searching Windows registry
instead of requiring hard coded paths.
- Windows: Python2 and Python3 find their headers now even if Python is not
installed to specific paths.
The installation path now is passed on to Scons which then uses it.
- Better error checking for ``--recurse-to`` and ``--recurse-not-to`` arguments,
tell the user not to use directory paths.
- Added a warning for ``--recurse-to`` arguments that end up having no effect to
the final result.
Cleanups
--------
- Import mechanism got cleaned up, stopped using "PyImport_ExtendInittab". It
does not handle packages, and the ``sys.meta_path`` based importer is now well
proven.
- Moved some of the constraint collection code mess into proper places. It still
remains a mess.
Organizational
--------------
- Added ``LICENSE.txt`` file with Apache License 2.0 text to make it more
immediately obvious which license Nuitka is under.
- Added section about Nuitka license to the "`User Manual
`__".
- Added `Nuitka Logo `__
to the distribution.
- Use Nuitka Logo as the bitmap in the Windows installer.
- Use Nuitka Logo in the documentation ("`User Manual
`__" and "`Developer Manual
`__").
- Enhanced documentation to number page numbers starting after table of
contents, removed header/footer from cover pages.
Summary
-------
This release is mostly the result of improvements made based on the surge of
users after Europython 2013. Some people went to extents and reported their
experience very detailed, and so I could aim at making e.g. their misconceptions
about how recursion options work, more obvious through warnings and errors.
This release is not addressing performance improvements. The next release will
be able to focus on that. I am taking my claim of full compatibility very
serious, so any time it's broken, it's the highest priority to restore it.
Nuitka Release 0.4.4
====================
This release marks the point, where Nuitka for the first time supports all major
current Python versions and all major features. It adds Python 3.3 support and
it adds support for threading. And then there is a massive amount of fixes that
improve compatibility even further.
Aside of that, there is major performance work. One side is the optimization of
call performance (to CPython non-compiled functions) and to compiled functions,
both. This gave a serious improvement to performance.
Then of course, we are making other, long term performance progress, as in
"--experimental" mode, the SSA code starts to optimize unused code away. That
code is not yet ready for prime time yet, but the trace structure will hold.
New Features
------------
- Python3.3 support.
The test suite of CPython3.3 passes now too. The ``yield from`` is now
supported, but the improved argument parsing error messages are not
implemented yet.
- Tracing user provided constants, now Nuitka warns about too large constants
produced during optimization.
- Line numbers of expressions are now updates as evaluation progresses. This
almost corrects.
Finally improves `Issue#9 `__. Now only
expression parts that cannot raise, do not update, which can still cause
difference, but much less often, and then definitely useless.
- Experimental support for threads.
Threading appears to work just fine in the most cases. It's not as optimal as
I wanted it to be, but that's going to change with time.
New Optimization
----------------
- Previous corrections for ``==``, ``!=``, and ``<=``, caused a performance
regression for these operations in case of handling identical objects.
For built-in objects of sane types (not ``float``), these operations are now
accelerated again. The overreaching acceleration of ``>=`` was still there
(bug, see below) and has been adapted too.
- Calling non-compiled Python functions from compiled functions was slower than
in CPython. It is now just as fast.
- Calling compiled functions without keyword arguments has been accelerated with
a dedicated entry point that may call the implementation directly and avoid
parameter parsing almost entirely.
- Making calls to compiled and non-compiled Python functions no longer requires
to build a temporary tuple and therefore is much faster.
- Parameter parsing code is now more compact, and re-uses error raises, or
creates them on the fly, instead of hard coding it. Saves binary size and
should be more cache friendly.
Bug Fixes
---------
- Corrected false optimization of ``a >= a`` on C++ level.
When it's not done during Nuitka compile time optimization, the rich
comparison helper still contained short cuts for ``>=``. This is now the same
for all the comparison operators.
- Calling a function with default values, not providing it, and not providing a
value for a value without default, was not properly detecting the error, and
instead causing a run time crash.
.. code-block:: python
def f( a, b = 2 ):
pass
f( b = 2 )
This now properly raises the ``TypeError`` exception.
- Constants created with ``+`` could become larger than the normally enforced
limits. Not as likely to become huge, but still potentially an issue.
- The ``vars`` built-in, when used on something without ``__dict__`` attribute,
was giving ``AttributeError`` instead of ``TypeError``.
- When re-cursing to modules at compile time, script directory and current
directory were used last, while at run time, it was the other way around,
which caused overloaded standard library modules to not be embedded. Corrects
`Issue#94 `__.
Thanks for the patch to James Michael DuPont.
- Super without arguments was not raising the correct ``RuntimeError`` exception
in functions that cannot be methods, but ``UnboundLocalError`` instead.
.. code-block:: python
def f():
super() # Error, cannot refer to first argument of f
- Generators no longer use ``raise StopIteration`` for return statements,
because that one is not properly handled in ``try``/``except`` clauses, where
it's not supposed to trigger, while ``try``/``finally`` should be honored.
- Exception error message when throwing non-exceptions into generators was not
compatible.
- The use of ``return`` with value in generators is a ``SyntaxError`` before
Python3.3, but that was not raised.
- Variable names of the "__var" style need to be mangled. This was only done for
classes, but not for functions contained in classes, there they are now
mangled too.
- Python3: Exceptions raised with causes were not properly chaining.
- Python3: Specifying the file encoding corrupted line numbers, making them all
of by one.
Cleanups
--------
- For containers (``tuple``, ``list``, ``set``, ``dict``) defined on the source
code level, Nuitka immediately created constant references from them.
For function calls, class creations, slice objects, this code is now re-used,
and its dictionaries and tuples, may now become constants immediately,
reducing noise in optimization steps.
- The parameter parsing code got cleaned up. There were a lot of relics from
previously explored paths. And error raises were part of the templates, but
now are external code.
- Global variable management moved to module objects and out of "Variables"
module.
- Make sure, nodes in the tree are not shared by accident.
This helped to find a case of duplicate use in the complex call helpers
functions. Code generation will now notice this kind of duplication in debug
mode.
- The complex call helper functions were manually taking variable closure, which
made these functions inconsistent to other functions, e.g. no variable version
was allocated to assignments.
Removing the manual setting of variables allowed a huge reduction of code
volume, as it became more generic code.
- Converting user provided constants to create containers into constants
immediately, to avoid noise from doing this in optimization.
- The ``site`` module is now imported explicitly in the ``__main__`` module, so
it can be handled by the recursion code as well. This will help portable mode.
- Many line length 80 changes, improved comments.
New Tests
---------
- The CPython3.3 test suite was added, and run with both Python3.2 and
Python3.3, finding new bugs.
- The ``doctest`` to code generation didn't successfully handle all tests, most
notably, "test_generators.py" was giving a ``SyntaxError`` and therefore not
actually active. Correcting that improved the coverage of generator testing.
Organizational
--------------
- The portable code is still delayed.
Support for Python3.3 was a higher priority, but the intention is to get it
into shape for Europython still.
Added notes about it being disabled it in the "`User Manual
`__" documentation.
Summary
-------
This release is in preparation for Europython 2013. Wanted to get this much out,
as it changes the status slides quite a bit, and all of that was mostly done in
my Cyprus holiday a while ago.
The portable code has not seen progress. The idea here is to get this into a
development version later.
Nuitka Release 0.4.3
====================
This release expands the reach of Nuitka substantially, as new platforms and
compilers are now supported. A lot of polish has been applied. Under the hood
there is the continued and in-progress effort to implement SSA form in Nuitka.
New Features
------------
- Support for new compiler: Microsoft Visual C++.
You can now use Visual Studio 2008 or Visual Studio 2010 for compiling under
Windows.
- Support for NetBSD.
Nuitka works for at least NetBSD 6.0, older versions may or may not work. This
required fixing bugs in the generic "fibers" implementation.
- Support for Python3 under Windows too.
Nuitka uses Scons to build the generated C++ files. Unfortunately it requires
Python2 to execute, which is not readily available to call from Python3. It
now guesses the default installation paths of CPython 2.7 or CPython 2.6 and
it will use it for running Scons instead. You have to install it to
``C:\Python26`` or ``C:\Python27`` for Nuitka to be able to find it.
- Enhanced Python 3.3 compatibility.
The support the newest version of Python has been extended, improving
compatibility for many minor corner cases.
- Added warning when a user compiles a module and executes it immediately when
that references ``__name__``.
Because very likely the intention was to create an executable. And esp. if
there is code like this:
.. code-block:: python
if __name__ == "__main__":
main()
In module mode, Nuitka will optimize it away, and nothing will happen on
execution. This is because the command
.. code-block:: sh
nuitka --execute module
is behavioral more like
python -c "import module"
and that was a trap for new users.
- All Linux architectures are now supported. Due to changes in how evaluation
order is enforced, we don't have to implement for specific architectures
anymore.
Bug Fixes
---------
- Dictionary creation was not fully compatible.
As revealed by using Nuitka with CPython3.3, the order in which dictionaries
are to be populated needs to be reversed, i.e. CPython adds the last item
first. We didn't observe this before, and it's likely the new dictionary
implementation that finds it.
Given that hash randomization makes dictionaries item order undetermined
anyway, this is more an issue of testing.
- Evaluation order for arguments of calls was not effectively enforced. It is
now done in a standards compliant and therefore fully portable way. The
compilers and platforms so far supported were not affected, but the newly
supported Visual Studio C++ compiler was.
- Using a ``__future__`` import inside a function was giving an assertion,
instead of the proper syntax error.
- Python3: Do not set the attributes ``sys.exc_type``, ``sys.exc_value``,
``sys.exc_traceback``.
- Python3: Annotations of function worked only as long as their definition was
not referring to local variables.
New Optimization
----------------
- Calls with no positional arguments are now using the faster call methods.
The generated C++ code was using the ``()`` constant at call site, when doing
calls that use no positional arguments, which is of course useless.
- For Windows now uses OS "Fibers" for Nuitka "Fibers".
Using threads for fibers was causing only overhead and with this API, MSVC had
less issues too.
Organizational
--------------
- Accepting `Donations `__ via Paypal,
please support funding travels, website, etc.
- The "`User Manual `__" has been
updated with new content. We now do support Visual Studio, documented the
required LLVM version for clang, Win64 and modules may include modules too,
etc. Lots of information was no longer accurate and has been updated.
- The Changelog has been improved for consistency, wordings, and styles.
- Nuitka is now available on the social code platforms as well
* `Bitbucket `__
* `Github `__
* `Gitorious `__
* `Google Code `__
- Removed "clean-up.sh", which is practically useless, as tests now clean up
after themselves reasonably, and with ``git clean -dfx`` working better.
- Removed "create-environment.sh" script, which was only setting the ``PATH``
variable, which is not necessary.
- Added ``check-with-pylint --emacs`` option to make output its work with Emacs
compilation mode, to allow easier fixing of warnings from PyLint.
- Documentation is formatted for 80 columns now, source code will gradually aim
at it too. So far 90 columns were used, and up to 100 tolerated.
Cleanups
--------
- Removed useless manifest and resource file creation under Windows.
Turns out this is no longer needed at all. Either CPython, MinGW, or Windows
improved to no longer need it.
- PyLint massive cleanups and annotations bringing down the number of warnings
by a lot.
- Avoid use of strings and built-ins as run time pre-computed constants that are
not needed for specific Python versions, or Nuitka modes.
- Do not track needed tuple, list, and dict creation code variants in context,
but e.g. in ``nuitka.codegen.TupleCodes`` module instead.
- Introduced an "internal" module to host the complex call helper functions,
instead of just adding it to any module that first uses it.
New Tests
---------
- Added basic tests for order evaluation, where there currently were None.
- Added support for "2to3" execution under Windows too, so we can run tests for
Python3 installations too.
Summary
-------
The release is clearly major step ahead. The new platform support triggered a
whole range of improvements, and means this is truly complete now.
Also there is very much polish in this release, reducing the number of warnings,
updated documentation, the only thing really missing is visible progress with
optimization.
Nuitka Release 0.4.2
====================
This release comes with many bug fixes, some of which are severe. It also
contains new features, like basic Python 3.3 support. And the `performance
diagrams `__ got expanded.
New Features
------------
- Support for FreeBSD.
Nuitka works for at least FreeBSD 9.1, older versions may or may not
work. This required only fixing some "Linuxisms" in the build process.
- New option for warning about compile time detected exception raises.
Nuitka can now warn about exceptions that will be raised at run time.
- Basic Python3.3 support.
The test suite of CPython3.2 passes and fails in a compatible way. New feature
``yield from`` is not yet supported, and the improved argument parsing error
messages are not implemented yet.
Bug Fixes
---------
- Nuitka already supported compilation of "main directories", i.e. directories
with a "__main__.py" file inside. The resulting binary name was "__main__.exe"
though, but now it is "directory.exe"
.. code-block:: sh
# ls directory
__main__.py
# nuitka --exe directory
# ls
directory directory.exe
This makes this usage more obvious, and fixes the older issue `Issue#49
`__ for this feature.
- Evaluation order of binary operators was not enforced.
Nuitka already enforces evaluation order for just about everything. But not
for binary operators it seems. Corrects `Issue#61
`__.
- Providing an ``# coding: no-exist`` was crashing under Python2, and ignored
under Python3, now it does the compatible thing for both.
- Global statements on the compiler level are legal in Python, and were not
handled by Nuitka, they now are.
.. code-block:: python
global a # Not in a function, but on module level. Pointless but legal!
a = 1
Effectively these statements can be ignored. Corrects part of `Issue#65
`__.
- Future imports are only legal when they are at the start of the file. This was
not enforced by Nuitka, making it accept code, which CPython would reject. It
now properly raises a syntax error. Corrects part of `Issue#65
`__.
- Raising exceptions from context was leaking references.
.. code-block:: python
raise ValueError() from None
Under CPython3.2 the above is not allowed (it is acceptable starting
CPython3.3), and was also leaking references to its arguments. Corrects
`Issue#76 `__.
- Importing the module that became ``__main__`` through the module name, didn't
recurse to it.
This also gives a warning. PyBench does it, and then stumbles over the
non-found "pybench" module. Of course, programmers should use ``sys.modules[
"__main__" ]`` to access main module code. Not only because the duplicated
modules don't share data. Corrects `Issue#68
`__.
- Compiled method ``repr`` leaked references when printed.
When printing them, they would not be freed, and subsequently hold references
to the object (and class) they belong to. This could trigger bugs for code
that expects ``__del__`` to run at some point. Corrects `Issue#81
`__.
- The ``super`` built-in leaked references to given object.
This was added, because Python3 needs it. It supplies the arguments to
``super`` automatically, whereas for Python2 the programmer had to do it. And
now it turns out that the object lost a reference, causing similar issues as
above, preventing ``__del__`` to run. Corrects `Issue#81
`__.
- The ``raise`` statement didn't enforce type of third argument.
This Python2-only form of exception raising now checks the type of the third
argument before using it. Plus, when it's None (which is also legal), no
reference to None is leaked.
- Python3 built-in exceptions were strings instead of exceptions.
A gross mistake that went uncaught by test suites. I wonder how. Them being
strings doesn't help their usage of course, fixed. Corrects `Issue#82
`__.
- The ``-nan`` and ``nan`` both exist and make a difference.
A older story continued. There is a sign to ``nan``, which can be copied away
and should be present. This is now also supported by Nuitka. Corrects
`Issue#75 `__.
- Wrong optimization of ``a == a``, ``a != a``, ``a <= a`` on C++ level.
While it's not done during Nuitka optimization, the rich comparison helpers
still contained short cuts for ``==``, ``!=``, and ``<=``.
- The ``sys.executable`` for ``nuitka-python --python-version 3.2`` was still
``python``.
When determining the value for ``sys.executable`` the CPython library code
looks at the name ``exec`` had received. It was ``python`` in all cases, but
now it depends on the running version, so it propagates.
- Keyword only functions with default values were loosing references to
defaults.
.. code-block:: python
def f( *, a = X() )
pass
f()
f() # Can crash, X() should already be released.
This is now corrected. Of course, a Python3 only issue.
- Pressing CTRL-C didn't generate ``KeyboardInterrupt`` in compiled code.
Nuitka never executes "pending calls". It now does, with the upside, that the
solution used, appears to be suitable for threading in Nuitka too. Expect more
to come out of this.
- For ``with`` statements with ``return``, ``break``, or ``continue`` to leave
their body, the ``__exit__`` was not called.
.. code-block:: python
with a: # This called a.__enter__().
return 2 # This didn't call a.__exit__( None, None, None ).
This is of course quite huge, and unfortunately wasn't covered by any test
suite so far. Turns out, the re-formulation of ``with`` statements, was
wrongly using ``try/except/else``, but these ignore the problematic
statements. Only ``try/finally`` does. The enhanced re-formulation now does
the correct thing. Corrects `Issue#59 `__.
- Starting with Python3, absolute imports are now the default.
This was already present for Python3.3, and it turns out that all of Python3
does it.
New Optimization
----------------
- Constants are now much less often created with ``pickle`` module, but created
directly.
This esp. applies for nested constants, now more values become ``is``
identical instead of only ``==`` identical, which indicates a reduced memory
usage.
.. code-block:: python
a = ( "something_special", )
b = "something_special"
assert a[0] is b # Now true
This is not only about memory efficiency, but also about performance. Less
memory usage is more cache friendly, and the "==" operator will be able to
shortcut dramatically in cases of identical objects.
Constants now created without ``pickle`` usage, cover ``float``, ``list``, and
``dict``, which is enough for PyStone to not use it at all, which has been
added support for as well.
- Continue statements might be optimized away.
A terminal ``continue`` in a loop, was not optimized away:
.. code-block:: python
while 1:
something
continue # Now optimized away
The trailing ``continue`` has no effect and can therefore be removed.
.. code-block:: python
while 1:
something
- Loops with only break statements are optimized away.
.. code-block:: python
while 1:
break
A loop immediately broken has of course no effect. Loop conditions are
re-formulated to immediate "if ... : break" checks. Effectively this means
that loops with conditions detected to be always false to see the loop
entirely removed.
New Tests
---------
- Added tests for the found issues.
- Running the programs test suite (i.e. recursion) for Python3.2 and Python3.2
as well, after making adaptation so that the absolute import changes are now
covered.
- Running the "CPython3.2" test suite with Python3.3 based Nuitka works and
found a few minor issues.
Organizational
--------------
- The `Downloads `__ page now offers RPMs
for RHEL6, CentOS6, F17, F18, and openSUSE 12.1, 12.2, 12.3. This large
coverage is thanks to openSUSE build service and "ownssh" for contributing an
RPM spec file.
The page got improved with logos for the distributions.
- Added "ownssh" as contributor.
- Revamped the "`User Manual `__" in
terms of layout, structure, and content.
Summary
-------
This release is the result of much validation work. The amount of fixes the
largest of any release so far. New platforms, basic Python3.3 support,
consolidation all around.
Nuitka Release 0.4.1
====================
This release is the first follow-up with a focus on optimization. The major
highlight is progress towards SSA form in the node tree.
Also a lot of cleanups have been performed, for both the tree building, which is
now considered mostly finished, and will be only reviewed. And for the
optimization part there have been large amounts of changes.
New Features
------------
- Python 3.3 experimental support
* Now compiles many basic tests. Ported the dictionary quick access and update
code to a more generic and useful interface.
* Added support for ``__qualname__`` to classes and functions.
* Small compatibility changes. Some exceptions changed, absolute imports are
now default, etc.
* For comparison tests, the hash randomization is disabled.
- Python 3.2 support has been expanded.
The Python 3.2 on Ubuntu is not providing a helper function that was used by
Nuitka, replaced it with out own code.
Bug fixes
---------
- Default values were not "is" identical.
.. code-block:: python
def defaultKeepsIdentity( arg = "str_value" ):
print arg is "str_value"
defaultKeepsIdentity()
This now prints "True" as it does with CPython. The solution is actually a
general code optimization, see below. `Issue#55
`__
- Usage of ``unicode`` built-in with more than one argument could corrupt the
encoding argument string.
An implementation error of the ``unicode`` was releasing references to
arguments converted to default encoding, which could corrupt it.
- Assigning Python3 function annotations could cause a segmentation fault.
New Optimization
----------------
- Improved propagation of exception raise statements, eliminating more
code. They are now also propagated from all kinds of expressions. Previously
this was more limited. An assertion added will make sure that all raises are
propagated. Also finally, raise expressions are converted into raise
statements, but without any normalization.
.. code-block:: python
# Now optimizing:
raise TypeError, 1/0
# into (minus normalization):
raise ZeroDivisionError, "integer division or modulo by zero"
# Now optimizing:
(1/0).something
# into (minus normalization):
raise ZeroDivisionError, "integer division or modulo by zero"
# Now optimizing:
function( a, 1/0 ).something
# into (minus normalization), notice the side effects of first checking
# function and a as names to be defined, these may be removed only if
# they can be demonstrated to have no effect.
function
a
raise ZeroDivisionError, "integer division or modulo by zero"
There is more examples, where the raise propagation is new, but you get the
idea.
- Conditional expression nodes are now optimized according to the truth value of
the condition, and not only for compile time constants. This covers
e.g. container creations, and other things.
.. code-block:: python
# This was already optimized, as it's a compile time constant.
a if ( "a", ) else b
a if True else b
# These are now optimized, as their truth value is known.
a if ( c, ) else b
a if not (c, ) else b
This is simply taking advantage of infrastructure that now exists. Each node
kind can overload "getTruthValue" and benefit from it. Help would be welcome
to review which ones can be added.
- Function creations only have side effects, when their defaults or annotations
(Python3) do. This allows to remove them entirely, should they be found to be
unused.
- Code generation for constants now shares element values used in tuples.
The general case is currently too complex to solve, but we now make sure
constant tuples (as e.g. used in the default value for the compiled function),
and string constants share the value. This should reduce memory usage and
speed up program start-up.
Cleanups
--------
- Optimization was initially designed around visitors that each did one thing,
and did it well. It turns out though, that this approach is unnecessary, and
constraint collection, allows for the most consistent results. All remaining
optimization has been merged into constraint collection.
- The names of modules containing node classes were harmonized to always be
plural. In the beginning, this was used to convey the information that only a
single node kind would be contained, but that has long changed, and is
unimportant information.
- The class names of nodes were stripped from the "CPython" prefix. Originally
the intent was to express strict correlation to CPython, but with increasing
amounts of re-formulations, this was not used at all, and it's also not
important enough to dominate the class name.
- The re-formulations performed in tree building have moved out of the
"Building" module, into names "ReformulationClasses" e.g., so they are easier
to locate and review. Helpers for node building are now in a separate module,
and generally it's much easier to find the content of interest now.
- Added new re-formulation of ``print`` statements. The conversion to strings is
now made explicit in the node tree.
New Tests
---------
- Added test to cover default value identity.
Organizational
--------------
- The upload of `Nuitka to PyPI `__ has
been repaired and now properly displays project information again.
Summary
-------
The quicker release is mostly a consolidation effort, without actual performance
progress. The progress towards SSA form matter a lot on the outlook front. Once
this is finished, standard compiler algorithms can be added to Nuitka which go
beyond the current peephole optimization.
Nuitka Release 0.4.0
====================
This release brings massive progress on all fronts. The big highlight is of
course: Full Python3.2 support. With this release, the test suite of CPython3.2
is considered passing when compiled with Nuitka.
Then lots of work on optimization and infrastructure. The major goal of this
release was to get in shape for actual optimization. This is also why for the
first time, it is tested that some things are indeed compile time optimized to
spot regressions easier. And we are having performance diagrams, `even if weak
ones `__:
New Features
------------
- Python3.2 is now fully supported.
- Fully correct ``metaclass =`` semantics now correctly supported. It had been
working somewhat previously, but now all the corner cases are covered too.
- Keyword only parameters.
- Annotations of functions return value and their arguments.
- Exception causes, chaining, automatic deletion of exception handlers ``as``
values.
- Added support for starred assigns.
- Unicode variable names are also supported, although it's of course ugly, to
find a way to translate these to C++ ones.
Bug fixes
---------
- Checking compiled code with ``instance( some_function, types.FunctionType )``
as "zope.interfaces" does, was causing compatibility problems. Now this kind
of check passes for compiled functions too. `Issue#53
`__
- The frame of modules had an empty locals dictionary, which is not compatible
to CPython which puts the globals dictionary there too. Also discussed in
`Issue#53 `__
- For nested exceptions and interactions with generator objects, the exceptions
in "sys.exc_info()" were not always fully compatible. They now are.
- The ``range`` builtin was not raising exceptions if given arguments appeared
to not have side effects, but were still illegal, e.g. ``range( [], 1, -1 )``
was optimized away if the value was not used.
- Don't crash on imported modules with syntax errors. Instead, the attempted
recursion is simply not done.
- Doing a ``del`` on ``__defaults`` and ``__module__`` of compiled functions was
crashing. This was noticed by a Python3 test for ``__kwdefaults__`` that
exposed this compiled functions weakness.
- Wasn't detecting duplicate arguments, if one of them was not a plain
arguments. Star arguments could collide with normal ones.
- The ``__doc__`` of classes is now only set, where it was in fact
specified. Otherwise it only polluted the name space of ``locals()``.
- When ``return`` from the tried statements of a ``try/finally`` block, was
overridden, by the final block, a reference was leaked. Example code:
.. code-block:: python
try:
return 1
finally:
return 2
- Raising exception instances with value, was leaking references, and not
raising the ``TypeError`` error it is supposed to do.
- When raising with multiple arguments, the evaluation order of them was not
enforced, it now is. This fixes a reference leak when raising exceptions,
where building the exception was raising an exception.
New Optimization
----------------
- Optimizing attribute access to compile time constants for the first time. The
old registry had no actual user yet.
- Optimizing subscript and slices for all compile time constants beyond constant
values, made easy by using inheritance.
- Built-in references now convert to strings directly, e.g. when used in a print
statement. Needed for the testing approach "compiled file contains only prints
with constant value".
- Optimizing calls to constant nodes directly into exceptions.
- Optimizing built-in ``bool`` for arguments with known truth value. This would
be creations of tuples, lists, and dictionaries.
- Optimizing ``a is b`` and ``a is not b`` based on aliasing interface, which at
this time effectively is limited to telling that ``a is a`` is true and ``a is
not a`` is false, but this will expand.
- Added support for optimizing ``hasattr``, ``getattr``, and ``setattr``
built-ins as well. The ``hasattr`` was needed for the ``class`` re-formulation
of Python3 anyway.
- Optimizing ``getattr`` with string argument and no default to simple attribute
access.
- Added support for optimizing ``isinstance`` built-in.
- Was handling "BreakException" and "ContinueException" in all loops that used
``break`` or ``continue`` instead of only where necessary.
- When catching "ReturnValueException", was raising an exception where a normal
return was sufficient. Raising them now only where needed, which also means,
function need not catch them ever.
Cleanups
--------
- The handling of classes for Python2 and Python3 have been re-formulated in
Python more completely.
* The calling of the determined "metaclass" is now in the node tree, so this
call may possible to inline in the future. This eliminated some static C++
code.
* Passing of values into dictionary creation function is no longer using hard
coded special parameters, but temporary variables can now have closure
references, making this normal and visible to the optimization.
* Class dictionary creation functions are therefore no longer as special as
they used to be.
* There is no class creation node anymore, it's merely a call to ``type`` or
the metaclass detected.
- Re-formulated complex calls through helper functions that process the star
list and dict arguments and do merges, checks, etc.
* Moves much C++ code into the node tree visibility.
* Will allow optimization to eliminate checks and to compile time merge, once
inline functions and loop unrolling are supported.
- Added "return None" to function bodies without a an aborting statement at the
end, and removed the hard coded fallback from function templates. Makes it
explicit in the node tree and available for optimization.
- Merged C++ classes for frame exception keeper with frame guards.
* The exception is now saved in the compiled frame object, making it
potentially more compatible to start with.
* Aligned module and function frame guard usage, now using the same class.
* There is now a clear difference in the frame guard classes. One is for
generators and one is for functions, allowing to implement their different
exception behavior there.
- The optimization registries for calls, subscripts, slices, and attributes have
been replaced with attaching them to nodes.
* The ensuing circular dependency has been resolved by more local imports for
created nodes.
* The package "nuitka.transform.optimization.registries" is no more.
* New per node methods "computeNodeCall", "computeNodeSubscript",
etc. dispatch the optimization process to the nodes directly.
- Use the standard frame guard code generation for modules too.
* Added a variant "once", that avoids caching of frames entirely.
- The variable closure taking has been cleaned up.
* Stages are now properly numbered.
* Python3 only stage is not executed for Python2 anymore.
* Added comments explaining things a bit better.
* Now an early step done directly after building a tree.
- The special code generation used for unpacking from iterators and catching
"StopIteration" was cleaned up.
* Now uses template, Generator functions, and proper identifiers.
- The ``return`` statements in generators are now re-formulated into ``raise
StopIteration`` for generators, because that's what they really are. Allowed
to remove special handling of ``return`` nodes in generators.
- The specialty of CPython2.6 yielding non-None values of lambda generators, was
so far implemented in code generation. This was moved to tree building as a
re-formulation, making it subject to normal optimization.
- Mangling of attribute names in functions contained in classes, has been moved
into the early tree building. So far it was done during code generation,
making it invisible to the optimization stages.
- Removed tags attribute from node classes. This was once intended to make up
for non-inheritance of similar node kinds, but since we have function
references, the structure got so clean, it's no more needed.
- Introduced new package ``nuitka.tree``, where the building of node trees, and
operations on them live, as well as recursion and variable closure.
- Removed ``nuitka.transform`` and move its former children
``nuitka.optimization`` and ``nuitka.finalization`` one level up. The deeply
nested structure turned out to have no advantage.
- Checks for Python version was sometimes "> 300", where of course ">= 300" is
the only thing that makes sense.
- Split out helper code for exception raising from the handling of exception
objects.
New Tests
---------
- The complete CPython3.2 test suite was adapted (no ``__code__``, no
``__closure__``, etc.) and is now passing, but only without "--debug", because
otherwise some of the generated C++ triggers (harmless) warnings.
- Added new test suite designed to prove that expressions that are known to be
compile time constant are indeed so. This works using the XML output done with
"--dump-xml" and then searching it to only have print statements with constant
values.
- Added new basic CPython3.2 test "Functions32" and "ParameterErrors32" to cover
keyword only parameter handling.
- Added tests to cover generator object and exception interactions.
- Added tests to cover ``try/finally`` and ``return`` in one or both branches
correctly handling the references.
- Added tests to cover evaluation order of arguments when raising exceptions.
Organizational
--------------
- Changed my email from GMX over to Gmail, the old one will still continue to
work. Updated the copyright notices accordingly.
- Uploaded `Nuitka to PyPI `__ as well.
Summary
-------
This release marks a milestone. The support of Python3 is here. The
re-formulation of complex calls, and the code generation improvements are quite
huge. More re-formulation could be done for argument parsing, but generally this
is now mostly complete.
The 0.3.x series had a lot releases. Many of which brought progress with
re-formulations that aimed at making optimization easier or possible. Sometimes
small things like making "return None" explicit. Sometimes bigger things, like
making class creations normal functions, or getting rid of ``or`` and
``and``. All of this was important ground work, to make sure, that optimization
doesn't deal with complex stuff.
So, the 0.4.x series begins with this. The focus from now on can be almost
purely optimization. This release contains already some of it, with frames being
optimized away, with the assignment keepers from the ``or`` and ``and``
re-formulation being optimized away. This will be about achieving goals from the
"ctypes" plan as discussed in the developer manual.
Also the performance page will be expanded with more benchmarks and diagrams as
I go forward. I have finally given up on "codespeed", and do my own diagrams.
Nuitka Release 0.3.25
=====================
This release brings about changes on all fronts, bug fixes, new features. Also
very importantly Nuitka no longer uses C++11 for its code, but mere C++03. There
is new re-formulation work, and re-factoring of functions.
But the most important part is this: Mercurial unit tests are
working. Nearly. With the usual disclaimer of me being wrong, all remaining
errors are errors of the test, or minor things. Hope is that these unit tests
can be added as release tests to Nuitka. And once that is done, the next big
Python application can come.
Bug fixes
---------
- Local variables were released when an exception was raised that escaped the
local function. They should only be released, after another exception was
raised somewhere. `Issue#39 `__.
- Identifiers of nested tuples and lists could collide.
.. code-block:: python
a = ( ( 1, 2 ), 3 )
b = ( ( 1, ), 2, 3 )
Both tuples had the same name previously, not the end of the tuple is marked
too. Fixed in 0.3.24.1 already.
- The ``__name__`` when used read-only in modules in packages was optimized to a
string value that didn't contain the package name.
- Exceptions set when entering compiled functions were unset at function exit.
New Features
------------
- Compiled frames support. Before, Nuitka was creating frames with the standard
CPython C/API functions, and tried its best to cache them. This involved some
difficulties, but as it turns out, it is actually possible to instead provide
a compatible type of our own, that we have full control over.
This will become the base of enhanced compatibility. Keeping references to
local variables attached to exception tracebacks is something we may be able
to solve now.
- Enhanced Python3 support, added support for ``nonlocal`` declarations and many
small corrections for it.
- Writable ``__defaults__`` attribute for compiled functions, actually changes
the default value used at call time. Not supported is changing the amount of
default parameters.
Cleanups
--------
- Keep the functions along with the module and added "FunctionRef" node kind to
point to them.
- Reformulated ``or`` and ``and`` operators with the conditional expression
construct which makes the "short-circuit" branch.
- Access ``self`` in methods from the compiled function object instead of
pointer to context object, making it possible to access the function object.
- Removed "OverflowCheck" module and its usage, avoids one useless scan per
function to determine the need for "locals dictionary".
- Make "compileTree" of "MainControl" module to only do what the name says and
moved the rest out, making the top level control clearer.
- Don't export module entry points when building executable and not
modules. These exports cause MinGW and MSVC compilers to create export
libraries.
New Optimization
----------------
- More efficient code for conditional expressions in conditions:
.. code-block:: python
if a if b else c
See above, this code is now the typical pattern for each ``or`` and ``and``,
so this was much needed now.
Organizational
--------------
- The remaining uses of C++11 have been removed. Code generated with Nuitka and
complementary C++ code now compile with standard C++03 compilers. This lowers
the Nuitka requirements and enables at least g++ 4.4 to work with Nuitka.
- The usages of the GNU extension operation ``a ?: b`` have replaced with
standard C++ constructs. This is needed to support MSVC which doesn't have
this.
- Added examples for the typical use cases to the "`User Manual
`__".
- The "compare_with_cpython" script has gained an option to immediately remove
the Nuitka outputs (build directory and binary) if successful. Also the
temporary files are now put under "/var/tmp" if available.
- Debian package improvements, registering with "doc-base" the "`User Manual
`__" so it is easier to discover. Also
suggest "mingw32" package which provides the cross compiler to Windows.
- Partial support for MSVC (Visual Studio 2008 to be exact, the version that
works with CPython2.6 and CPython2.7).
All basic tests that do not use generators are working now, but those will
currently cause crashes.
- Renamed the ``--g++-only`` option to ``--c++-only``.
The old name is no longer correct after clang and MSVC have gained support,
and it could be misunderstood to influence compiler selection, rather than
causing the C++ source code to not be updated, so manual changes will the
used. This solves `Issue#47 `__.
- Catch exceptions for ``continue``, ``break``, and ``return`` only where needed
for ``try``/``finally`` and loop constructs.
New Tests
---------
- Added CPython3.2 test suite as "tests/CPython32" from 3.2.3 and run it with
CPython2.7 to check that Nuitka gives compatible error messages. It is not
expected to pass yet on Python3.2, but work will be done towards this goal.
- Make CPython2.7 test suite runner also execute the generated "doctest"
modules.
- Enabled tests for default parameters and their reference counts.
Summary
-------
This release marks an important point. The compiled frames are exciting new
technology, that will allow even better integration with CPython, while
improving speed. Lowering the requirements to C++03 means, we will become usable
on Android and with MSVC, which will make adoption of Nuitka on Windows easier
for many.
Structurally the outstanding part is the function as references cleanup. This
was a blocker for value propagation, because now functions references can be
copied, whereas previously this was duplicating the whole function body, which
didn't work, and wasn't acceptable. Now, work can resume in this domain.
Also very exciting when it comes to optimization is the remove of special code
for ``or`` and ``and`` operators, as these are now only mere conditional
expressions. Again, this will make value propagation easier with two special
cases less.
And then of course, with Mercurial unit tests running compiled with Nuitka, an
important milestone has been hit.
For a while now, the focus will be on completing Python3 support, XML based
optimization regression tests, benchmarks, and other open ends. Once that is
done, and more certainty about Mercurial tests support, I may call it a 0.4 and
start with local type inference for actual speed gains.
Nuitka Release 0.3.24
=====================
This release contains progress on many fronts, except performance.
The extended coverage from running the CPython 2.7 and CPython 3.2 (partially)
test suites shows in a couple of bug fixes and general improvements in
compatibility.
Then there is a promised new feature that allows to compile whole packages.
Also there is more Python3 compatibility, the CPython 3.2 test suite now
succeeds up to "test_builtin.py", where it finds that ``str`` doesn't support
the new parameters it has gained, future releases will improve on this.
And then of course, more re-formulation work, in this case, class definitions
are now mere simple functions. This and later function references, is the
important and only progress towards type inference.
Bug fixes
---------
- The compiled method type can now be used with ``copy`` module. That means,
instances with methods can now be copied too. `Issue#40
`__. Fixed in 0.3.23.1 already.
- The ``assert`` statement as of Python2.7 creates the ``AssertionError`` object
from a given value immediately, instead of delayed as it was with
Python2.6. This makes a difference for the form with 2 arguments, and if the
value is a tuple. `Issue#41 `__. Fixed in
0.3.23.1 already.
- Sets written like this didn't work unless they were predicted at compile time:
.. code-block:: python
{ value }
This apparently rarely used Python2.7 syntax didn't have code generation yet
and crashed the compiler. `Issue#42 `__. Fixed
in 0.3.23.1 already.
- For Python2, the default encoding for source files is ``ascii``, and it is now
enforced by Nuitka as well, with the same ``SyntaxError``.
- Corner cases of ``exec`` statements with nested functions now give proper
``SyntaxError`` exceptions under Python2.
- The ``exec`` statement with a tuple of length 1 as argument, now also gives a
``TypeError`` exception under Python2.
- For Python2, the ``del`` of a closure variable is a ``SyntaxError``.
New Features
------------
- Added support creating compiled packages. If you give Nuitka a directory with
an "__init__.py" file, it will compile that package into a ".so" file. Adding
the package contents with ``--recurse-dir`` allows to compile complete
packages now. Later there will be a cleaner interface likely, where the later
is automatic.
- Added support for providing directories as main programs. It's OK if they
contain a "__main__.py" file, then it's used instead, otherwise give
compatible error message.
- Added support for optimizing the ``super`` built-in. It was already working
correctly, but not optimized on CPython2. But for CPython3, the variant
without any arguments required dedicated code.
- Added support for optimizing the ``unicode`` built-in under Python2. It was
already working, but will become the basis for the ``str`` built-in of Python3
in future releases.
- For Python3, lots of compatibility work has been done. The Unicode issues
appear to be ironed out now. The ``del`` of closure variables is allowed and
supported now. Built-ins like ``ord`` and ``chr`` work more correctly and
attributes are now interned strings, so that monkey patching classes works.
Organizational
--------------
- Migrated "bin/benchmark.sh" to Python as "misc/run-valgrind.py" and made it a
bit more portable that way. Prefers "/var/tmp" if it exists and creates
temporary files in a secure manner. Triggered by the Debian "insecure temp
file" bug.
- Migrated "bin/make-dependency-graph.sh" to Python as
"misc/make-dependency-graph.py" and made a more portable and powerful that
way.
The filtering is done a more robust way. Also it creates temporary files in a
secure manner, also triggered by the Debian "insecure temp file" bug.
And it creates SVG files and no longer PostScript as the first one is more
easily rendered these days.
- Removed the "misc/gist" git sub-module, which was previously used by
"misc/make-doc.py" to generate HTML from "`User Manual
`__" and "`Developer Manual
`__".
These are now done with Nikola, which is much better at it and it integrates
with the web site.
- Lots of formatting improvements to the change log, and manuals:
* Marking identifiers with better suited ReStructured Text markup.
* Added links to the bug tracker all Issues.
* Unified wordings, quotation, across the documents.
Cleanups
--------
- The creation of the class dictionaries is now done with normal function
bodies, that only needed to learn how to throw an exception when directly
called, instead of returning ``NULL``.
Also the assignment of ``__module__`` and ``__doc__`` in these has become
visible in the node tree, allowing their proper optimization.
These re-formulation changes allowed to remove all sorts of special treatment
of ``class`` code in the code generation phase, making things a lot simpler.
- There was still a declaration of ``PRINT_ITEMS`` and uses of it, but no
definition of it.
- Code generation for "main" module and "other" modules are now merged, and no
longer special.
- The use of raw strings was found unnecessary and potentially still buggy and
has been removed. The dependence on C++11 is getting less and less.
New Tests
---------
- Updated CPython2.6 test suite "tests/CPython26" to 2.6.8, adding tests for
recent bug fixes in CPython. No changes to Nuitka were needed in order to
pass, which is always good news.
- Added CPython2.7 test suite as "tests/CPython27" from 2.7.3, making it public
for the first time. Previously a private copy of some age, with many no longer
needed changes had been used by me. Now it is up to par with what was done
before for "tests/CPython26", so this pending action is finally done.
- Added test to cover Python2 syntax error of having a function with closure
variables nested inside a function that is an overflow function.
- Added test "BuiltinSuper" to cover ``super`` usage details.
- Added test to cover ``del`` on nested scope as syntax error.
- Added test to cover ``exec`` with a tuple argument of length 1.
- Added test to cover ``barry_as_FLUFL`` future import to work.
- Removed "Unicode" from known error cases for CPython3.2, it's now working.
Summary
-------
This release brought forward the most important remaining re-formulation changes
needed for Nuitka. Removing class bodies, makes optimization yet again
simpler. Still, making function references, so they can be copied, is missing
for value propagation to progress.
Generally, as usual, a focus has been laid on correctness. This is also the
first time, I am release with a known bug though: That is `Issue#39
`__ which I believe now, may be the root cause
of the mercurial tests not yet passing.
The solution will be involved and take a bit of time. It will be about "compiled
frames" and be a (invasive) solution. It likely will make Nuitka faster too. But
this release includes lots of tiny improvements, for Python3 and also for
Python2. So I wanted to get this out now.
As usual, please check it out, and let me know how you fare.
Nuitka Release 0.3.23
=====================
This release is the one that completes the Nuitka "sun rise phase".
All of Nuitka is now released under `Apache License 2.0
`__ which is a very liberal license,
and compatible with basically all Free Software licenses there are. It's only
asking to allow integration, of what you send back, and patent grants for the
code.
In the first phase of Nuitka development, I wanted to keep control over Nuitka,
so it wouldn't repeat mistakes of other projects. This is no longer a concern
for me, it's not going to happen anymore.
I would like to thank Debian Legal team, for originally bringing to my
attention, that this license will be better suited, than any copyright
assignment could be.
Bug fixes
---------
- The compiled functions could not be used with ``multiprocessing`` or
``copy.copy``. `Issue#19 `__. Fixed in
0.3.22.1 already.
- In-place operations for slices with not both bounds specified crashed the
compiler. `Issue#36 `__. Fixed in 0.3.22.1
already.
- Cyclic imports could trigger an endless loop, because module import
expressions became the parent of the imported module object. `Issue#37
`__. Fixed in 0.3.22.2 already.
- Modules named ``proc`` or ``func`` could not be compiled to modules or
embedded due to a collision with identifiers of CPython2.7 includes. `Issue#38
`__. Fixed in 0.3.22.2 already.
New Features
------------
- The fix for `Issue#19 `__ also makes pickling
of compiled functions available. As it is the case for non-compiled functions
in CPython, no code objects are stored, only names of module level variables.
Organizational
--------------
- Using the Apache License 2.0 for all of Nuitka now.
- Speedcenter has been re-activated, but is not yet having a lot of benchmarks
yet, subject to change.
.. admonition:: Update
We have given up on speedcenter meanwhile, and generate static pages with
graphs instead.
New Tests
---------
- Changed the "CPython26" tests to no longer disable the parts that relied on
copying of functions to work, as `Issue#19 `__
is now supported.
- Extended in-place assignment tests to cover error cases of `Issue#36
`__.
- Extended compile library test to also try and compile the path where ``numpy``
lives. This is apparently another path, where Debian installs some modules,
and compiling this would have revealed `Issue#36
`__ sooner.
Summary
-------
The release contains bug fixes, and the huge step of changing `the license
`__. It is made in preparation to
`PyCON EU `__.
Nuitka Release 0.3.22
=====================
This release is a continuation of the trend of previous releases, and added more
re-formulations of Python that lower the burden on code generation and
optimization.
It also improves Python3 support substantially. In fact this is the first
release to not only run itself under Python3, but for Nuitka to *compile itself*
with Nuitka under Python3, which previously only worked for Python2. For the
common language subset, it's quite fine now.
Bug fixes
---------
- List contractions produced extra entries on the call stack, after they became
functions, these are no more existent. That was made possible my making frame
stack entries an optional element in the node tree, left out for list
contractions.
- Calling a compiled function in an exception handler cleared the exception on
return, it no longer does that.
- Reference counter handling with generator ``throw`` method is now correct.
- A module "builtins" conflicted with the handling of the Python ``builtins``
module. Those now use different identifiers.
New Features
------------
- New ``metaclass`` syntax for the ``class`` statement works, and the old
``__metaclass__`` attribute is properly ignored.
.. code-block:: python
# Metaclass syntax in Python3, illegal in Python2
class X(metaclass = Y):
pass
.. code-block:: python
# Metaclass syntax in Python2, no effect in Python3
class X:
__metaclass__ = Y
.. note::
The way to make a use of a metaclass in a portable way, is to create a
based class that has it and then inherit from it. Sad, isn' it. Surely, the
support for ``__metaclass__`` could still live.
.. code-block:: python
# For Python2/3 compatible source, we create a base class that has the
# metaclass used and doesn't require making a choice.
CPythonNodeMetaClassBase = NodeCheckMetaClass( "CPythonNodeMetaClassBase", (object, ), {} )
- The ``--dump-xml`` option works with Nuitka running under Python3. This was
not previously supported.
- Python3 now also has compatible parameter errors and compatible exception
error messages.
- Python3 has changed scope rules for list contractions (assignments don't
affect outside values) and this is now respected as well.
- Python3 has gained support for recursive programs and stand alone extension
modules, these are now both possible as well.
New Optimization
----------------
- Avoid frame stack entries for functions that cannot raise exceptions,
i.e. where they would not be used.
This avoids overhead for the very simple functions. And example of this can be
seen here:
.. code-block:: python
def simple():
return 7
- Optimize ``len`` built-in for non-constant, but known length values.
An example can be seen here:
.. code-block:: python
# The range isn't constructed at compile time, but we still know its
# length.
len( range( 10000000 ) )
# The string isn't constructed at compile time, but we still know its
# length.
len( "*" * 1000 )
# The tuple isn't constructed, instead it's known length is used, and
# side effects are maintained.
len( ( a(), b() ) )
This new optimization applies to all kinds of container creations and the
``range`` built-in initially.
- Optimize conditions for non-constant, but known truth values.
At this time, known truth values of non-constants means ``range`` built-in
calls with know size and container creations.
An example can be seen here:
.. code-block:: python
if ( a, ):
print "In Branch"
It's clear, that the tuple will be true, we just need to maintain the side
effect, which we do.
- Optimize ``or`` and ``and`` operators for known truth values.
See above for what has known truth values currently. This will be most useful
to predict conditions that need not be evaluated at all due to short circuit
nature, and to avoid checking against constant values. Previously this could
not be optimized, but now it can:
.. code-block:: python
# The access and call to "something()" cannot possibly happen
0 and something()
# Can be replaced with "something()", as "1" is true. If it had a side effect, it
# would be maintained.
1 and something()
# The access and call to "something()" cannot possibly happen, the value is already
# decided, it's "1".
1 or something()
# Can be replaced with "something()", as "0" is false. If it had a side effect, it
# would be maintained.
0 or something()
- Optimize print arguments to become strings.
The arguments to ``print`` statements are now converted to strings at compile
time if possible.
.. code-block:: python
print 1
becomes:
.. code-block:: python
print "1"
- Combine print arguments to single ones.
When multiple strings are printed, these are now combined.
.. code-block:: python
print "1+1=", 1+1
becomes:
.. code-block:: python
print "1+1= 2"
Organizational
--------------
- Enhanced Python3 support, enabling support for most basic tests.
- Check files with PyLint in deterministic (alphabetical) order.
Cleanups
--------
- Frame stack entries are now part of the node tree instead of part of the
template for every function, generator, class or module.
- The ``try``/``except``/``else`` has been re-formulated to use an indicator
variable visible in the node tree, that tells if a handler has been executed
or not.
- Side effects are now a dedicated node, used in several optimization to
maintain the effect of an expression with known value.
New Tests
---------
- Expanded and adapted basic tests to work for Python3 as well.
- Added reference count tests for generator functions ``throw``, ``send``, and
``close`` methods.
- Cover calling a function with ``try``/``except`` in an exception handler
twice. No test was previously doing that.
Summary
-------
This release offers enhanced compatibility with Python3, as well as the solution
to many structural problems. Calculating lengths of large non-constant values at
compile time, is technically a break through, as is avoiding lengthy
calculations. The frame guards as nodes is a huge improvement, making that
costly operational possible to be optimized away.
There still is more work ahead, before value propagation will be safe enough to
enable, but we are seeing the glimpse of it already. Not for long, and looking
at numbers will make sense.
Nuitka Release 0.3.21
=====================
This releases contains some really major enhancements, all heading towards
enabling value propagation inside Nuitka. Assignments of all forms are now all
simple and explicit, and as a result, now it will be easy to start tracking
them.
Contractions have become functions internally, with statements use temporary
variables, complex unpacking statement were reduced to more simple ones, etc.
Also there are the usual few small bug fixes, and a bunch of organizational
improvements, that make the release complete.
Bug fixes
---------
- The built-in ``next`` could causes a program crash when iterating past the end
of an iterator. `Issue#34 `__. Fixed in
0.3.20.1 already.
- The ``set`` constants could cause a compiler error, as that type was not
considered in the "mutable" check yet. Fixed in 0.3.20.2 already.
- Performance regression. Optimize expression for exception types caught as well
again, this was lost in last release.
- Functions that contain ``exec``, are supposed to have a writable locals. But
when removing that ``exec`` statement as part of optimization, this property
of the function could get lost.
- The so called "overflow functions" are once again correctly handled. These
once were left behind in some refactoring and had not been repaired until
now. An overflow function is a nested function with an ``exec`` or a star
import.
- The syntax error for ``return`` outside of a function, was not given, instead
the code returned at run time. Fixed to raise a ``SyntaxError`` at compile
time.
New Optimization
----------------
- Avoid ``tuple`` objects to be created when catching multiple exception types,
instead call exception match check function multiple times.
- Removal of dead code following ``break``, ``continue``, ``return``, and
``raise``. Code that follows these statements, or conditional statements,
where all branches end with it.
.. note::
These may not actually occur often in actual code, but future
optimization may produce them more frequently, and their removal may in
turn make other possible optimization.
- Detect module variables as "read only" after all writes have been detected to
not be executed as removed. Previously the "read only indicator" was
determined only once and then stayed the same.
- Expanded conditional statement optimization to detect cases, where condition
is a compile time constant, not just a constant value.
- Optimize away assignments from a variable to the same variable, they have no
effect. The potential side effect of accessing the variable is left intact
though, so exceptions will be raised still.
.. note::
An exception is where ``len = len`` actually does have an impact, because
that variable becomes assignable. The "compile itself" test of Nuitka found
that to happen with ``long`` from the ``nuitka.__past__`` module.
- Created Python3 variant of quick ``unicode`` string access, there was no such
thing in the CPython C/API, but we make the distinction in the source code, so
it makes sense to have it.
- Created an optimized implementation for the built-in ``iter`` with 2
parameters as well. This allows for slightly more efficient code to be created
with regards to reference handling, rather than using the CPython C/API.
- For all types of variable assigned in the generated code, there are now
methods that accept already taken references or not, and the code generator
picks the optimal variant. This avoids the drop of references, that e.g. the
local variable will insist to take.
- Don't use a "context" object for generator functions (and generator
expressions) that don't need one. And even if it does to store e.g. the given
parameter values, avoid to have a "common context" if there is no closure
taken. This avoids useless ``malloc`` calls and speeds up repeated generator
object creation.
Organizational
--------------
- Changed the Scons build file database to reside in the build directory as
opposed to the current directory, not polluting it anymore. Thanks for the
patch go to Michael H Kent, very much appreciated.
- The ``--experimental`` option is no longer available outside of checkouts of
git, and even there not on stable branches (``master``, ``hotfix/...``). It
only pollutes ``--help`` output as stable releases have no experimental code
options, not even development version will make a difference.
- The binary "bin/Nuitka.py" has been removed from the git repository. It was
deprecated a while ago, not part of the distribution and served no good use,
as it was a symbolic link only anyway.
- The ``--python-version`` option is applied at Nuitka start time to re-launch
Nuitka with the given Python version, to make sure that the Python run time
used for computations and link time Python versions are the same. The allowed
values are now checked (2.6, 2.7 and 3.2) and the user gets a nice error with
wrong values.
- Added ``--keep-pythonpath`` alias for ``--execute-with-pythonpath`` option,
probably easier to remember.
- Support ``--debug`` with clang, so it can also be used to check the generated
code for all warnings, and perform assertions. Didn't report anything new.
- The contents environment variable ``CXX`` determines the default C++ compiler
when set, so that checking with ``CXX=g++-4.7 nuitka-python ...`` has become
supported.
- The ``check-with-pylint`` script now has a real command line option to control
the display of ``TODO`` items.
Cleanups
--------
- Changed complex assignments, i.e. assignments with multiple targets to such
using a temporary variable and multiple simple assignments instead.
.. code-block:: python
a = b = c
.. code-block:: python
_tmp = c
b = _tmp
a = _tmp
In CPython, when one assignment raises an exception, the whole thing is
aborted, so the complexity of having multiple targets is no more needed, now
that we have temporary variables in a block.
All that was really needed, was to evaluate the complete source expression
only once, but that made code generation contain ugly loops that are no more
needed.
- Changed unpacking assignments to use temporary variables. Code like this:
.. code-block:: python
a, b = c
Is handled more like this:
.. code-block:: python
_tmp_iter = iter( c )
_tmp1 = next( _tmp_iter )
_tmp2 = next( _tmp_iter )
if not finished( _tmp_iter ):
raise ValueError( "too many values to unpack" )
a = _tmp1
b = _tmp2
In reality, not really ``next`` is used, as it wouldn't raise the correct
exception for unpacking, and the ``finished`` check is more condensed into it.
Generally this cleanup allowed that the ``AssignTargetTuple`` and associated
code generation was removed, and in the future value propagation may optimize
these ``next`` and ``iter`` calls away where possible. At this time, this is
not done yet.
- Exception handlers assign caught exception value through assignment statement.
Previously the code generated for assigning from the caught exception was not
considered part of the handler. It now is the first statement of an exception
handler or not present, this way it may be optimized as well.
- Exception handlers now explicitly catch more than one type.
Catching multiple types worked by merits of the created tuple object working
with the Python C/API function called, but that was not explicit at all. Now
every handler has a tuple of exceptions it catches, which may only be one, or
if None, it's all.
- Contractions are now functions as well.
Contractions (list, dict, and set) are now re-formulated as function bodies
that contain for loops and conditional statements. This allowed to remove a
lot of special code that dealt with them and will make these easier to
understand for optimization and value propagation.
- Global is handled during tree building.
Previously the global statement was its own node, which got removed during the
optimization phase in a dedicated early optimization that applied its effect,
and then removed the node.
It was determined, that there is no reason to not immediately apply the effect
of the global variable and take closure variables and add them to the provider
of that ``global`` statement, allowing to remove the node class.
- Read only module variable detection integrated to constraint collection.
The detection of read only module variables was so far done as a separate
step, which is no more necessary as the constraint collection tracks the
usages of module variables anyway, so this separate and slow step could be
removed.
New Tests
---------
- Added test to cover order of calls for complex assignments that unpack, to see
that they make a fresh iterator for each part of a complex assignment.
- Added test that unpacks in an exception catch. It worked, due to the generic
handling of assignment targets by Nuitka, and I didn't even know it can be
done, example:
.. code-block:: python
try:
raise ValueError(1,2)
except ValueError as (a,b):
print "Unpacking caught exception and unpacked", a, b
Will assign ``a=1`` and ``b=2``.
- Added test to cover return statements on module level and class level, they
both must give syntax errors.
- Cover exceptions from accessing unassigned global names.
- Added syntax test to show that star imports do not allow other names to be
imported at the same time as well.
- Python3 is now also running the compile itself test successfully.
Summary
-------
The progress made towards value propagation and type inference is *very*
significant, and makes those appears as if they are achievable.
Nuitka Release 0.3.20
=====================
This time there are a few bug fixes and some really major cleanups, lots of new
optimization and preparations for more. And then there is a new compiler clang
and a new platform supported. MacOS X appears to work mostly, thanks for the
patches from Pete Hunt.
Bug fixes
---------
- The use of a local variable name as an expression was not covered and lead to
a compiler crash. Totally amazing, but true, nothing in the test suite of
CPython covered this. `Issue#30 `__. Fixed in
release 0.3.19.1 already.
- The use of a closure variable name as an expression was not covered as
well. And in this case corrupted the reference count. `Issue#31
`__. Fixed in release 0.3.19.1 already.
- The ``from x import *`` attempted to respect ``__all__`` but failed to do
so. `Issue#32 `__. Fixed in release 0.3.19.2
already.
- The ``from x import *`` didn't give a ``SyntaxError`` when used on
Python3. Fixed in release 0.3.19.2 already.
- The syntax error messages for "global for function argument name" and
"duplicate function argument name" are now identical as well.
- Parameter values of generator function could cause compilation errors when
used in the closure of list contractions. Fixed.
New Features
------------
- Added support for disabling the console for Windows binaries. Thanks for the
patch go to Michael H Kent.
- Enhanced Python3 support for syntax errors, these are now also compatible.
- Support for MacOS X was added.
- Support for using the clang compiler was added, it can be enforced via
``--clang`` option. Currently this option is mainly intended to allow testing
the "MacOS X" support as good as possible under Linux.
New Optimization
----------------
- Enhanced all optimization that previously worked on "constants" to work on
"compile time constants" instead. A "compile time constant" can currently also
be any form of a built-in name or exception reference. It is intended to
expand this in the future.
- Added support for built-ins ``bin``, ``oct``, and ``hex``, which also can be
computed at compile time, if their arguments are compile time constant.
- Added support for the ``iter`` built-in in both forms, one and two
arguments. These cannot be computed at compile time, but now will execute
faster.
- Added support for the ``next`` built-in, also in its both forms, one and two
arguments. These also cannot be computed at compile time, but now will execute
faster as well.
- Added support for the ``open`` built-in in all its form. We intend for future
releases to be able to track file opens for including them into the executable
if data files.
- Optimize the ``__debug__`` built-in constant as well. It cannot be assigned,
yet code can determine a mode of operation from it, and apparently some code
does. When compiling the mode is decided.
- Optimize the ``Ellipsis`` built-in constant as well. It falls in the same
category as ``True``, ``False``, ``None``, i.e. names of built-in constants
that a singletons.
- Added support for anonymous built-in references, i.e. built-ins which have
names that are not normally accessible. An example is ``type(None)`` which is
not accessible from anywhere. Other examples of such names are
``compiled_method_or_function``.
Having these as represented internally, and flagged as "compile time
constants", allows the compiler to make more compile time optimization and to
generate more efficient C++ code for it that won't e.g. call the ``type``
built-in with ``None`` as an argument.
- All built-in names used in the program are now converted to "built-in name
references" in a first step. Unsupported built-ins like e.g. ``zip``, for
which Nuitka has no own code or understanding yet, remained as "module
variables", which made access to them slow, and difficult to recognize.
- Added optimization for module attributes ``__file__``, ``__doc__`` and
``__package__`` if they are read only. It's the same as ``__name__``.
- Added optimization for slices and subscripts of "compile time constant"
values. These will play a more important role, once value propagation makes
them more frequent.
Organizational
--------------
- Created a "change log" from the previous release announcements. It's as
ReStructured Text and converted to PDF for the release as well, but I chose
not to include that in Debian, because it's so easy to generate the PDF on
that yourself.
- The posting of release announcements is now prepared by a script that converts
the ReStructured Text to HTML and adds it to Wordpress as a draft posting or
updates it, until it's release time. Simple, sweet and elegant.
Cleanups
--------
- Split out the ``nuitka.nodes.Nodes`` module into many topic nodes, so that
there are now ``nuitka.nodes.BoolNodes`` or ``nuitka.nodes.LoopNodes`` to host
nodes of similar kinds, so that it is now cleaner.
- Split ``del`` statements into their own node kind, and use much simpler node
structures for them. The following blocks are absolutely the same:
.. code-block:: python
del a, b.c, d
.. code-block:: python
del a
del b.c
del d
So that's now represented in the node tree. And even more complex looking
cases, like this one, also the same:
.. code-block:: python
del a, (b.c, d)
This one gives a different parse tree, but the same bytecode. And so Nuitka
need no longer concern itself with this at all, and can remove the tuple from
the parse tree immediately. That makes them easy to handle. As you may have
noted already, it also means, there is no way to enforce that two things are
deleted or none at all.
- Turned the function and class builder statements into mere assignment
statements, where defaults and base classes are handled by wrapping
expressions.
Previously they are also kind of assignment statements too, which is not
needed. Now they were reduced to only handle the ``bases`` for classes and the
``defaults`` for functions and make optional.
- Refactored the decorator handling to the tree building stage, presenting them
as function calls on "function body expression" or class body expression".
This allowed to remove the special code for decorators from code generation
and C++ templates, making decorations easy subjects for future optimization,
as they practically are now just function calls.
.. code-block:: python
@some_classdecorator
class C:
@staticmethod
def f():
pass
It's just a different form of writing things. Nothing requires the
implementation of decorators, it's just functions calls with function bodies
before the assignment.
The following is only similar:
.. code-block:: python
class C:
def f():
pass
f = staticmethod( f )
C = some_classdecorator(C)
It's only similar, because the assignment to an intermediate value of ``C``
and ``f`` is not done, and if an exception was raised by the decoration, that
name could persist. For Nuitka, the function and class body, before having a
name, are an expression, and so can of course be passed to decorators already.
- The in-place assignments statements are now handled using temporary variable
blocks
Adding support for scoped temporary variables and references to them, it was
possible to re-formulate in-place assignments expressions as normal look-ups,
in-place operation call and then assignment statement. This allowed to remove
static templates and will yield even better generated code in the future.
- The for loop used to have has a "source" expression as child, and the iterator
over it was only taken at the code generation level, so that step was
therefore invisible to optimization. Moved it to tree building stage instead,
where optimization can work on it then.
- Tree building now generally allows statement sequences to be ``None``
everywhere, and pass statements are immediately eliminated from them
immediately. Empty statement sequences are now forbidden to exist.
- Moved the optimization for ``__name__`` to compute node of variable
references, where it doesn't need anything complex to replace with the
constant value if it's only read.
- Added new bases classes and mix-in classes dedicated to expressions, giving a
place for some defaults.
- Made the built-in code more reusable.
New Tests
---------
- Added some more diagnostic tests about complex assignment and ``del``
statements.
- Added syntax test for star import on function level, that must fail on
Python3.
- Added syntax test for duplicate argument name.
- Added syntax test for global on a function argument name.
Summary
-------
The decorator and building changes, the assignment changes, and the node
cleanups are all very important progress for the type inference work, because
they remove special casing the that previously would have been required. Lambdas
and functions now really are the same thing right after tree building. The
in-place assignments are now merely done using standard assignment code, the
built functions and classes are now assigned to names in assignment statements,
much *more* consistency there.
Yet, even more work will be needed in the same direction. There may e.g. be work
required to cover ``with`` statements as well. And assignments will become no
more complex than unpacking from a temporary variable.
For this release, there is only minimal progress on the Python3 front, despite
the syntax support, which is only miniscule progress. The remaining tasks appear
all more or less difficult work that I don't want to touch now.
There are still remaining steps, but we can foresee that a release may be done
that finally actually does type inference and becomes the effective Python
compiler this project is all about.
Nuitka Release 0.3.19
=====================
This time there are a few bug fixes, major cleanups, more Python3 support, and
even new features. A lot things in this are justifying a new release.
Bug fixes
---------
- The man pages of ``nuitka`` and ``nuitka-python`` had no special layout for
the option groups and broken whitespace for ``--recurse-to`` option. Also
``--g++-only`` was only partially bold. Released as 0.3.18.1 hot fix already.
- The command line length improvement we made to Scons for Windows was not
portable to Python2.6. Released as 0.3.18.2 hot fix already.
- Code to detect already considered packages detection was not portable to
Windows, for one case, there was still a use of ``/`` instead of using a
``joinpath`` call. Released as 0.3.18.3 already.
- A call to the range built-in with no arguments would crash the compiler, see
`Issue#29 `__. Released as 0.3.18.4 already.
- Compatibility Fix: When rich comparison operators returned false value other
``False``, for comparison chains, these would not be used, but ``False``
instead, see .
- The support for ``__import__`` didn't cover keyword arguments, these were
simply ignored. See `Issue#28 `__. Fixed, but
no warning is given yet.
New Features
------------
- A new option has been added, one can now specify ``--recurse-directory`` and
Nuitka will attempt to embed these modules even if not obviously
imported. This is not yet working perfect yet, but will receive future
improvements.
- Added support for the ``exec`` built-in of Python3, this enables us to run one
more basic test, ``GlobalStatement.py`` with Python3. The test ``ExecEval.py``
nearly works now.
New Optimization
----------------
- The no arguments ``range()`` call now optimized into the static CPython
exception it raises.
- Parts of comparison chains with constant arguments are now optimized away.
Cleanups
--------
- Simplified the ``CPythonExpressionComparison`` node, it now always has only 2
operands.
If there are more, the so called "comparison chain", it's done via ``and``
with assignments to temporary variables, which are expressed by a new node
type ``CPythonExpressionTempVariableRef``. This allowed to remove
``expression_temps`` from C++ code templates and generation, reducing the
overall complexity.
- When executing a module (``--execute`` but not ``--exe``), no longer does
Nuitka import it into itself, instead a new interpreter is launched with a
fresh environment.
- The calls to the variadic ``MAKE_TUPLE`` were replaced with calls the
``MAKE_TUPLExx`` (where ``xx`` is the number of arguments), that are generated
on a as-needed basis. This gives more readable code, because no
``EVAL_ORDERED_xx`` is needed at call site anymore.
- Many node classes have moved to new modules in ``nuitka.nodes`` and grouped by
theme. That makes them more accessible.
- The choosing of the debug python has moved from Scons to Nuitka itself. That
way it can respect the ``sys.abiflags`` and works with Python3.
- The replacing of ``.py`` in filenames was made more robust. No longer is
``str.replace`` used, but instead proper means to assure that having ``.py``
as other parts of the filenames won't be a trouble.
- Module recursion was changed into its own module, instead of being hidden in
the optimization that considers import statements.
- As always, some PyLint work, and some minor ``TODO`` were solved.
Organizational
--------------
- Added more information to the "`Developer Manual
`__", e.g. documenting the tree
changes for ``assert`` to become a conditional statement with a raise
statement, etc.
- The Debian package is as of this version verified to be installable and
functional on to Ubuntu Natty, Maverick, Oneiric, and Precise.
- Added support to specify the binary under test with a ``NUITKA`` environment,
so the test framework can run with installed version of Nuitka too.
- Made sure the test runners work under Windows as well. Required making them
more portable. And a workaround for ``os.execl`` not propagating exit codes
under Windows. See `Issue#26 `__ for more
information.
- For windows target the MinGW library is now linked statically. That means
there is no requirement for MinGW to be in the ``PATH`` or even installed to
execute the binary.
New Tests
---------
- The ``basic``, ``programs``, ``syntax``, and ``reflected`` were made
executable under Windows. Occasionally this meant to make the test runners
more portable, or to work around limitations.
- Added test to cover return values of rich comparisons in comparison chains,
and order of argument evaluation for comparison chains.
- The ``Referencing.py`` test was made portable to Python3.
- Cover no arguments ``range()`` exception as well.
- Added test to demonstrate that ``--recurse-directory`` actually works. This is
using an ``__import__`` that cannot be predicted at run time (yet).
- The created source package is now tested on pbuilder chroots to be pass
installation and the basic tests, in addition to the full tests during package
build time on these chroots. This will make sure, that Nuitka works fine on
Ubuntu Natty and doesn't break without notice.
Summary
-------
This releases contains many changes. The "temporary variable ref" and
"assignment expression" work is ground breaking. I foresee that it will lead to
even more simplifications of code generation in the future, when e.g. in-place
assignments can be reduced to assignments to temporary variables and conditional
statements.
While there were many improvements related to Windows support and fixing
portability bugs, or the Debian package, the real focus is the optimization
work, which will ultimately end with "value propagation" working.
These are the real focus. The old comparison chain handling was a big
wart. Working, but no way understood by any form of analysis in Nuitka. Now they
have a structure which makes their code generation based on semantics and allows
for future optimization to see through them.
Going down this route is an important preparatory step. And there will be more
work like this needed. Consider e.g. handling of in-place assignments. With an
"assignment expression" to a "temporary variable ref", these become the same as
user code using such a variable. There will be more of these to find.
So, that is where the focus is. The release now was mostly aiming at getting
involved fixes out. The bug fixed by comparison chain reworking, and the
``__import__`` related one, were not suitable for hot fix releases, so that is
why the 0.3.19 release had to occur now. But with plugin support, with this
comparison chain cleanup, with improved Python3 support, and so on, there was
plenty of good stuff already, also worth to get out.
Nuitka Release 0.3.18
=====================
This is to inform you about the new stable release of Nuitka. This time there
are a few bug fixes, and the important step that triggered the release: Nuitka
has entered Debian Unstable. So you if want, you will get stable Nuitka releases
from now on via ``apt-get install nuitka``.
The release cycle was too short to have much focus. It merely includes fixes,
which were available as hot fixes, and some additional optimization and node
tree cleanups, as well as source cleanups. But not much else.
Bug fixes
---------
- Conditional statements with both branches empty were not optimized away in all
cases, triggering an assertion of code generation. `Issue#16
`__. Released as 0.3.17a hot fix already.
- Nuitka was considering directories to contain packages that had no
"__init__.py" which could lead to errors when it couldn't find the package
later in the compilation process. Released as 0.3.17a hot fix already.
- When providing ``locals()`` to ``exec`` statements, this was not making the
``locals()`` writable. The logic to detect the case that default value is used
(None) and be pessimistic about it, didn't consider the actual value
``locals()``. Released as 0.3.17b hot fix already.
- Compatibility Fix: When no defaults are given, CPython uses ``None`` for
``func.func_defaults``, but Nuitka had been using ``None``.
New Optimization
----------------
- If the condition of assert statements can be predicted, these are now
optimized in a static raise or removed.
- For built-in name references, there is now dedicated code to look them up,
that doesn't check the module level at all. Currently these are used in only a
few cases though.
- Cleaner code is generated for the simple case of ``print`` statements. This is
not only faster code, it's also more readable.
Cleanups
--------
- Removed the ``CPythonStatementAssert`` node.
It's not needed, instead at tree building, assert statements are converted to
conditional statements with the asserted condition result inverted and a raise
statement with ``AssertionError`` and the assertion argument.
This allowed to remove code and complexity from the subsequent steps of
Nuitka, and enabled existing optimization to work on assert statements as
well.
- Moved built-in exception names and built-in names to a new module
``nuitka.Builtins`` instead of having in other places. This was previously a
bit spread-out and misplaced.
- Added cumulative ``tags`` to node classes for use in checks. Use it annotate
which node kinds to visit in e.g. per scope finalization steps. That avoids
kinds and class checks.
- New node for built-in name loopups, which allowed to remove tricks played with
adding module variable lookups for ``staticmethod`` when adding them for
``__new__`` or module variable lookups for ``str`` when predicting the result
of ``type( 'a' )``, which was unlikely to cause a problem, but an important
``TODO`` item still.
Organizational
--------------
- The `"Download" <../pages/download.html>`__ page is now finally updated for
releases automatically.
This closes `Issue#7 `__ completely. Up to this
release, I had to manually edit that page, but now mastered the art of
upload via XMLRCP and a Python script, so that don't loose as much time with
editing, checking it, etc.
- The Debian package is backportable to Ubuntu Natty, Maverick, Oneiric, I
expect to make a separate announcement with links to packages.
- Made sure the test runners worth with bare ``python2.6`` as well.
New Tests
---------
- Added some tests intended for type inference development.
Summary
-------
This releases contains not as much changes as others, mostly because it's the
intended base for a Debian upload.
The ``exec`` fix was detected by continued work on the branch
``feature/minimize_CPython26_tests_diff`` branch, but that work is now complete.
It is being made pretty (many git rebase iterations) with lots of Issues being
added to the bug tracker and referenced for each change. The intention is to
have a clean commits repository with the changed made.
But of course, the real excitement is the "type inference" work. It will give a
huge boost to Nuitka. With this in place, new benchmarks may make sense. I am
working on getting it off the ground, but also to make us more efficient.
So when I learn something. e.g. ``assert`` is not special, I apply it to the
``develop`` branch immediately, to keep the differences as small as possible,
and to immediately benefit from such improvements.
Nuitka Release 0.3.17
=====================
This is to inform you about the new stable release of Nuitka. This time there
are a few bug fixes, lots of very important organisational work, and yet again
improved compatibility and cleanups. Also huge is the advance in making
``--deep`` go away and making the recursion of Nuitka controllable, which means
a lot for scalability of projects that use a lot of packages that use other
packages, because now you can choose which ones to embed and which ones one.
The release cycle had a focus on improving the quality of the test scripts, the
packaging, and generally to prepare the work on "type inference" in a new
feature branch.
I have also continued to work towards CPython3.2 compatibility, and this
version, while not there, supports Python3 with a large subset of the basic
tests programs running fine (of course via ``2to3`` conversion) without
trouble. There is still work to do, exceptions don't seem to work fully yet,
parameter parsing seems to have changed, etc. but it seems that CPython3.2 is
going to work one day.
And there has been a lot of effort, to address the Debian packaging to be
cleaner and more complete, addressing issues that prevented it from entering the
Debian repository.
Bug fixes
---------
- Fixed the handling of modules and packages of the same name, but with
different casing. Problem showed under Windows only. Released as 0.3.16a
hot fix already.
- Fixed an error where the command line length of Windows was exceeded when many
modules were embedded, Christopher Tott provided a fix for it. Released as
0.3.16a hot fix already.
- Fix, avoid to introduce new variables for where built-in exception references
are sufficient. Released as 0.3.16b hot fix already.
- Fix, add the missing ``staticmethod`` decorator to ``__new__`` methods before
resolving the scopes of variables, this avoids the use of that variable before
it was assigned a scope. Released as 0.3.16b hot fix already.
New Features
------------
- Enhanced compatibility again, provide enough ``co_varnames`` in the code
objects, so that slicing them up to ``code_object.co_argcount`` will
work. They are needed by ``inspect`` module and might be used by some
decorators as well.
- New options to control the recursion:
``--recurse-none`` (do not warn about not-done recursions)
``--recurse-all`` (recurse to all otherwise warned modules)
``--recurse-to`` (confirm to recurse to those modules)
``--recurse-not-to`` (confirm to not recurse to those modules)
New Optimization
----------------
- The optimization of constant conditional expressions was not done yet. Added
this missing constant propagation case.
- Eliminate near empty statement sequences (only contain a pass statement) in
more places, giving a cleaner node structure for many constructs.
- Use the pickle "protocol 2" on CPython2 except for ``unicode`` strings where
it does not work well. It gives a more compressed and binary representation,
that is generally more efficient to un-stream as well. Also use the cPickle
protocol, the use of ``pickle`` was not really necessary anymore.
Organizational
--------------
- Added a "`Developer Manual `__"
to the release. It's incomplete, but it details some of the existing stuff,
coding rules, plans for "type inference", etc.
- Improved the ``--help`` output to use ``metavar`` where applicable. This makes
it more readable for some options.
- Instead of error message, give help output when no module or program file name
was given. This makes Nuitka help out more convenient.
- Consistently use ``#!/usr/bin/env python`` for all scripts, this was
previously only done for some of them.
- Ported the PyLint check script to Python as well, enhancing it on the way to
check the exit code, and to only output changes things, as well as making the
output of warnings for ``TODO`` items optional.
- All scripts used for testing, PyLint checking, etc. now work with Python3 as
well. Most useful on Arch Linux, where it's also already the default for
``Python``.
- The help output of Nuitka was polished a lot more. It is now more readable and
uses option groups to combine related options together.
- Make the tests run without any dependence on ``PATH`` to contain the
executables of Nuitka. This makes it easier to use.
- Add license texts to 3rd party file that were missing them, apply
``licensecheck`` results to cleanup Nuitka. Also removed own copyright
statement from inline copy of Scons, it had been added by accident only.
- Release the tests that I own as well as the Debian packaging I created under
"Apache License 2.0" which is very liberal, meaning every project will be able
to use it.
- Don't require copyright assignment for contributions anymore, instead only
"Apache License 2.0", the future Nuitka license, so that the code won't be a
problem when changing the license of all of Nuitka to that license.
- Give contributors listed in the "`User Manual
`__" an exception to the GPL terms
until Nuitka is licensed under "Apache License 2.0" as well.
- Added an ``--experimental`` option which can be used to control experimental
features, like the one currently being added on ``feature/ctypes_annotation``,
where "type inference" is currently only activated when that option is
given. For this stable release, it does nothing.
- Check the static C++ files of Nuitka with ``cppcheck`` as well. Didn't find
anything.
- Arch Linux packages have been contributed, these are linked for download, but
the stable package may lag behind a bit.
Cleanups
--------
- Changed ``not`` boolean operation to become a normal operator. Changed ``and``
and ``or`` boolean operators to a new base class, and making their interface
more similar to that of operations.
- Added cumulative ``tags`` to node classes for use in checks. Use it annotate
which node kinds to visit in e.g. per scope finalization steps. That avoids
kinds and class checks.
- Enhanced the "visitor" interface to provide more kinds of callbacks, enhanced
the way "each scope" visiting is achieved by generalizing is as "child has not
tag 'closure_taker'" and that for every "node that has tag 'closure_taker'".
- Moved ``SyntaxHighlighting`` module to ``nuitka.gui`` package where it
belongs.
- More white listing work for imports. As recursion is now the default, and
leads to warnings for non-existent modules, the CPython tests gave a lot of
good candidates for import errors that were white listed.
- Consistently use ``nuitka`` in test scripts, as there isn't a ``Nuitka.py`` on
all platforms. The later is scheduled for removal.
- Some more PyLint cleanups.
New Tests
---------
- Make sure the basic tests pass with CPython or else fail the test. This is to
prevent false positives, where a test passes, but only because it fails in
CPython early on and then does so with Nuitka too. For the syntax tests we
make sure they fail.
- The basic tests can now be run with ``PYTHON=python3.2`` and use ``2to3``
conversion in that case. Also the currently not passing tests are not run, so
the passing tests continue to do so, with this run from the release test
script ``check-release``.
- Include the syntax tests in release tests as well.
- Changed many existing tests so that they can run under CPython3 too. Of course
this is via ``2to3`` conversion.
- Don't fail if the CPython test suites are not there.
Currently they remain largely unpublished, and as such are mostly only
available to me (exception, ``feature/minimize_CPython26_tests_diff`` branch
references the CPython2.6 tests repository, but that remains work in
progress).
- For the compile itself test: Make the presence of the Scons inline copy
optional, the Debian package doesn't contain it.
- Also make it more portable, so it runs under Windows too, and allow to choose
the Python version to test. Check this test with both CPython2.6 and
CPython2.7 not only the default Python.
- Before releasing, test that the created Debian package builds fine in a
minimal Debian ``unstable`` chroot, and passes all the tests included in the
package (``basics``, ``syntax``, ``programs``, ``reflected``). Also many other
Debian packaging improvements.
Summary
-------
The "git flow" was used again in this release cycle and proved to be useful not
only for hot fix, but also for creating the branch ``feature/ctypes_annotation``
and rebasing it often while things are still flowing.
The few hot fixes didn't require a new release, but the many organizational
improvements and the new features did warrant the new release, because of
e.g. the much better test handling in this release and the improved recursion
control.
The work on Python3 support has slowed down a bit. I mostly only added some bits
for compatibility, but generally it has slowed down. I wanted to make sure it
doesn't regress by accident, so running with CPython3.2 is now part of the
normal release tests.
What's still missing is more "hg" completeness. Only the ``co_varnames`` work
for ``inspect`` was going in that direction, and this has slowed down. It was
more important to make Nuitka's recursion more accessible with the new options,
so that was done first.
And of course, the real excitement is the "type inference" work. It will give a
huge boost to Nuitka, and I am happy that it seems to go well. With this in
place, new benchmarks may make sense. I am working on getting it off the ground,
so other people can work on it too. My idea of ``ctypes`` native calls may
become true sooner than expected. To support that, I would like to add more
tools to make sure we discover changes earlier on, checking the XML
representations of tests to discover improvements and regressions more clearly.
Nuitka Release 0.3.16
=====================
This time there are many bug fixes, some important scalability work, and again
improved compatibility and cleanups.
The release cycle had a focus on fixing the bug reports I received. I have also
continued to look at CPython3 compatibility, and this is the first version to
support Python3 somewhat, at least some of the basic tests programs run (of
course via ``2to3`` conversion) without trouble. I don't know when, but it seems
that it's going to work one day.
Also there has an effort to make the Debian packaging cleaner, addressing all
kinds of small issues that prevented it from entering the Debian
repository. It's still not there, but it's making progress.
Bug fixes
---------
- Fixed a packaging problem for Linux and x64 platform, the new ``swapFiber.S``
file for the fiber management was not included. Released as 0.3.15a hot fix
already.
- Fixed an error where optimization was performed on removed unreachable code,
which lead to an error. Released as 0.3.15b hot fix already.
- Fixed an issue with ``__import__`` and recursion not happening in any case,
because when it did, it failed due to not being ported to new internal
APIs. Released as 0.3.15c hot fix already.
- Fixed ``eval()`` and ``locals()`` to be supported in generator expressions and
contractions too. Released as 0.3.15d hot fix already.
- Fixed the Windows batch files ``nuitka.bat`` and ``nuitka-python.bat`` to not
output the ``rem`` statements with the copyright header. Released as 0.3.15d
hot fix already.
- Fixed re-raise with ``raise``, but without a current exception set. Released
as 0.3.15e hot fix already.
- Fixed ``vars()`` call on the module level, needs to be treated as
``globals()``. Released as 0.3.15e hot fix already.
- Fix handling of broken new lines in source files. Read the source code in
"universal line ending mode". Released as 0.3.15f hot fix already.
- Fixed handling of constant module attribute ``__name__`` being replaced. Don't
replace local variables of the same name too. Released as 0.3.15g hot fix
already.
- Fixed assigning to ``True``, ``False`` or ``None``. There was this old
``TODO``, and some code has compatibility craft that does it. Released as
0.3.15g hot fix already.
- Fix constant dictionaries not always being recognized as shared. Released as
0.3.15g hot fix already.
- Fix generator function objects to not require a return frame to exist. In
finalize cleanup it may not.
- Fixed non-execution of cleanup codes that e.g. flush ``sys.stdout``, by adding
``Py_Finalize()``.
- Fix ``throw()`` method of generator expression objects to not check arguments
properly.
- Fix missing fallback to subscript operations for slicing with non-indexable
objects.
- Fix, in-place subscript operations could fail to apply the update, if the
intermediate object was e.g. a list and the handle just not changed by the
operation, but e.g. the length did.
- Fix, the future spec was not properly preserving the future division flag.
New Optimization
----------------
- The optimization scales now much better, because per-module optimization only
require the module to be reconsidered, but not all modules all the time. With
many modules recursed into, this makes a huge difference in compilation time.
- The creation of dictionaries from constants is now also optimized.
New Features
------------
- As a new feature functions now have the ``func_defaults`` and ``__defaults__``
attribute. It works only well for non-nested parameters and is not yet fully
integrated into the parameter parsing. This improves the compatibility
somewhat already though.
- The names ``True``, ``False`` and ``None`` are now converted to constants only
when they are read-only module variables.
- The ``PYTHONPATH`` variable is now cleared when immediately executing a
compiled binary unless ``--execute-with-pythonpath`` is given, in which case
it is preserved. This allows to make sure that a binary is in fact containing
everything required.
Organizational
--------------
- The help output of Nuitka was polished a lot more. It is now more readable and
uses option groups to combine related options together.
- The inline copy of Scons is not checked with PyLint anymore. We of course
don't care.
- Program tests are no longer executed in the program directory, so failed
module inclusions become immediately obvious.
- The basic tests can now be run with ``PYTHON=python3.2`` and use ``2to3``
conversion in that case.
Cleanups
--------
- Moved ``tags`` to a separate module, make optimization emit only documented
tags, checked against the list of allowed ones.
- The Debian package has seen lots of improvements, to make it "lintian clean",
even in pedantic mode. The homepage of Nuitka is listed, a watch file can
check for new releases, the git repository and the gitweb are referenced, etc.
- Use ``os.path.join`` in more of the test code to achieve more Windows
portability for them.
- Some more PyLint cleanups.
New Tests
---------
- There is now a ``Crasher`` test, for tests that crashed Nuitka previously.
- Added a program test where the imported module does a ``sys.exit()`` and make
sure it really doesn't continue after the ``SystemExit`` exception that
creates.
- Cover the type of ``__builtins__`` in the main program and in imported modules
in tests too. It's funny and differs between module and dict in CPython2.
- Cover a final ``print`` statement without newline in the test. Must still
receive a newline, which only happens when ``Py_Finalize()`` is called.
- Added test with functions that makes a ``raise`` without an exception set.
- Cover the calling of ``vars()`` on module level too.
- Cover the use of eval in contractions and generator expressions too.
- Cover ``func_defaults`` and ``__default__`` attributes for a function too.
- Added test function with two ``raise`` in an exception handler, so that one
becomes dead code and removed without the crash.
Summary
-------
The "git flow" was really great in this release cycle. There were many hot fix
releases being made, so that the bugs could be addressed immediately without
requiring the overhead of a full release. I believe that this makes Nuitka
clearly one of the best supported projects.
This quick turn-around also encourages people to report more bugs, which is only
good. And the structure is there to hold it. Of course, the many bug fixes meant
that there is not as much new development, but that is not the priority,
correctness is.
The work on Python3 is a bit strange. I don't need Python3 at all. I also
believe it is that evil project to remove cruft from the Python core and make
developers of all relevant Python software, add compatibility cruft to their
software instead. Yet, I can't really stop to work on it. It has that appeal of
small fixups here and there, and then something else works too.
Python3 work is like when I was first struggling with Nuitka to pass the
CPython2 unit tests for a first time. It's fun. And then it finds real actual
bugs that apply to CPython2 too. Not doing ``Py_Finalize`` (but having to), the
slice operations shortcomings, the bug of subscript in-place, and so on. There
is likely more things hidden, and the earlier Python3 is supported, the more
benefit from increased test covered.
What's missing is more "hg" completeness. I think only the ``raise`` without
exception set and the ``func_defaults`` issue were going into its direction, but
it won't be enough yet.
Nuitka Release 0.3.15
=====================
This is to inform you about the new stable release of Nuitka. This time again
many organizational improvements, some bug fixes, much improved compatibility
and cleanups.
This release cycle had a focus on packaging Nuitka for easier consumption,
i.e. automatic packaging, making automatic uploads, improvement documentation,
and generally cleaning things up, so that Nuitka becomes more compatible and
ultimately capable to run the "hg" test suite. It's not there yet, but this is a
huge jump for usability of Nuitka and its compatibility, again.
Then lots of changes that make Nuitka approach Python3 support, the generated
C++ for at least one large example is compiling with this new release. It won't
link, but there will be later releases.
And there is a lot of cleanup going on, geared towards compatibility with line
numbers in the frame object.
Bug fixes
---------
- The main module was using ``__main__`` in tracebacks, but it must be
````. Released as 0.3.14a hot fix already.
- Workaround for "execfile cannot be used as an expression". It wasn't possible
to use ``execfile`` in an expression, only as a statement.
But then there is crazy enough code in e.g. mercurial that uses it in a lambda
function, which made the issue more prominent. The fix now allows it to be an
expression, except on the class level, which wasn't seen yet.
- The inline copy of Scons was not complete enough to work for "Windows" or with
``--windows-target`` for cross compile. Fixed.
- Cached frames didn't release the "back" frame, therefore holding variables of
these longer than CPython does, which could cause ordering problems. Fixed for
increased compatibility.
- Handle "yield outside of function" syntax error in compiled source
correctly. This one was giving a Nuitka backtrace, now it gives a
``SyntaxError`` as it needs to.
- Made syntax/indentation error output absolutely identical to CPython.
- Using the frame objects ``f_lineno`` may fix endless amounts bugs related to
traceback line numbers.
New Features
------------
- Guesses the location of the MinGW compiler under Windows to default install
location, so it need not be added to ``PATH`` environment variable. Removes
the need to modify ``PATH`` environment just for Nuitka to find it.
- Added support for "lambda generators". You don't want to know what it is. Lets
just say, it was the last absurd language feature out there, plus that didn't
work. It now works perfect.
Organizational
--------------
- You can now download a Windows installer and a Debian package that works on
Debian Testing, current Ubuntu and Mint Linux.
- New release scripts give us the ability to have hot fix releases as download
packages immediately. That means the "git flow" makes even more beneficial to
the users.
- Including the generated "README.pdf" in the distribution archives, so it can
be read instead of "README.txt". The text file is fairly readable, due to the
use of ReStructured Text, but the PDF is even nicer to read, due to
e.g. syntax highlighting of the examples.
- Renamed the main binaries to ``nuitka`` and ``nuitka-python``, so that there
is no dependency on case sensitive file systems.
- For Windows there are batch files ``nuitka.bat`` and ``nuitka-python.bat`` to
make Nuitka directly executable without finding the ``Python.exe``, which the
batch files can tell from their own location.
- There are now man pages of ``nuitka`` and ``nuitka-python`` with examples for
the most common use cases. They are of course included in the Debian package.
- Don't strip the binary when executing it to analyse compiled binary with
``valgrind``. It will give better information that way, without changing the
code.
New Optimization
----------------
- Implemented ``swapcontext`` alike (``swapFiber``) for x64 to achieve 8 times
speedup for Generators. It doesn't do useless syscalls to preserve signal
masks. Now Nuitka is faster at frame switching than CPython on x64, which is
already good by design.
Cleanups
--------
- Using the frame objects to store current line of execution avoids the need to
store it away in helper code at all. It ought to also help a lot with
threading support, and makes Nuitka even more compatible, because now line
numbers will be correct even outside tracebacks, but for mere stack frame
dumps.
- Moved the ``for_return`` detection from code generation to tree building where
it belongs. Yield statements used as return statements need slightly different
code for Python2.6 difference. That solved an old ``TODO``.
- Much Python3 portability work. Sometimes even improving existing code, the
Python compiler code had picked up a few points, where the latest Nuitka
didn't work with Python3 anymore, when put to actual compile.
The test covered only syntax, but e.g. meta classes need different code in
CPython3, and that's now supported. Also helper code was made portable in more
places, but not yet fully. This will need more work.
- Cleaned up uses of debug defines, so they are now more consistent and in one
place.
- Some more PyLint cleanups.
New Tests
---------
- The tests are now executed by Python scripts and cover ``stderr`` output
too. Before we only checked ``stdout``. This unveiled a bunch of issues Nuitka
had, but went unnoticed so far, and triggered e.g. the frame line number
improvements.
- Separate syntax tests.
- The scripts to run the tests now are all in pure Python. This means, no more
MinGW shell is needed to execute the tests.
Summary
-------
The Debian package, Windows installer, etc. are now automatically updated and
uploaded. From here on, there can be such packages for the hot fix releases too.
The exception tracebacks are now correct by design, and better covered.
The generator performance work showed that the approach taken by Nuitka is in
fact fast. It was fast on ARM already, but it's nice to see that it's now also
fast on x64. Programs using generators will be affected a lot by this.
Overall, this release brings Nuitka closer to usability. Better binary names,
man pages, improved documentation, issue tracker, etc. all there now. I am in
fact now looking for a sponsor for the Debian package to upload it into Debian
directly.
.. admonition:: Update
The upload to Debian happened for 0.3.18 and was done by Yaroslav Halchenko.
What's missing is more "hg" completeness. The frame release issue helped it, but
``inspect.getargs()`` doesn't work yet, and is a topic for a future
release. Won't be easy, as ``func_defaults`` will be an invasive change too.
Nuitka Release 0.3.14
=====================
This is to inform you about the new stable release of Nuitka. This time it
contains mostly organisational improvements, some bug fixes, improved
compatibility and cleanups.
It is again the result of working towards compilation of a real program
(Mercurial). This time, I have added support for proper handling of compiled
types by the ``inspect`` module.
Bug fixes
---------
- Fix for "Missing checks in parameter parsing with star list, star dict and
positional arguments". There was whole in the checks for argument counts, now
the correct error is given. Fixed in 0.3.13a already.
- The simple slice operations with 2 values, not extended with 3 values, were
not applying the correct order for evaluation. Fixed in 0.3.13a already.
- The simple slice operations couldn't handle ``None`` as the value for lower or
upper index. Fixed in 0.3.11a already.
- The in-place simple slice operations evaluated the slice index expressions
twice, which could cause problems if they had side effects. Fixed in 0.3.11a
already.
New Features
------------
- Run time patching the ``inspect`` module so it accepts compiled functions,
compiled methods, and compiled generator objects. The ``test_inspect`` test of
CPython is nearly working unchanged with this.
- The generator functions didn't have ``CO_GENERATOR`` set in their code object,
setting it made compatible with CPython in this regard too. The inspect module
will therefore return correct value for ``inspect.isgeneratorfunction()`` too.
New Optimization
----------------
- Slice indexes that are ``None`` are now constant propagated as well.
- Slightly more efficient code generation for dual star arg functions, removing
useless checks.
Cleanups
--------
- Moved the Scons, static C++ files, and assembler files to new package
``nuitka.build`` where also now ``SconsInterface`` module lives.
- Moved the Qt dialog files to ``nuitka.gui``
- Moved the "unfreezer" code to its own static C++ file.
- Some PyLint cleanups.
New Tests
---------
- New test ``Recursion`` to cover recursive functions.
- New test ``Inspection`` to cover the patching of ``inspect`` module.
- Cover ``execfile`` on the class level as well in ``ExecEval`` test.
- Cover evaluation order of simple slices in ``OrderCheck`` too.
Organizational
--------------
- There is a new issue tracker available under http://bugs.nuitka.net
Please register and report issues you encounter with Nuitka. I have put all
the known issues there and started to use it recently. It's Roundup based like
http://bugs.python.org is, so people will find it familiar.
- The ``setup.py`` is now apparently functional. The source releases for
download are made it with, and it appears the binary distributions work
too. We may now build a windows installer. It's currently in testing, we will
make it available when finished.
Summary
-------
The new source organisation makes packaging Nuitka really easy now. From here,
we can likely provide "binary" package of Nuitka soon. A windows installer will
be nice.
The patching of ``inspect`` works wonders for compatibility for those programs
that insist on checking types, instead of doing duck typing. The function call
problem, was an issue found by the Mercurial test suite.
For the "hg.exe" to pass all of its test suite, more work may be needed, this is
the overall goal I am currently striving for. Once real world programs like
Mercurial work, we can use these as more meaningful benchmarks and resume work
on optimization.
Nuitka Release 0.3.13
=====================
This release is mostly the result of working towards compilation of a real
programs (Mercurial) and to merge and finalize the frame stack work. Now Nuitka
has a correct frame stack at all times, and supports ``func_code`` and
``gi_code`` objects, something previously thought to be impossible.
Actually now it's only the "bytecode" objects that won't be there. And not
attributes of ``func_code`` are meaningful yet, but in theory can be supported.
Due to the use of the "git flow" for Nuitka, most of the bugs listed here were
already fixed in on the stable release before this release. This time there were
5 such hot fix releases, sometimes fixing multiple bugs.
Bug fixes
---------
- In case of syntax errors in the main program, an exception stack was giving
that included Nuitka code. Changed to make the same output as CPython
does. Fixed in 0.3.12a already.
- The star import (``from x import *``) didn't work for submodules. Providing
``*`` as the import list to the respective code allowed to drop the complex
lookups we were doing before, and to simply trust CPython C/API to do it
correctly. Fixed in 0.3.12 already.
- The absolute import is *not* the default of CPython 2.7 it seems. A local
``posix`` package shadows the standard library one. Fixed in 0.3.12 already.
- In ``--deep`` mode, a module may contain a syntax error. This is e.g. true of
"PyQt" with ``port_v3`` included. These files contain Python3 syntax and fail
to be imported in Python2, but that is not to be considered an error. These
modules are now skipped with a warning. Fixed in 0.3.12b already.
- The code to import modules wasn't using the ``__import__`` built-in, which
prevented ``__import__`` overriding code to work. Changed import to use the
built-in. Fixed in 0.3.12c already.
- The code generated for the ``__import__`` built-in with constant values was
doing relative imports only. It needs to attempt relative and absolut
imports. Fixed in 0.3.12c already.
- The code of packages in "__init__.py" believed it was outside of the package,
giving problems for package local imports. Fixed in 0.3.12d already.
- It appears that "Scons", which Nuitka uses internally and transparent to you,
to execute the compilation and linking tasks, was sometimes not building the
binaries or shared libraries, due to a false caching. As a workaround, these
are now erased before doing the build. Fixed in 0.3.12d already.
- The use of ``in`` and ``not in`` in comparison chains (e.g. ``a < b < c`` is
one), wasn't supported yet. The use of these in comparison chains ``a in b in
c`` is very strange.
Only in the ``test_grammar.py`` it was ever used I believe. Anyway, it's
supported now, solving this ``TODO`` and reducing the difference. Fixed in
0.3.12e already.
- The order of evaluation for ``in`` and ``not in`` operators wasn't enforced in
a portable way. Now it is correct on "ARM" too. Fixed in 0.3.12e already.
New Optimization
----------------
- The built-ins ``GeneratorExit`` and ``StopIteration`` are optimized to their
Python C/API names where possible as well.
Cleanups
--------
- The ``__file__`` attribute of modules was the relative filename, but for
absolute filenames these become a horrible mess at least on Linux.
- Added assertion helpers for sane frame and code objects and use them.
- Make use of ``assertObject`` in more places.
- Instead of using ``os.path.sep`` all over, added a helper ``Utils.joinpath``
that hides this and using ``os.path.join``. This gives more readable code.
- Added traces to the "unfreezer" guarded by a define. Helpful in analyzing
import problems.
- Some PyLint cleanups removing dead code, unused variables, useless pass
statement, etc.
New Tests
---------
- New tests to cover ``SyntaxError`` and ``IndentationError`` from ``--deep``
imports and in main program.
- New test to cover evaluation order of ``in`` and ``not in`` comparisons.
- New test to cover package local imports made by the "__init__.py" of the
package.
Organizational
--------------
- Drop "compile_itself.sh" in favor of the new "compile_itself.py", because the
later is more portable.
- The logging output is now nicer, and for failed recursions, outputs the line
that is having the problem.
Summary
-------
The frame stack work and the ``func_code`` are big for compatibility.
The ``func_code`` was also needed for "hg" to work. For Mercurial to pass all of
its test suite, more work will be needed, esp. the ``inspect`` module needs to
be run-time patched to accept compiled functions and generators too.
Once real world programs like Mercurial work, we can use these as more
meaningful benchmarks and resume work on optimization.
Nuitka Release 0.3.12
=====================
This is to inform you about the new release of Nuitka many bug fixes, and
substantial improvements especially in the organizational area. There is a new
"`User Manual `__" (`PDF
`__), with much improved content, a
``sys.meta_path`` based import mechanism for ``--deep`` mode, git flow goodness.
This release is generally also the result of working towards compilation of a
real programs (Mercurial) and to get things work more nicely on Windows by
default. Thanks go to Liu Zhenhai for helping me with this goal.
Due to the use of the "git flow", most of the bugs listed here were already
fixed in on the stable release before this release. And there were many of
these.
Bug fixes
---------
- The order of evaluation for base classes and class dictionaries was not
enforced.
Apparently nothing in the CPython test suite did that, I only noticed during
debugging that Nuitka gave a different error than CPython did, for a class
that had an undefined base class, because both class body and base classes
were giving an error. Fixed in 0.3.11a already.
- Method objects didn't hold a reference to the used class.
The effect was only noticed when ``--python-debug`` was used, i.e. the debug
version of Python linked, because then the garbage collector makes
searches. Fixed in 0.3.11b already.
- Set ``sys.executable`` on Linux as well. On Debian it is otherwise
``/usr/bin/python`` which might be a different version of Python
entirely. Fixed in 0.3.11c already.
- Embedded modules inside a package could hide package variables of the same
name. Learned during PyCON DE about this corner case. Fixed in 0.3.11d
already.
- Packages could be duplicated internally. This had no effect on generated code
other than appearing twice in the list if frozen modules. Fixed in 0.3.11d
already.
- When embedding modules from outside current directory, the look-up failed. The
embedding only ever worked for the compile itself and programs test cases,
because they are all in the current directory then. Fixed in 0.3.11e already.
- The check for ARM target broke Windows support in the Scons file. Fixed in
0.3.11f already.
- The star import from external modules failed with an error in ``--deep``
mode. Fixed in 0.3.11g already.
- Modules with a parent package could cause a problem under some
circumstances. Fixed in 0.3.11h already.
- One call variant, with both list and dict star arguments and keyword
arguments, but no positional parameters, didn't have the required C++ helper
function implemented. Fixed in 0.3.11h already.
- The detection of the CPU core count was broken on my hexacore at least. Gave
36 instead of 6, which is a problem for large programs. Fixed in 0.3.11h
already.
- The inline copy of Scons didn't really work on Windows, which was sad, because
we added it to simplify installation on Windows precisely because of this.
- Cleaning up the build directory from old sources and object files wasn't
portable to Windows and therefore wasn't effective there.
- From imports where part of the imported were found modules and parts were not,
didn't work. Solved by the feature branch ``meta_path_import`` that was merged
for this release.
- Newer MinGW gave warnings about the default visibility not being possible to
apply to class members. Fixed by not setting this default visibility anymore
on Windows.
- The ``sys.executable`` gave warnings on Windows because of backslashes in the
path. Using a raw string to prevent such problems.
- The standard library path was hard coded. Changed to run time detection.
Cleanups
--------
- Version checks on Python runtime now use a new define ``PYTHON_VERSION`` that
makes it easier. I don't like ``PY_VERSION_HEX``, because it is so
unreadable. Makes some of the checks a lot more safe.
- The ``sys.meta_path`` based import from the ``meta_path_import`` feature
branch allowed the cleanup the way importing is done. It's a lot less code
now.
- Removed some unused code. We will aim at making Nuitka the tool to detect dead code
really.
- Moved ``nuitka.Nodes`` to ``nuitka.nodes.Nodes``, that is what the package is
intended for, the split will come later.
New Tests
---------
- New tests for import variants that previously didn't work: Mixed
imports. Imports from a package one level up. Modules hidden by a package
variable, etc.
- Added test of function call variant that had no test previously. Only found it
when compiling "hg". Amazing how nothing in my tests, CPython tests, etc. used
it.
- Added test to cover the partial success of import statements.
- Added test to cover evaluation order of class definitions.
Organizational
--------------
- Migrated the "README.txt" from org-mode to ReStructured Text, which allows for
a more readable document, and to generate a nice "`User Manual
`__" in PDF form.
- The amount of information in "README.txt" was increased, with many more
subjects are now covered, e.g. "git flow" and how to join Nuitka
development. It's also impressive to see what code blocks and syntax
highlighting can do for readability.
- The Nuitka git repository has seen multiple hot fixes.
These allowed to publish bug fixes immediately after they were made, and
avoided the need for a new release just to get these out. This really saves me
a lot of time too, because I can postpone releasing the new version until it
makes sense because of other things.
- Then there was a feature branch ``meta_path_import`` that lived until being
merged to ``develop`` to improve the import code, which is now released on
``master`` as stable. Getting that feature right took a while.
- And there is the feature branch ``minimize_CPython26_tests_diff`` which has
some success already in documenting the required changes to the "CPython26"
test suite and in reducing the amount of differences, while doing it. We have
a frame stack working there, albeit in too ugly code form.
- The release archives are now built using ``setuptools``. You can now also
download a zip file, which is probably more Windows friendly. The intention is
to work on that to make ``setup.py`` produce a Nuitka install that won't rely
on any environment variables at all. Right now ``setup.py`` won't even allow
any other options than ``sdist`` to be given.
- Ported "compile_itself.sh" to "compile_itself.py", i.e. ported it to
Python. This way, we can execute it easily on Windows too, where it currently
still fails. Replacing ``diff``, ``rm -rf``, etc. is a challenge, but it
reduces the dependency on MSYS tools on Windows.
- The compilation of standard library is disabled by default, but ``site`` or
``dist`` packages are now embedded. To include even standard library, there is
a ``--really-deep`` option that has to be given in addition to ``--deep``,
which forces this.
Summary
-------
Again, huge progress. The improved import mechanism is very beautiful. It
appears that little is missing to compile real world programs like "hg" with
Nuitka. The next release cycle will focus on that and continue to improve the
Windows support which appears to have some issues.
Nuitka Release 0.3.11
=====================
This is to inform you about the new release of Nuitka with some bug fixes and
portability work.
This release is generally cleaning up things, and makes Nuitka portable to ARM
Linux. I used to host the Nuitka homepage on that machine, but now that it's no
longer so, I can run heavy compile jobs on it. To my surprise, it found many
portability problems. So I chose to fix that first, the result being that Nuitka
now works on ARM Linux too.
Bug fixes
---------
- The order of slice expressions was not correct on x86 as well, and I found
that with new tests only. So the porting to ARM revealed a bug category, I
previously didn't consider.
- The use of ``linux2`` in the Scons file is potentially incompatible with Linux
3.0, although it seems that at least on Debian the ``sys.platform`` was
changed back to ``linux2``. Anyway, it's probably best to allow just anything
that starts with ``linux`` these days.
- The ``print`` statement worked like a ``print`` function, i.e. it first
evaluated all printed expressions, and did the output only then. That is
incompatible in case of exceptions, where partial outputs need to be done, and
so that got fixed.
New Optimization
----------------
- Function calls now each have a dedicated helper function, avoiding in some
cases unnecessary work. We will may build further on this and inline
``PyObject_Call`` differently for the special cases.
Cleanups
--------
- Moved many C++ helper declarations and inline implementations to dedicated
header files for better organisation.
- Some dependencies were removed and consolidated to make the dependency graph
sane.
- Multiple decorators were in reverse order in the node tree. The code
generation reversed it back, so no bug, yet that was a distorted tree.
Finding this came from the ARM work, because the "reversal" was in fact just
the argument evaluation order of C++ under x86/x64, but on ARM that
broke. Correcting it highlighted this issue.
- The deletion of slices, was not using ``Py_ssize`` for indexes, disallowing
some kinds of optimization, so that was harmonized.
- The function call code generation got a general overhaul. It is now more
consistent, has more helpers available, and creates more readable code.
- PyLint is again happier than ever.
New Tests
---------
- There is a new basic test ``OrderChecks`` that covers the order of expression
evaluation. These problems were otherwise very hard to detect, and in some
cases not previously covered at all.
- Executing Nuitka with Python3 (it won't produce correct Python3 C/API code) is
now part of the release tests, so non-portable code of Nuitka gets caught.
Organizational
--------------
- Support for ARM Linux. I will make a separate posting on the challenges of
this. Suffice to say now, that C++ leaves way too much things unspecified.
- The Nuitka git repository now uses "git flow". The new git policy will be
detailed in another `separate posting
`__.
- There is an unstable ``develop`` branch in which the development occurs. For
this release ca. 40 commits were done to this branch, before merging it. I am
also doing more fine grained commits now.
- Unlike previously, there is ``master`` branch for the stable release.
- There is a script "make-dependency-graph.sh" (Update: meanwhile it was renamed
to "make-dependency-graph.py") to produce a dependency graphs of Nuitka. I
detected a couple of strange things through this.
- The Python3 ``__pycache__`` directories get removed too by the cleanup script.
Numbers
-------
We only have "PyStone" now, and on a new machine, so the numbers cannot be
compared to previous releases:
python 2.6::
Pystone(1.1) time for 50000 passes = 0.48
This machine benchmarks at 104167 pystones/second
Nuitka 0.3.11 (driven by python 2.6)::
Pystone(1.1) time for 50000 passes = 0.19
This machine benchmarks at 263158 pystones/second
So this a speedup factor of 258%, last time on another machine it was 240%. Yet
it only proves that the generated and compiled are more efficient than bytecode,
but Nuitka doesn't yet do the relevant optimization. Only once it does, the
factor will be significantly higher.
Summary
-------
Overall, there is quite some progress. Nuitka is a lot cleaner now, which will
help us later only. I wanted to get this out, mostly because of the bug fixes,
and of course just in case somebody attempts to use it on ARM.
Nuitka Release 0.3.10
=====================
This new release is major milestone 2 work, enhancing practically all areas of
Nuitka. The focus was roundup and breaking new grounds with structural
optimization enhancements.
Bug fixes
---------
- Exceptions now correctly stack.
When you catch an exception, there always was the exception set, but calling a
new function, and it catching the exception, the values of ``sys.exc_info()``
didn't get reset after the function returned.
This was a small difference (of which there are nearly none left now) but one
that might effect existing code, which affects code that calls functions in
exception handling to check something about it.
So it's good this is resolved now too. Also because it is difficult to
understand, and now it's just like CPython behaves, which means that we don't
have to document anything at all about it.
- Using ``exec`` in generator functions got fixed up. I realized that this
wouldn't work while working on other things. It's obscure yes, but it ought to
work.
- Lambda generator functions can now be nested and in generator functions. There
were some problems here with the allocation of closure variables that got
resolved.
- List contractions could not be returned by lambda functions. Also a closure
issue.
- When using a mapping for globals to ``exec`` or ``eval`` that had a side
effect on lookup, it was evident that the lookup was made twice. Correcting
this also improves the performance for the normal case.
New Optimization
----------------
- Statically raised as well as predicted exceptions are propagated upwards,
leading to code and block removal where possible, while maintaining the side
effects.
This is brand new and doesn't do everything possible yet. Most notable, the
matching of raised exception to handlers is not yet performed.
- Built-in exception name references and creation of instances of them are now
optimized as well, which leads to faster exception raising/catching for these
cases.
- More kinds of calls to built-ins are handled, positional parameters are
checked and more built-ins are covered.
Notable is that now checks are performed if you didn't potentially overload
e.g. the ``len`` with your own version in the module. Locally it was always
detected already. So it's now also safe.
- All operations and comparisons are now simulated if possible and replaced with
their result.
- In the case of predictable true or false conditions, not taken branches are
removed.
- Empty branches are now removed from most constructs, leading to sometimes
cleaner code generated.
Cleanups
--------
- Removed the lambda body node and replaced it with function body. This is a
great win for the split into body and builder. Regular functions and lambda
functions now only differ in how the created body is used.
- Large cleanup of the operation/comparison code. There is now only use of a
simulator function, which exists for every operator and comparison. This one
is then used in a prediction call, shared with the built-in predictions.
- Added a ``Tracing`` module to avoid future imports of ``print_function``,
which annoyed me many times by causing syntax failures for when I quickly
added a print statement, not noting it must have the braces.
- PyLint is happier than ever.
New Tests
---------
- Enhanced ``OverflowFunctions`` test to cover even deeper nesting of overflow
functions taking closure from each level. While it's not yet working, this
makes clearer what will be needed. Even if this code is obscure, I would like
to be that correct here.
- Made ``Operators`` test to cover the `` operator as well.
- Added to ``ListContractions`` the case where a contraction is returned by a
lambda function, but still needs to leak its loop variable.
- Enhanced ``GeneratorExpressions`` test to cover lambda generators, which is
really crazy code:
.. code-block:: python
def y():
yield((yield 1),(yield 2))
- Added to ``ExecEval`` a case where the ``exec`` is inside a generator, to
cover that too.
- Activated the testing of ``sys.exc_info()`` in ``ExceptionRaising`` test. This
was previously commented out, and now I added stuff to illustrate all of the
behaviour of CPython there.
- Enhanced ``ComparisonChains`` test to demonstrate that the order of
evaluations is done right and that side effects are maintained.
- Added ``BuiltinOverload`` test to show that overloaded built-ins are actually
called and not the optimized version. So code like this has to print 2 lines:
.. code-block:: python
from __builtin__ import len as _len
def len( x ):
print x
return _len(x)
print len(range(9))
Organizational
--------------
- Changed "README.txt" to no longer say that "Scons" is a requirement. Now that
it's included (patched up to work with ``ctypes`` on Windows), we don't have
to say that anymore.
- Documented the status of optimization and added some more ideas.
- There is now an option to dump the node tree after optimization as XML. Not
currently use, but is for regression testing, to identify where new
optimization and changes have an impact. This make it more feasible to be sure
that Nuitka is only becoming better.
- Executable with Python3 again, although it won't do anything, the necessary
code changes were done.
Summary
-------
It's nice to see, that I some long standing issues were resolved, and that
structural optimization has become almost a reality.
The difficult parts of exception propagation are all in place, now it's only
details. With that we can eliminate and predict even more of the stupid code of
"pybench" at compile time, achieving more infinite speedups.
Nuitka Release 0.3.9
====================
This is about the new release of Nuitka which some bug fixes and offers a good
speed improvement.
This new release is major milestone 2 work, enhancing practically all areas of
Nuitka. The main focus was on faster function calls, faster class attributes
(not instance), faster unpacking, and more built-ins detected and more
thoroughly optimizing them.
Bug fixes
---------
- Exceptions raised inside with statements had references to the exception and
traceback leaked.
- On Windows the binaries ``sys.executable`` pointed to the binary itself
instead of the Python interpreter. Changed, because some code uses
``sys.executable`` to know how to start Python scripts.
- There is a bug (fixed in their repository) related to C++ raw strings and C++
"trigraphs" that affects Nuitka, added a workaround that makes Nuitka not emit
"trigraphs" at all.
- The check for mutable constants was erroneous for tuples, which could lead to
assuming a tuple with only mutable elements to be not mutable, which is of
course wrong.
New Optimization
----------------
This time there are so many new optimization, it makes sense to group them by
the subject.
Exceptions
~~~~~~~~~~
- The code to add a traceback is now our own, which made it possible to use
frames that do not contain line numbers and a code object capable of lookups.
- Raising exceptions or adding to tracebacks has been made way faster by reusing
a cached frame objects for the task.
- The class used for saving exceptions temporarily (e.g. used in
``try``/``finally`` code, or with statement) has been improved so it doesn't
make a copy of the exception with a C++ ``new`` call, but it simply stores the
exception properties itself and creates the exception object only on demand,
which is more efficient.
- When catching exceptions, the addition of tracebacks is now done without
exporting and re-importing the exception to Python, but directly on the
exception objects traceback, this avoids a useless round trip.
Function Calls
~~~~~~~~~~~~~~
- Uses of PyObject_Call provide ``NULL`` as the dictionary, instead of an empty
dictionary, which is slightly faster for function calls.
- There are now dedicated variants for complex function calls with ``*`` and
``**`` arguments in all forms. These can take advantage of easier cases. For
example, a merge with star arguments is only needed if there actually were any
of these.
- The check for non-string values in the ``**`` arguments can now be completely
short-cut for the case of a dictionary that has never had a string
added. There is now code that detects this case and skips the check,
eliminating it as a performance concern.
Parameter Parsing
~~~~~~~~~~~~~~~~~
- Reversed the order in which parameters are checked.
Now the keyword dictionary is iterated first and only then the positional
arguments after that is done. This iteration is not only much faster (avoiding
repeated lookups for each possible parameter), it also can be more correct, in
case the keyword argument is derived from a dictionary and its keys mutate it
when being compared.
- Comparing parameter names is now done with a fast path, in which the pointer
values are compare first. This can avoid a call to the comparison at all,
which has become very likely due to the interning of parameter name strings,
see below.
- Added a dedicated call to check for parameter equality with rich equality
comparison, which doesn't raise an exception.
- Unpacking of tuples is now using dedicated variants of the normal unpacking
code instead of rolling out everything themselves.
Attribute Access
~~~~~~~~~~~~~~~~
- The class type (in executables, not yet for extension modules) is changed to a
faster variant of our own making that doesn't consider the restricted mode a
possibility. This avoids very expensive calls, and makes accessing class
attributes in compiled code and in non-compiled code faster.
- Access to attributes (but not of instances) got inlined and therefore much
faster. Due to other optimization, a specific step to intern the string used
for attribute access is not necessary with Nuitka at all anymore. This made
access to attributes about 50% faster which is big of course.
Constants
~~~~~~~~~
- The bug for mutable tuples also caused non-mutable tuples to be considered as
mutable, which lead to less efficient code.
- The constant creation with the g++ bug worked around, can now use raw strings
to create string constants, without resorting to un-pickling them as a work
around. This allows us to use ``PyString_FromStringAndSize`` to create strings
again, which is obviously faster, and had not been done, because of the
confusion caused by the g++ bug.
- For string constants that are usable as attributes (i.e. match the identifier
regular expression), these are now interned, directly after creation. With
this, the check for identical value of pointers for parameters has a bigger
chance to succeed, and this saves some memory too.
- For empty containers (set, dict, list, tuple) the constants created are now
are not unstreamed, but created with the dedicated API calls, saving a bit of
code and being less ugly.
- For mutable empty constant access (set, dict, list) the values are no longer
made by copying the constant, but instead with the API functions to create new
ones. This makes code like ``a = []`` a tiny bit faster.
- For slice indices the code generation now takes advantage of creating a C++
``Py_ssize_t`` from constant value if possible. Before it was converting the
integer constant at run time, which was of course wasteful even if not (very)
slow.
Iteration
~~~~~~~~~
- The creation of iterators got our own code. This avoids a function call and is
otherwise only a small gain for anything but sequence iterators. These may be
much faster to create now, as it avoids another call and repeated checks.
- The next on iterator got our own code too, which has simpler code flow,
because it avoids the double check in case of NULL returned.
- The unpack check got simlar code to the next iterator, it also has simpler
code flow now and avoids double checks.
Built-ins
~~~~~~~~~
- Added support for the ``list``, ``tuple``, ``dict``, ``str``, ``float`` and
``bool`` built-ins along with optimizing their use with constant parameter.
- Added support for the ``int`` and ``long`` built-ins, based on a new "call
spec" object, that detects parameter errors at compile time and raises
appropriate exceptions as required, plus it deals with keyword arguments just
as well.
So, to Nuitka it doesn't matter now it you write ``int( value ) ``or ``int( x
= value )`` anymore. The ``base`` parameter of these built-ins is also
supported.
The use of this call spec mechanism will the expanded, currently it is not
applied to the built-ins that take only one parameter. This is a work in
progress as is the whole built-ins business as not all the built-ins are
covered yet.
Cleanups
~~~~~~~~
- In 0.3.8 per module global classes were introduced, but the ``IMPORT_MODULE``
kept using the old universal class, this got resolved and the old class is now
fully gone.
- Using ``assertObject`` in more cases, and in more places at all, catches
errors earlier on.
- Moved the addition to tracebacks into the ``_PythonException`` class, where it
works directly on the contained traceback. This is cleaner as it no longer
requires to export exceptions to Python, just to add a traceback entry.
- Some ``PyLint`` cleanups were done, reducing the number of reports a bit, but
there is still a lot to do.
- Added a ``DefaultValueIdentifier`` class that encapsulates the access to
default values in the parameter parsing more cleanly.
- The module ``CodeTemplatesListContractions`` was renamed to
``CodeTemplatesContractions`` to reflect the fact that it deals with all kinds
of contractions (also set and dict contractions), not just list contractions.
- Moved the with related template to its own module ``CodeTemplatesWith``, so
its easier to find.
- The options handling for g++ based compilers was cleaned up, so that g++ 4.6
and MinGW are better supported now.
- Documented more aspects of the Scons build file.
- Some more generated code white space fixes.
- Moved some helpers to dedicated files. There is now ``calling.hpp`` for
function calls, an ``importing.cpp`` for import related stuff.
- Moved the manifest generation to the scons file, which now produces ready to
use executables.
New Tests
---------
- Added a improved version of "pybench" that can cope with the "0 ms" execution
time that Nuitka has for some if its sub-tests.
- Reference counting test for with statement was added.
- Micro benchmarks to demonstrate try finally performance when an exception
travels through it.
- Micro benchmark for with statement that eats up exceptions raised inside the
block.
- Micro benchmarks for the read and write access to class attributes.
- Enhanced ``Printing`` test to cover the trigraphs constant bug case. Output is
required to make the error detectable.
- Enhanced ``Constants`` test to cover repeated mutation of mutable tuple
constants, this covers the bug mentioned.
Organizational
--------------
- Added a credits section to the "README.txt" where I give credit to the people
who contributed to Nuitka, and the projects it is using. I will make it a
separate posting to cite these.
- Documented the requirements on the compiler more clearly, document the fact
that we require scons and which version of Python (2.6 or 2.7).
- The is now a codespeed implementation up and running with historical data for
up to Nuitka 0.3.8 runs of "PyStone" and with pybench. It will be updated for
0.3.9 once I have the infrastructure in place to do that automatically.
- The cleanup script now also removes .so files.
- The handling of options for g++ got improved, so it's the same for g++ and
MinGW compilers, plus adequate errors messages are given, if the compiler
version is too low.
- There is now a ``--unstriped`` option that just keeps the debug information in
the file, but doesn't keep the assertions. This will be helpful when looking
at generated assembler code from Nuitka to not have the distortions that
``--debug`` causes (reduced optimization level, assertions, etc.) and instead
a clear view.
Nuitka Release 0.3.8
====================
This is to inform you about the new release of Nuitka with some real news and a
slight performance increase. The significant news is added "Windows
Support". You can now hope to run Nuitka on Windows too and have it produce
working executables against either the standard Python distribution or a MinGW
compiled Python.
There are still some small things to iron out, and clearly documentation needs
to be created, and esp. the DLL hell problem of ``msvcr90.dll``
vs. ``msvcrt.dll``, is not yet fully resolved, but appears to be not as harmful,
at least not on native Windows.
I am thanking Khalid Abu Bakr for making this possible. I was surprised to see
this happen. I clearly didn't make it easy. He found a good way around
``ucontext``, identifier clashes, and a very tricky symbol problems where the
CPython library under Windows exports less than under Linux. Thanks a whole lot.
Currently the Windows support is considered experimental and works with MinGW
4.5 or higher only.
Otherwise there have been the usual round of performance improvements and more
cleanups. This release is otherwise milestone 2 work only, which will have to
continue for some time more.
Bug fixes
---------
- Lambda generators were not fully compatible, their simple form could yield an
extra value. The behavior for Python 2.6 and 2.7 is also different and Nuitka
now mimics both correctly, depending on the used Python version
- The given parameter count cited in the error message in case of too many
parameters, didn't include the given keyword parameters in the error message.
- There was an ``assert False`` right after warning about not found modules in
the ``--deep`` mode, which was of course unnecessary.
New Optimization
----------------
- When unpacking variables in assignments, the temporary variables are now held
in a new temporary class that is designed for the task specifically.
This avoids the taking of a reference just because the ``PyObjectTemporary``
destructor insisted on releasing one. The new class ``PyObjectTempHolder``
hands the existing reference over and releases only in case of exceptions.
- When unpacking variable in for loops, the value from the iterator may be
directly assigned, if it's to a variable.
In general this would be possible for every assignment target that cannot
raise, but the infrastructure cannot tell yet, which these would be. This will
improve with more milestone 3 work.
- Branches with only ``pass`` inside are removed, ``pass`` statements are
removed before the code generation stage. This makes it easier to achieve and
decide empty branches.
- There is now a global variable class per module. It appears that it is indeed
faster to roll out a class per module accessing the ``module *`` rather than
having one class and use a ``module **``, which is quite disappointing from
the C++ compiler.
- Also ``MAKE_LIST`` and ``MAKE_TUPLE`` have gained special cases for the 0
arguments case. Even when the size of the variadic template parameters should
be known to the compiler, it seems, it wasn't eliminating the branch, so this
was a speedup measured with valgrind.
- Empty tried branches are now replaced when possible with ``try``/``except``
statements, ``try``/``finally`` is simplified in this case. This gives a
cleaner tree structure and less verbose C++ code which the compiler threw
away, but was strange to have in the first place.
- In conditions the ``or`` and ``and`` were evaluated with Python objects
instead of with C++ bool, which was unnecessary overhead.
- List contractions got more clever in how they assign from the iterator value.
It now uses a ``PyObjectTemporary`` if it's assigned to multiple values, a
``PyObjectTempHolder`` if it's only assigned once, to something that could
raise, or a ``PyObject *`` if an exception cannot be raised. This avoids
temporary references completely for the common case.
Cleanups
--------
- The ``if``, ``for``, and ``while`` statements had always empty ``else`` nodes
which were then also in the generated C++ code as empty branches. No harm to
performance, but this got cleaned up.
- Some more generated code white space fixes.
New Tests
---------
- The CPython 2.7 test suite now also has the ``doctests`` extracted to static
tests, which improves test coverage for Nuitka again.
This was previously only done for CPython 2.6 test suite, but the test suites
are different enough to make this useful, e.g. to discover newly changed
behavior like with the lambda generators.
- Added Shed Skin 0.7.1 examples as benchmarks, so we can start to compare
Nuitka performance in these tests. These will be the focus of numbers for the
0.4.x release series.
- Added a micro benchmark to check unpacking behavior. Some of these are needed
to prove that a change is an actual improvement, when its effect can go under
in noise of inline vs. no-inline behavior of the C++ compiler.
- Added "pybench" benchmark which reveals that Nuitka is for some things much
faster, but there are still fields to work on. This version needed changes to
stand the speed of Nuitka. These will be subject of a later posting.
Organizational
--------------
- There is now a "tests/benchmarks/micro" directory to contain tiny benchmarks
that just look at a single aspect, but have no other meaning, e.g. the
"PyStone" extracts fall into this category.
- There is now a ``--windows-target`` option that attempts a cross-platform
build on Linux to Windows executable. This is using "MingGW-cross-env" cross
compilation tool chain. It's not yet working fully correctly due to the DLL
hell problem with the C runtime. I hope to get this right in subsequent
releases.
- The ``--execute`` option uses wine to execute the binary if it's a
cross-compile for windows.
- Native windows build is recognized and handled with MinGW 4.5, the VC++ is not
supported yet due to missing C++0x support.
- The basic test suite ran with Windows so far only and some adaptations were
necessary. Windows new lines are now ignored in difference check, and
addresses under Windows are upper case, small things.
Numbers
-------
python 2.6::
Pystone(1.1) time for 50000 passes = 0.65
This machine benchmarks at 76923.1 pystones/second
Nuitka 0.3.8 (driven by python 2.6)::
Pystone(1.1) time for 50000 passes = 0.27
This machine benchmarks at 185185 pystones/second
This is a 140% speed increase of 0.3.8 compared to CPython, up from 132%
compared to the previous release.
Nuitka Release 0.3.7
====================
This is about the new release with focus on performance and cleanups. It
indicates significant progress with the milestone this release series really is
about as it adds a ``compiled_method`` type.
So far functions, generator function, generator expressions were compiled
objects, but in the context of classes, functions were wrapped in CPython
``instancemethod`` objects. The new ``compiled_method`` is specifically designed
for wrapping ``compiled_function`` and therefore more efficient at it.
Bug fixes
---------
- When using ``Python`` or ``Nuitka.py`` to execute some script, the exit code
in case of "file not found" was not the same as CPython. It should be 2,
not 1.
- The exit code of the created programs (``--deep`` mode) in case of an uncaught
exception was 0, now it an error exit with value 1, like CPython does it.
- Exception tracebacks created inside ``with`` statements could contain
duplicate lines, this was corrected.
New Optimization
----------------
- Global variable assignments now also use ``assign0`` where no reference
exists.
The assignment code for module variables is actually faster if it needs not
drop the reference, but clearly the code shouldn't bother to take it on the
outside just for that. This variant existed, but wasn't used as much so far.
- The instance method objects are now Nuitka's own compiled type too. This
should make things slightly faster by itself.
- Our new compiled method objects support dedicated method parsing code, where
``self`` is passed directly, allowing to make calls taking a fast path in
parameter parsing.
This avoids allocating/freeing a ``tuple`` object per method call, while
reduced 3% ticks in "PyStone" benchmark, so that's significant.
- Solved a ``TODO`` of ``BUILTIN_RANGE`` to change it to pre-allocating the list
in the final size as we normally do everywhere else. This was a tick reduction
of 0.4% in "PyStone" benchmark, but the measurement method normalizes on loop
speed, so it's not visible in the numbers output.
- Parameter variables cannot possibly be uninitialized at creation and most
often they are never subject to a ``del`` statement. Adding dedicated C++
variable classes gave a big speedup, around 3% of "PyStone" benchmark ticks.
- Some abstract object operations were re-implemented, which allows to avoid
function calls e.g. in the ``ITERATOR_NEXT`` case, this gave a few percent on
"PyStone" as well.
Cleanups
--------
- New package ``nuitka.codegen`` to contain all code generation related stuff,
moved ``nuitka.templates`` to ``nuitka.codegen.templates`` as part of that.
- Inside the ``nuitka.codegen`` package the ``MainControl`` module now longer
reaches into ``Generator`` for simple things, but goes through
``CodeGeneration`` for everything now.
- The ``Generator`` module uses almost no tree nodes anymore, but instead gets
information passed in function calls. This allows for a cleanup of the
interface towards ``CodeGeneration``. Gives a cleaner view on the C++ code
generation, and generally furthers the goal of other than C++ language
backends.
- More "PyLint" work, many of the reported warnings have been addressed, but
it's not yet happy.
- Defaults for ``yield`` and ``return`` are ``None`` and these values are now
already added (as constants) during tree building so that no such special
cases need to be dealt with in ``CodeGeneration`` and future analysis steps.
- Parameter parsing code has been unified even further, now the whole entry
point is generated by one of the function in the new
``nuitka.codegen.ParameterParsing`` module.
- Split variable, exception, built-in helper classes into separate header files.
New Tests
---------
- The exit codes of CPython execution and Nuitka compiled programs are now
compared as well.
- Errors messages of methods are now covered by the ``ParameterErrors`` test as
well.
Organizational
--------------
- A new script "benchmark.sh" (now called "run-valgrind.py") script now starts
"kcachegrind" to display the valgrind result directly.
One can now use it to execute a test and inspect valgrind information right
away, then improve it. Very useful to discover methods for improvements, test
them, then refine some more.
- The "check-release.sh" script needs to unset ``NUITKA_EXTRA_OPTIONS`` or else
the reflection test will trip over the changed output paths.
Numbers
-------
python 2.6::
Pystone(1.1) time for 50000 passes = 0.65
This machine benchmarks at 76923.1 pystones/second
Nuitka 0.3.7 (driven by python 2.6)::
Pystone(1.1) time for 50000 passes = 0.28
This machine benchmarks at 178571 pystones/second
This is a 132% speed of 0.3.7 compared to CPython, up from 109% compare to the
previous release. This is a another small increase, that can be fully attributed
to milestone 2 measures, i.e. not analysis, but purely more efficient C++ code
generation and the new compiled method type.
One can now safely assume that it is at least twice as fast, but I will try and
get the PyPy or Shedskin test suite to run as benchmarks to prove it.
No milestone 3 work in this release. I believe it's best to finish with
milestone 2 first, because these are quite universal gains that we should have
covered.
Nuitka Release 0.3.6
====================
The major point this for this release is cleanup work, and generally bug fixes,
esp. in the field of importing. This release cleans up many small open ends of
Nuitka, closing quite a bunch of consistency ``TODO`` items, and then aims at
cleaner structures internally, so optimization analysis shall become "easy". It
is a correctness and framework release, not a performance improvement at all.
Bug fixes
---------
- Imports were not respecting the ``level`` yet. Code like this was not working,
now it is:
.. code-block:: python
from .. import something
- Absolute and relative imports were e.g. both tried all the time, now if you
specify absolute or relative imports, it will be attempted in the same way
than CPython does. This can make a difference with compatibility.
- Functions with a "locals dict" (using ``locals`` built-in or ``exec``
statement) were not 100% compatible in the way the locals dictionary was
updated, this got fixed. It seems that directly updating a dict is not what
CPython does at all, instead it only pushes things to the dictionary, when it
believes it has to. Nuitka now does the same thing, making it faster and more
compatible at the same time with these kind of corner cases.
- Nested packages didn't work, they do now. Nuitka itself is now successfully
using nested packages (e.g. ``nuitka.transform.optimizations``)
New Features
------------
- The ``--lto`` option becomes usable. It's not measurably faster immediately,
and it requires g++ 4.6 to be available, but then it at least creates smaller
binaries and may provide more optimization in the future.
New Optimization
----------------
- Exceptions raised by pre-computed built-ins, unpacking, etc. are now
transformed to raising the exception statically.
Cleanups
--------
- There is now a ``getVariableForClosure`` that a variable provider can
use. Before that it guessed from ``getVariableForReference`` or
``getVariableForAssignment`` what might be the intention. This makes some
corner cases easier.
- Classes, functions and lambdas now also have separate builder and body nodes,
which enabled to make getSameScopeNodes() really simple. Either something has
children which are all in a new scope or it has them in the same scope.
- Twisted workarounds like ``TransitiveProvider`` are no longer needed, because
class builder and class body were separated.
- New packages ``nuitka.transform.optimizations`` and
``nuitka.transform.finalizations``, where the first was
``nuitka.optimizations`` before. There is also code in ``nuitka.transform``
that was previously in a dedicated module. This allowed to move a lot of
displaced code.
- ``TreeBuilding`` now has fast paths for all 3 forms, things that need a
"provider", "node", and "source_ref"; things that need "node" and
"source_ref"; things that need nothing at all, e.g. pass.
- Variables now avoid building duplicated instances, but instead share
one. Better for analysis of them.
New Tests
---------
- The Python 2.7 test suite is no longer run with Python 2.6 as it will just
crash with the same exception all the time, there is no ``importlib`` in 2.6,
but every test is using that through test_support.
- Nested packages are now covered with tests too.
- Imports of upper level packages are covered now too.
Organizational
--------------
- Updated the "README.txt" with the current plan on optimization.
Numbers
-------
python 2.6::
Pystone(1.1) time for 50000 passes = 0.65
This machine benchmarks at 76923.1 pystones/second
Nuitka 0.3.6 (driven by python 2.6)::
Pystone(1.1) time for 50000 passes = 0.31
This machine benchmarks at 161290 pystones/second
This is 109% for 0.3.6, but no change from the previous release. No surprise,
because no new effective new optimization means have been implemented. Stay
tuned for future release for actual progress.
Nuitka Release 0.3.5
====================
This new release of Nuitka is an overall improvement on many fronts, there is no
real focus this time, likely due to the long time it was in the making.
The major points are more optimization work, largely enhanced import handling
and another improvement on the performance side. But there are also many bug
fixes, more test coverage, usability and compatibility.
Something esp. noteworthy to me and valued is that many important changes were
performed or at least triggered by Nicolas Dumazet, who contributed a lot of
high quality commits as you can see from the gitweb history. He appears to try
and compile Mercurial and Nuitka, and this resulted in important contributions.
Bug fixes
---------
- Nicolas found a reference counting bug with nested parameter calls. Where a
function had parameters of the form ``a, (b,c)`` it could crash. This got
fixed and covered with a reference count test.
- Another reference count problem when accessing the locals dictionary was
corrected.
- Values ``0.0`` and ``-0.0`` were treated as the same. They are not though,
they have a different sign that should not get lost.
- Nested contractions didn't work correctly, when the contraction was to iterate
over another contraction which needs a closure. The problem was addressing by
splitting the building of a contraction from the body of the contraction, so
that these are now 2 nodes, making it easy for the closure handling to get
things right.
- Global statements in function with local ``exec()`` would still use the value
from the locals dictionary. Nuitka is now compatible to CPython with this too.
- Nicolas fixed problems with modules of the same name inside different
packages. We now use the full name including parent package names for code
generation and look-ups.
- The ``__module__`` attribute of classes was only set after the class was
created. Now it is already available in the class body.
- The ``__doc__`` attribute of classes was not set at all. Now it is.
- The relative import inside nested packages now works correctly. With Nicolas
moving all of Nuitka to a package, the compile itself exposed many weaknesses.
- A local re-raise of an exception didn't have the original line attached but
the re-raise statement line.
New Features
------------
- Modules and packages have been unified. Packages can now also have code in
"__init__.py" and then it will be executed when the package is imported.
- Nicolas added the ability to create deep output directory structures without
having to create them beforehand. This makes ``--output-dir=some/deep/path``
usable.
- Parallel build by Scons was added as an option and enabled by default, which
enhances scalability for ``--deep`` compilations a lot.
- Nicolas enhanced the CPU count detection used for the parallel build. Turned
out that ``multithreading.cpu_count()`` doesn't give us the number of
available cores, so he contributed code to determine that.
- Support for upcoming g++ 4.6 has been added. The use of the new option
``--lto`` has been been prepared, but right now it appears that the C++
compiler will need more fixes, before we can this feature with Nuitka.
- The ``--display-tree`` feature got an overhaul and now displays the node tree
along with the source code. It puts the cursor on the line of the node you
selected. Unfortunately I cannot get it to work two-way yet. I will ask for
help with this in a separate posting as we can really use a "python-qt" expert
it seems.
- Added meaningful error messages in the "file not found" case. Previously I
just didn't care, but we sort of approach end user usability with this.
New Optimization
----------------
- Added optimization for the built-in ``range()`` which otherwise requires a
module and ``builtin`` module lookup, then parameter parsing. Now this is much
faster with Nuitka and small ranges (less than 256 values) are converted to
constants directly, avoiding run time overhead entirely.
- Code for re-raise statements now use a simple re-throw of the exception where
possible, and only do the hard work where the re-throw is not inside an
exception handler.
- Constant folding of operations and comparisons is now performed if the
operands are constants.
- Values of some built-ins are pre-computed if the operands are constants.
- The value of module attribute ``__name__`` is replaced by a constant unless it
is assigned to. This is the first sign of upcoming constant propagation, even
if only a weak one.
- Conditional statement and/or their branches are eliminated where constant
conditions allow it.
Cleanups
--------
- Nicolas moved the Nuitka source code to its own ``nuitka`` package. That is
going to make packaging it a lot easier and allows cleaner code.
- Nicolas introduced a fast path in the tree building which often delegates (or
should do that) to a function. This reduced a lot of the dispatching code and
highlights more clearly where such is missing right now.
- Together we worked on the line length issues of Nuitka. We agreed on a style
and very long lines will vanish from Nuitka with time. Thanks for pushing me
there.
- Nicolas also did provide many style fixes and general improvements, e.g. using
``PyObjectTemporary`` in more places in the C++ code, or not using
``str.find`` where ``x in y`` is a better choice.
- The node structure got cleaned up towards the direction that assigments always
have an assignment as a child. A function definition, or a class definition,
are effectively assignments, and in order to not have to treat this as special
cases everywhere, they need to have assignment targets as child nodes.
Without such changes, optimization will have to take too many things into
account. This is not yet completed.
- Nicolas merged some node tree building functions that previously handled
deletion and assigning differently, giving us better code reuse.
- The constants code generation was moved to a ``__constants.cpp`` where it
doesn't make __main__.cpp so much harder to read anymore.
- The module declarations have been moved to their own header files.
- Nicolas cleaned up the scripts used to test Nuitka big time, removing
repetitive code and improving the logic. Very much appreciated.
- Nicolas also documented a things in the Nuitka source code or got me to
document things that looked strange, but have reasons behind it.
- Nicolas solved the ``TODO`` related to built-in module accesses. These will
now be way faster than before.
- Nicolas also solved the ``TODO`` related to the performance of "locals dict"
variable accesses.
- Generator.py no longer contains classes. The Contexts objects are supposed to
contain the state, and as such the generator objects never made much sense.
- Also with the help of Scons community, I figured out how to avoid having
object files inside the ``src`` directory of Nuitka. That should also help
packaging, now all build products go to the .build directory as they should.
- The vertical white space of the generated C++ got a few cleanups,
trailing/leading new line is more consistent now, and there were some
assertions added that it doesn't happen.
New Tests
---------
- The CPython 2.6 tests are now also run by CPython 2.7 and the other way around
and need to report the same test failure reports, which found a couple of
issues.
- Now the test suite is run with and without ``--debug`` mode.
- Basic tests got extended to cover more topics and catch more issues.
- Program tests got extended to cover code in packages.
- Added more exec scope tests. Currently inlining of exec statements is disabled
though, because it requires entirely different rules to be done right, it has
been pushed back to the next release.
Organizational
--------------
- The ``g++-nuitka`` script is no more. With the help of the Scons community,
this is now performed inside the scons and only once instead of each time for
every C++ file.
- When using ``--debug``, the generated C++ is compiled with ``-Wall`` and
``-Werror`` so that some form of bugs in the generated C++ code will be
detected immediately. This found a few issues already.
- There is a new git merge policy in place. Basically it says, that if you
submit me a pull request, that I will deal with it before publishing anything
new, so you can rely on the current git to provide you a good base to work
on. I am doing more frequent pre-releases already and I would like to merge
from your git.
- The "README.txt" was updated to reflect current optimization status and
plans. There is still a lot to do before constant propagation can work, but
this explains things a bit better now. I hope to expand this more and more
with time.
- There is now a "misc/clean-up.sh" script that prints the commands to erase all
the temporary files sticking around in the source tree. That is for you if you
like me, have other directories inside, ignored, that you don't want to
delete.
- Then there is now a script that prints all source filenames, so you can more
easily open them all in your editor.
- And very important, there is now a "check-release.sh" script that performs all
the tests I think should be done before making a release.
- Pylint got more happy with the current Nuitka source. In some places, I added
comments where rules should be granted exceptions.
Numbers
-------
python 2.6::
Pystone(1.1) time for 50000 passes = 0.65
This machine benchmarks at 76923.1 pystones/second
Nuitka 0.3.5 (driven by python 2.6)::
Pystone(1.1) time for 50000 passes = 0.31
This machine benchmarks at 161290 pystones/second
This is 109% for 0.3.5, up from 91% before.
Overall this release is primarily an improvement in the domain of compatibility
and contains important bug and feature fixes to the users. The optimization
framework only makes a first showing of with the framework to organize
them. There is still work to do to migrate optimization previously present
It will take more time before we will see effect from these. I believe that even
more cleanups of ``TreeBuilding``, ``Nodes`` and ``CodeGeneration`` will be
required, before everything is in place for the big jump in performance
numbers. But still, passing 100% feels good. Time to rejoice.
Nuitka Release 0.3.4
====================
This new release of Nuitka has a focus on re-organizing the Nuitka generated
source code and a modest improvement on the performance side.
For a long time now, Nuitka has generated a single C++ file and asked the C++
compiler to translate it to an executable or shared library for CPython to
load. This was done even when embedding many modules into one (the "deep"
compilation mode, option ``--deep``).
This was simple to do and in theory ought to allow the compiler to do the most
optimization. But for large programs, the resulting source code could have
exponential compile time behavior in the C++ compiler. At least for the GNU g++
this was the case, others probably as well. This is of course at the end a
scalability issue of Nuitka, which now has been addressed.
So the major advancement of this release is to make the ``--deep`` option
useful. But also there have been a performance improvements, which end up giving
us another boost for the "PyStone" benchmark.
Bug fixes
---------
- Imports of modules local to packages now work correctly, closing the small
compatibility gap that was there.
- Modules with a "-" in their name are allowed in CPython through dynamic
imports. This lead to wrong C++ code created. (Thanks to Li Xuan Ji for
reporting and submitting a patch to fix it.)
- There were warnings about wrong format used for ``Ssize_t`` type of
CPython. (Again, thanks to Li Xuan Ji for reporting and submitting the patch
to fix it.)
- When a wrong exception type is raised, the traceback should still be the one
of the original one.
- Set and dict contractions (Python 2.7 features) declared local variables for
global variables used. This went unnoticed, because list contractions don't
generate code for local variables at all, as they cannot have such.
- Using the ``type()`` built-in to create a new class could attribute it to the
wrong module, this is now corrected.
New Features
------------
- Uses Scons to execute the actual C++ build, giving some immediate
improvements.
- Now caches build results and Scons will only rebuild as needed.
- The direct use of ``__import__()`` with a constant module name as parameter is
also followed in "deep" mode. With time, non-constants may still become
predictable, right now it must be a real CPython constant string.
New Optimization
----------------
- Added optimization for the built-ins ``ord()`` and ``chr()``, these require a
module and built-in module lookup, then parameter parsing. Now these are
really quick with Nuitka.
- Added optimization for the ``type()`` built-in with one parameter. As above,
using from builtin module can be very slow. Now it is instantaneous.
- Added optimization for the ``type()`` built-in with three parameters. It's
rarely used, but providing our own variant, allowed to fix the bug mentioned
above.
Cleanups
--------
- Using scons is a big cleanup for the way how C++ compiler related options are
applied. It also makes it easier to re-build without Nuitka, e.g. if you were
using Nuitka in your packages, you can easily build in the same way than
Nuitka does.
- Static helpers source code has been moved to ".hpp" and ".cpp" files, instead
of being in ".py" files. This makes C++ compiler messages more readable and
allows us to use C++ mode in Emacs etc., making it easier to write things.
- Generated code for each module ends up in a separate file per module or
package.
- Constants etc. go to their own file (although not named sensible yet, likely
going to change too)
- Module variables are now created by the ``CPythonModule`` node only and are
unique, this is to make optimization of these feasible. This is a pre-step to
module variable optimization.
New Tests
---------
- Added "ExtremeClosure" from my Python quiz, it was not covered by existing
tests.
- Added test case for program that imports a module with a dash in its name.
- Added test case for main program that starts with a dash.
- Extended the built-in tests to cover ``type()`` as well.
Organizational
--------------
- There is now a new environment variable ``NUITKA_SCONS`` which should point to
the directory with the ``SingleExe.scons`` file for Nuitka. The scons file
could be named better, because it is actually one and the same who builds
extension modules and executables.
- There is now a new environment variable ``NUITKA_CPP`` which should point to
the directory with the C++ helper code of Nuitka.
- The script "create-environment.sh" can now be sourced (if you are in the top
level directory of Nuitka) or be used with eval. In either case it also
reports what it does.
.. admonition:: Update
The script has become obsolete now, as the environment variables are no
longer necessary.
- To cleanup the many "Program.build" directories, there is now a "clean-up.sh"
script for your use. Can be handy, but if you use git, you may prefer its
clean command.
.. admonition:: Update
The script has become obsolete now, as Nuitka test executions now by
default delete the build results.
Numbers
-------
python 2.6::
Pystone(1.1) time for 50000 passes = 0.65
This machine benchmarks at 76923.1 pystones/second
Nuitka 0.3.4::
Pystone(1.1) time for 50000 passes = 0.34
This machine benchmarks at 147059 pystones/second
This is 91% for 0.3.4, up from 80% before.
Nuitka Release 0.3.3
====================
This release of Nuitka continues the focus on performance. It also cleans up a
few open topics. One is "doctests", these are now extracted from the CPython 2.6
test suite more completely. The other is that the CPython 2.7 test suite is now
passed completely. There is some more work ahead though, to extract all of the
"doctests" and to do that for both versions of the tests.
This means an even higher level of compatibility has been achieved, then there
is performance improvements, and ever cleaner structure.
Bug fixes
---------
Generators
~~~~~~~~~~
- Generator functions tracked references to the common and the instance context
independently, now the common context is not released before the instance
contexts are.
- Generator functions didn't check the arguments to ``throw()`` the way they are
in CPython, now they are.
- Generator functions didn't trace exceptions to "stderr" if they occurred while
closing unfinished ones in "del".
- Generator functions used the slightly different wordings for some error
messages.
Function Calls
~~~~~~~~~~~~~~
- Extended call syntax with ``**`` allows that to use a mapping, and it is now
checked if it really is a mapping and if the contents has string keys.
- Similarly, extended call syntax with ``*`` allows a sequence, it is now
checked if it really is a sequence.
- Error message for duplicate keyword arguments or too little arguments now
describe the duplicate parameter and the callable the same way CPython does.
- Now checks to the keyword argument list first before considering the parameter
counts. This is slower in the error case, but more compatible with CPython.
Classes
~~~~~~~
- The "locals()" built-in when used in the class scope (not in a method) now is
correctly writable and writes to it change the resulting class.
- Name mangling for private identifiers was not always done entirely correct.
Others
~~~~~~
- Exceptions didn't always have the correct stack reported.
- The pickling of some tuples showed that "cPickle" can have non-reproducible
results, using "pickle" to stream constants now
New Optimization
----------------
- Access to instance attributes has become faster by writing specific code for
the case. This is done in JIT way, attempting at run time to optimize
attribute access for instances.
- Assignments now often consider what's cheaper for the other side, instead of
taking a reference to a global variable, just to have to release it.
- The function call code built argument tuples and dictionaries as constants,
now that is true for every tuple usage.
Cleanups
--------
- The static helper classes, and the prelude code needed have been moved to
separate C++ files and are now accessed "#include". This makes the code inside
C++ files as opposed to a Python string and therefore easier to read and or
change.
New Features
------------
- The generator functions and generator expressions have the attribute
"gi_running" now. These indicate if they are currently running.
New Tests
---------
- The script to extract the "doctests" from the CPython test suite has been
rewritten entirely and works with more doctests now. Running these tests
created increased the test coverage a lot.
- The Python 2.7 test suite has been added.
Organizational
--------------
- One can now run multiple "compare_with_cpython" instances in parallel, which
enables background test runs.
- There is now a new environment variable "NUITKA_INCLUDE" which needs to point
to the directory Nuitka's C++ includes live in. Of course the
"create-environment.sh" script generates that for you easily.
Numbers
-------
python 2.6::
Pystone(1.1) time for 50000 passes = 0.65
This machine benchmarks at 76923.1 pystones/second
Nuitka 0.3.3::
Pystone(1.1) time for 50000 passes = 0.36
This machine benchmarks at 138889 pystones/second
This is 80% for 0.3.3, up from 66% before.
Nuitka Release 0.3.2
====================
This release of Nuitka continues the focus on performance. But this release also
revisits the topic of feature parity. Before, feature parity had been reached
"only" with Python 2.6. This is of course a big thing, but you know there is
always more, e.g. Python 2.7.
With the addition of set contractions and dict contractions in this very
release, Nuitka is approaching Python support for 2.7, and then there are some
bug fixes.
Bug fixes
---------
- Calling a function with ``**`` and using a non-dict for it was leading to
wrong behavior. Now a mapping is good enough as input for the ``**`` parameter
and it's checked.
- Deeply nested packages "package.subpackage.module" were not found and gave a
warning from Nuitka, with the consequence that they were not embedded in the
executable. They now are.
- Some error messages for wrong parameters didn't match literally. For example
"function got multiple..." as opposed to "function() got multiple..." and
alike.
- Files that ended in line with a "#" but without a new line gave an error from
"ast.parse". As a workaround, a new line is added to the end of the file if
it's "missing".
- More correct exception locations for complex code lines. I noted that the
current line indication should not only be restored when the call at hand
failed, but in any case. Otherwise sometimes the exception stack would not be
correct. It now is - more often. Right now, this has no systematic test.
- Re-raised exceptions didn't appear on the stack if caught inside the same
function, these are now correct.
- For ``exec`` the globals argument needs to have "__builtins__" added, but the
check was performed with the mapping interface. That is not how CPython does
it, and so e.g. the mapping could use a default value for "__builtins__" which
could lead to incorrect behavior. Clearly a corner case, but one that works
fully compatible now.
New Optimization
----------------
- The local and shared local variable C++ classes have a flag "free_value" to
indicate if an "PY_DECREF" needs to be done when releasing the object. But
still the code used "Py_XDECREF" (which allows for "NULL" values to be
ignored.) when the releasing of the object was done. Now the inconsistency of
using "NULL" as "object" value with "free_value" set to true was removed.
- Tuple constants were copied before using them without a point. They are
immutable anyway.
Cleanups
--------
- Improved more of the indentation of the generated C++ which was not very good
for contractions so far. Now it is. Also assignments should be better now.
- The generation of code for contractions was made more general and templates
split into multiple parts. This enabled reuse of the code for list
contractions in dictionary and set contractions.
- The with statement has its own template now and got cleaned up regarding
indentation.
New Tests
---------
- There is now a script to extract the "doctests" from the CPython test suite
and it generates Python source code from them. This can be compiled with
Nuitka and output compared to CPython. Without this, the doctest parts of the
CPython test suite is mostly useless. Solving this improved test coverage,
leading to many small fixes. I will dedicate a later posting to the tool,
maybe it is useful in other contexts as well.
- Reference count tests have been expanded to cover assignment to multiple
assignment targets, and to attributes.
- The deep program test case, now also have a module in a sub-package to cover
this case as well.
Organizational
--------------
- The `gitweb interface `__ might be considered an
alternative to downloading the source if you want to provide a pointer, or
want to take a quick glance at the source code. You can already download with
git, follow the link below to the page explaining it.
- The "README.txt" has documented more of the differences and I consequently
updated the Differences page. There is now a distinction between generally
missing functionality and things that don't work in ``--deep`` mode, where
Nuitka is supposed to create one executable.
I will make it a priority to remove the (minor) issues of ``--deep`` mode in
the next release, as this is only relatively little work, and not a good
difference to have. We want these to be empty, right? But for the time being,
I document the known differences there.
Numbers
-------
python 2.6::
Pystone(1.1) time for 50000 passes = 0.65
This machine benchmarks at 76923.1 pystones/second
Nuitka 0.3.2::
Pystone(1.1) time for 50000 passes = 0.39
This machine benchmarks at 128205 pystones/second
This is 66% for 0.3.2, slightly up from the 58% of 0.3.1 before. The
optimization done were somewhat fruitful, but as you can see, they were also
more cleanups, not the big things.
Nuitka Release 0.3.1
====================
This release of Nuitka continues the focus on performance and contains only
cleanups and optimization. Most go into the direction of more readable code,
some aim at making the basic things faster, with good results as to performance
as you can see below.
New Optimization
----------------
- Constants in conditions of conditional expressions (``a if cond else d``),
``if``/``elif`` or ``while`` are now evaluated to ``true`` or ``false``
directly. Before there would be temporary python object created from it which
was then checked if it had a truth value.
All of that is obviously overhead only. And it hurts the typically ``while
1:`` infinite loop case badly.
- Do not generate code to catch ``BreakException`` or ``ContinueException``
unless a ``break`` or ``continue`` statement being in a ``try: finally:``
block inside that loop actually require this.
Even while uncaught exceptions are cheap, it is still an improvement
worthwhile and it clearly improves the readability for the normal case.
- The compiler more aggressively prepares tuples, lists and dicts from the
source code as constants if their contents is "immutable" instead of building
at run time. An example of a "mutable" tuple would be ``( {}, )`` which is not
safe to share, and therefore will still be built at run time.
For dictionaries and lists, copies will be made, under the assumption that
copying a dictionary will always be faster, than making it from scratch.
- The parameter parsing code was dynamically building the tuple of argument
names to check if an argument name was allowed by checking the equivalent of
``name in argument_names``. This was of course wasteful and now a pre-built
constant is used for this, so it should be much faster to call functions with
keyword arguments.
- There are new templates files and also actual templates now for the ``while``
and ``for`` loop code generation. And I started work on having a template for
assignments.
Cleanups
--------
- Do not generate code for the else of ``while`` and ``for`` loops if there is
no such branch. This uncluttered the generated code somewhat.
- The indentation of the generated C++ was not very good and whitespace was
often trailing, or e.g. a real tab was used instead of "\t". Some things
didn't play well together here.
Now much of the generated C++ code is much more readable and white space
cleaner. For optimization to be done, the humans need to be able to read the
generated code too. Mind you, the aim is not to produce usable C++, but on the
other hand, it must be possible to understand it.
- To the same end of readability, the empty ``else {}`` branches are avoided for
``if``, ``while`` and ``for`` loops. While the C++ compiler can be expected to
remove these, they seriously cluttered up things.
- The constant management code in ``Context`` was largely simplified. Now the
code is using the ``Constant`` class to find its way around the problem that
dicts, sets, etc. are not hashable, or that ``complex`` is not being ordered;
this was necessary to allow deeply nested constants, but it is also a simpler
code now.
- The C++ code generated for functions now has two entry points, one for Python
calls (arguments as a list and dictionary for parsing) and one where this has
happened successfully. In the future this should allow for faster function
calls avoiding the building of argument tuples and dictionaries all-together.
- For every function there was a "traceback adder" which was only used in the
C++ exception handling before exit to CPython to add to the traceback
object. This was now inlined, as it won't be shared ever.
Numbers
-------
python 2.6::
Pystone(1.1) time for 50000 passes = 0.65
This machine benchmarks at 76923.1 pystones/second
Nuitka 0.3.1::
Pystone(1.1) time for 50000 passes = 0.41
This machine benchmarks at 121951 pystones/second
This is 58% for 0.3.1, up from the 25% before. So it's getting somewhere. As
always you will find its latest version here.
Nuitka Release 0.3.0
====================
This release 0.3.0 is the first release to focus on performance. In the 0.2.x
series Nuitka achieved feature parity with CPython 2.6 and that was very
important, but now it is time to make it really useful.
Optimization has been one of the main points, although I was also a bit forward
looking to Python 2.7 language constructs. This release is the first where I
really started to measure things and removed the most important bottlenecks.
New Features
------------
- Added option to control ``--debug``. With this option the C++ debug
information is present in the file, otherwise it is not. This will give much
smaller ".so" and ".exe" files than before.
- Added option ``--no-optimization`` to disable all optimization. It enables C++
asserts and compiles with less aggressive C++ compiler optimization, so it can
be used for debugging purposes.
- Support for Python 2.7 set literals has been added.
Performance Enhancements
------------------------
- Fast global variables: Reads of global variables were fast already. This was
due to a trick that is now also used to check them and to do a much quicker
update if they are already set.
- Fast ``break``/``continue`` statements: To make sure these statements execute
the finally handlers if inside a try, these used C++ exceptions that were
caught by ``try``/``finally`` in ``while`` or ``for`` loops.
This was very slow and had very bad performance. Now it is checked if this is
at all necessary and then it's only done for the rare case where a
``break``/``continue`` really is inside the tried block. Otherwise it is now
translated to a C++ ``break``/``continue`` which the C++ compiler handles more
efficiently.
- Added ``unlikely()`` compiler hints to all errors handling cases to allow the
C++ compiler to generate more efficient branch code.
- The for loop code was using an exception handler to make sure the iterated
value was released, using ``PyObjectTemporary`` for that instead now, which
should lead to better generated code.
- Using constant dictionaries and copy from them instead of building them at run
time even when contents was constant.
New Tests
---------
- Merged some bits from the CPython 2.7 test suite that do not harm 2.6, but
generally it's a lot due to some ``unittest`` module interface changes.
- Added CPython 2.7 tests ``test_dictcomps.py`` and ``test_dictviews.py`` which
both pass when using Python 2.7.
- Added another benchmark extract from "PyStone" which uses a while loop with
break.
Numbers
-------
python 2.6::
Pystone(1.1) time for 50000 passes = 0.65
This machine benchmarks at 76923.1 pystones/second
Nuitka 0.3.0::
Pystone(1.1) time for 50000 passes = 0.52
This machine benchmarks at 96153.8 pystones/second
That's a 25% speedup now and a good start clearly. It's not yet in the range of
where i want it to be, but there is always room for more. And the
``break``/``continue`` exception was an important performance regression fix.
Nuitka Release 0.2.4
====================
This release 0.2.4 is likely the last 0.2.x release, as it's the one that
achieved feature parity with CPython 2.6, which was the whole point of the
release series, so time to celebrate. I have stayed away (mostly) from any
optimization, so as to not be premature.
From now on speed optimization is going to be the focus though. Because right
now, frankly, there is not much of a point to use Nuitka yet, with only a minor
run time speed gain in trade for a long compile time. But hopefully we can
change that quickly now.
New Features
------------
- The use of exec in a local function now adds local variables to scope it is
in.
- The same applies to ``from module_name import *`` which is now compiled
correctly and adds variables to the local variables.
Bug Fixes
---------
- Raises ``UnboundLocalError`` when deleting a local variable with ``del``
twice.
- Raises ``NameError`` when deleting a global variable with ``del`` twice.
- Read of to uninitialized closure variables gave ``NameError``, but
``UnboundLocalError`` is correct and raised now.
Cleanups
--------
- There is now a dedicated pass over the node tree right before code generation
starts, so that some analysis can be done as late as that. Currently this is
used for determining which functions should have a dictionary of locals.
- Checking the exported symbols list, fixed all the cases where a ``static`` was
missing. This reduces the "module.so" sizes.
- With gcc the "visibility=hidden" is used to avoid exporting the helper
classes. Also reduces the "module.so" sizes, because classes cannot be made
static otherwise.
New Tests
---------
- Added "DoubleDeletions" to cover behaviour of ``del``. It seems that this is
not part of the CPython test suite.
- The "OverflowFunctions" (those with dynamic local variables) now has an
interesting test, exec on a local scope, effectively adding a local variable
while a closure variable is still accessible, and a module variable too. This
is also not in the CPython test suite.
- Restored the parts of the CPython test suite that did local star imports or
exec to provide new variables. Previously these have been removed.
- Also "test_with.py" which covers PEP 343 has been reactivated, the with
statement works as expected.
Nuitka Release 0.2.3
====================
This new release is marking a closing in on feature parity to CPython 2.6 which
is an important mile stone. Once this is reached, a "Nuitka 0.3.x" series will
strive for performance.
Bug Fixes
---------
- Generator functions no longer leak references when started, but not finished.
- Yield can in fact be used as an expression and returns values that the
generator user ``send()`` to it.
Reduced Differences / New Features
----------------------------------
- Generator functions already worked quite fine, but now they have the
``throw()``, ``send()`` and ``close()`` methods.
- Yield is now an expression as is ought to be, it returns values put in by
``send()`` on the generator user.
- Support for extended slices:
.. code-block:: python
x = d[:42, ..., :24:, 24, 100]
d[:42, ..., :24:, 24, 100] = "Strange"
del d[:42, ..., :24:, 24, 100]
Tests Work
----------
- The "test_contextlib" is now working perfectly due to the generator functions
having a correct ``throw()``. Added that test back, so context managers are
now fully covered.
- Added a basic test for "overflow functions" has been added, these are the ones
which have an unknown number of locals due to the use of language constructs
``exec`` or ``from bla import *`` on the function level. This one currently
only highlights the failure to support it.
- Reverted removals of extended slice syntax from some parts of the CPython test
suite.
Cleanups
--------
- The compiled generator types are using the new C++0x type safe enums feature.
- Resolved a circular dependency between ``TreeBuilding`` and
``TreeTransforming`` modules.
Nuitka Release 0.2.2
====================
This is some significant progress, a lot of important things were addressed.
Bug Fixes
---------
- Scope analysis is now done during the tree building instead of sometimes
during code generation, this fixed a few issues that didn't show up in tests
previously.
- Reference leaks of generator expressions that were not fishing, but then
deleted are not more.
- Inlining of exec is more correct now.
- More accurate exception lines when iterator creation executes compiled code,
e.g. in a for loop
- The list of base classes of a class was evaluated in the context of the class,
now it is done in the context of the containing scope.
- The first iterated of a generator expression was evaluated in its own context,
now it is done in the context of the containing scope.
Reduced Differences
-------------------
- With the enhanced scope analysis, ``UnboundLocalError`` is now correctly
supported.
- Generator expressions (but not yet functions) have a ``throw()``, ``send()``
and ``close()`` method.
- Exec can now write to local function namespace even if ``None`` is provided at
run time.
- Relative imports inside packages are now correctly resolved at compile time
when using ``--deep``.
Cleanups
--------
- The compiled function type got further enhanced and cleaned up.
- The compiled generator expression function type lead to a massive cleanup of
the code for generator expressions.
- Cleaned up namespaces, was still using old names, or "Py*" which is reserved
to core CPython.
- Overhaul of the code responsible for ``eval`` and ``exec``, it has been split,
and it pushed the detection defaults to the C++ compiler which means, we can
do it at run time or compile time, depending on circumstances.
- Made ``PyTemporaryObject`` safer to use, disabling copy constructor it should
be also a relief to the C++ compiler if it doesn't have to eliminate all its
uses.
- The way delayed work is handled in ``TreeBuilding`` step has been changed to
use closured functions, should be more readable.
- Some more code templates have been created, making the code generation more
readable in some parts. More to come.
New Features
------------
- As I start to consider announcing Nuitka, I moved the version logic so that
the version can now be queried with ``--version``.
New Optimization
----------------
- Name lookups for ``None``, ``True`` and ``False`` and now always detected as
constants, eliminating many useless module variable lookups.
New Tests
---------
- More complete test of generator expressions.
- Added test program for packages with relative imports inside the package.
- The built-in ``dir()`` in a function was not having fully deterministic output
list, now it does.
Summary
-------
Overall, the amount of differences between CPython and Nuitka is heading towards
zero. Also most of the improvements done in this release were very
straightforward cleanups and not much work was required, mostly things are about
cleanups and then it becomes easily right. The new type for the compiled
generator expressions was simple to create, esp. as I could check what CPython
does in its source code.
For optimization purposes, I decided that generator expressions and generator
functions will be separate compiled types, as most of their behavior will not be
shared. I believe optimizing generator expressions to run well is an important
enough goal to warrant that they have their own implementation. Now that this is
done, I will repeat it with generator functions.
Generator functions already work quite fine, but like generator expressions did
before this release, they can leak references if not finished , and they don't
have the ``throw()`` method, which seems very important to the correct operation
of ``contextlib``. So I will introduce a decicated type for these too, possibly
in the next release.
Nuitka Release 0.2.1
====================
The march goes on, this is another minor release with a bunch of substantial
improvements:
Bug Fixes
---------
- Packages now also can be embedded with the ``--deep`` option too, before they
could not be imported from the executable.
- Inlined exec with their own future statements leaked these to the surrounding
code.
Reduced Differences
-------------------
- The future print function import is now supported too.
Cleanups
--------
- Independence of the compiled function type. When I started it was merely
``PyCFunction`` and then a copy of it patched at run time, using increasingly
less code from CPython. Now it's nothing at all anymore.
- This lead to major cleanup of run time compiled function creation code, no
more ``methoddefs``, ``PyCObject`` holding context, etc.
- PyLint was used to find the more important style issues and potential bugs,
also helping to identify some dead code.
Summary
-------
The major difference now is the lack of a throw method for generator
functions. I will try to address that in a 0.2.2 release if possible. The plan
is that the 0.2.x series will complete these tasks, and 0.3 could aim at some
basic optimization finally.
Nuitka Release 0.2
==================
Good day, this is a major step ahead, improvements everywhere.
Bug fixes
---------
- Migrated the Python parser from the deprecated and problematic ``compiler``
module to the ``ast`` module which fixes the ``d[a,] = b`` parser problem. A
pity it was not available at the time I started, but the migration was
relatively painless now.
- I found and fixed wrong encoding of binary data into C++ literals. Now Nuitka
uses C++0x raw strings, and these problems are gone.
- The decoding of constants was done with the ``marshal`` module, but that
appears to not deeply care enough about unicode encoding it seems. Using
``cPickle`` now, which seems less efficient, but is more correct.
- Another difference is gone: The ``continue`` and ``break`` inside loops do no
longer prevent the execution of finally blocks inside the loop.
Organizational
--------------
- I now maintain the "README.txt" in org-mode, and intend to use it as the issue
tracker, but I am still a beginner at that.
.. admonition:: Update
Turned out I never master it, and used ReStructured Text instead.
- There is a public git repository for you to track Nuitka releases. Make your
changes and then ``git pull --rebase``. If you encounter conflicts in things
you consider useful, please submit the patches and a pull offer. When you make
your clones of Nuitka public, use ``nuitka-unofficial`` or not the name
``Nuitka`` at all.
- There is a now a `mailing list `__
available too.
Reduced Differences
-------------------
- Did you know you could write ``lambda : (yield something)`` and it gives you a
lambda that creates a generator that produces that one value? Well, now Nuitka
has support for lambda generator functions.
- The ``from __future__ import division`` statement works as expected now,
leading to some newly passing CPython tests.
- Same for ``from __future__ import unicode_literals`` statement, these work as
expected now, removing many differences in the CPython tests that use this
already.
New Features
------------
- The ``Python`` binary provided and ``Nuitka.py`` are now capable of accepting
parameters for the program executed, in order to make it even more of a
drop-in replacement to ``python``.
- Inlining of ``exec`` statements with constant expressions. These are now
compiled at compile time, not at run time anymore. I observed that an
increasing number of CPython tests use exec to do things in isolation or to
avoid warnings, and many more these tests will now be more effective. I intend
to do the same with eval expressions too, probably in a minor release.
Summary
-------
So give it a whirl. I consider it to be substantially better than before, and
the list of differences to CPython is getting small enough, plus there is
already a fair bit of polish to it. Just watch out that it needs gcc-4.5 or
higher now.
Nuitka Release 0.1.1
====================
I just have just updated Nuitka to version 0.1.1 which is a bug fix release to
0.1, which corrects many of the small things:
- Updated the CPython test suite to 2.6.6rc and minimized much of existing
differences in the course.
- Compiles standalone executable that includes modules (with --deep option), but
packages are not yet included successfully.
- Reference leaks with exceptions are no more.
- sys.exc_info() works now mostly as expected (it's not a stack of exceptions).
- More readable generated code, better organisation of C++ template code.
- Restored debug option ``--g++-only``.
The biggest thing probably is the progress with exception tracebacks objects in
exception handlers, which were not there before (always ``None``). Having these
in place will make it much more compatible. Also with manually raised exceptions
and assertions, tracebacks will now be more correct to the line.
On a bad news, I discovered that the ``compiler`` module that I use to create
the AST from Python source code, is not only deprecated, but also broken. I
created the `CPython bug `__ about it,
basically it cannot distinguish some code of the form ``d[1,] = None`` from
``d[1] = None``. This will require a migration of the ``ast`` module, which
should not be too challenging, but will take some time.
I am aiming at it for a 0.2 release. Generating wrong code (Nuitka sees ``d[1] =
None`` in both cases) is a show blocker and needs a solution.
So, yeah. It's better, it's there, but still experimental. You will find its
latest version here. Please try it out and let me know what you think in the
comments section.
Nuitka Release 0.1 (Releasing Nuitka to the World)
==================================================
Obviously this is very exciting step for me. I am releasing Nuitka
today. Finally. For a long time I knew I would, but actually doing it, is a
different beast. Reaching my goals for release turned out to be less far away
than I hope, so instead of end of August, I can already release it now.
Currently it's not more than 4% faster than CPython. No surprise there, if all
you did, is removing the bytecode interpretation so far. It's not impressive at
all. It's not even a reason to use it. But it's also only a start. Clearly, once
I get into optimizing the code generation of Nuitka, it will only get better,
and then probably in sometimes dramatic steps. But I see this as a long term
goal.
I want to have infrastructure in the code place, before doing lots of possible
optimization that just make Nuitka unmaintainable. And I will want to have a
look at what others did so far in the domain of type inference and how to apply
that for my project.
I look forward to the reactions about getting this far. The supported language
volume is amazing, and I have a set of nice tricks used. For example the way
generator functions are done is a clever hack.
Where to go from here? Well, I guess, I am going to judge it by the feedback I
receive. I personally see "constant propagation" as a laudable first low hanging
fruit, that could be solved.
Consider this readable code on the module level:
.. code-block:: python
meters_per_nautical_mile = 1852
def convertMetersToNauticalMiles( meters ):
return meters / meters_per_nautical_mile
def convertNauticalMilesToMeters( miles ):
return miles * meters_per_nautical_mile
Now imagine you are using this very frequently in code. Quickly you determine
that the following will be much faster:
.. code-block:: python
def convertMetersToNauticalMiles( meters ):
return meters / 1852
def convertNauticalMilesToMeters( miles ):
return miles * 1852
Still good? Well, probably next step you are going to inline the function calls
entirely. For optimization, you are making your code less readable. I do not all
appreciate that. My first goal is there to make the more readable code perform
as well or better as the less readable variant.
Nuitka-0.5.0.1/tests/ 0000755 0001750 0001750 00000000000 12265271051 014503 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/packages/ 0000755 0001750 0001750 00000000000 12265271051 016261 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/packages/sub_package/ 0000755 0001750 0001750 00000000000 12265271051 020525 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/packages/sub_package/kitty/ 0000755 0001750 0001750 00000000000 12265271051 021671 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/packages/sub_package/kitty/smallkitty.py 0000644 0001750 0001750 00000001615 12265264105 024445 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
size = "small"
Nuitka-0.5.0.1/tests/packages/sub_package/kitty/bigkitty.py 0000644 0001750 0001750 00000001613 12265264105 024074 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
size = "big"
Nuitka-0.5.0.1/tests/packages/sub_package/kitty/speak/ 0000755 0001750 0001750 00000000000 12265271051 022774 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/packages/sub_package/kitty/speak/miau.py 0000644 0001750 0001750 00000001634 12265264105 024307 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def speak():
print "miau"
Nuitka-0.5.0.1/tests/packages/sub_package/kitty/speak/__init__.py 0000644 0001750 0001750 00000001576 12265264105 025120 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/tests/packages/sub_package/kitty/speak/purr.py 0000644 0001750 0001750 00000001636 12265264105 024346 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def speak():
print "mrrruu"
Nuitka-0.5.0.1/tests/packages/sub_package/kitty/speak/hello.py 0000644 0001750 0001750 00000002033 12265264105 024451 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def speak():
print( "hello kitty" )
# Test Issue#115, in recursing modules, this was misbehaving.
import types
assert type(speak) == types.FunctionType
Nuitka-0.5.0.1/tests/packages/sub_package/kitty/__init__.py 0000644 0001750 0001750 00000002046 12265264105 024006 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print("__name__ is ", __name__)
print("__package__ is ", __package__)
import sys
print("From sys.modules", sys.modules["kitty"])
from kitty.speak.hello import speak
Nuitka-0.5.0.1/tests/packages/run_all.py 0000755 0001750 0001750 00000005616 12265264105 020304 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import os, sys
# Find common code relative in file system. Not using packages for test stuff.
sys.path.insert(
0,
os.path.normpath(
os.path.join(
os.path.dirname( os.path.abspath( __file__ ) ),
".."
)
)
)
from test_common import (
my_print,
setup,
convertUsing2to3,
compareWithCPython,
getTempDir
)
python_version = setup()
search_mode = len( sys.argv ) > 1 and sys.argv[1] == "search"
start_at = sys.argv[2] if len( sys.argv ) > 2 else None
if start_at:
active = False
else:
active = True
for filename in sorted( os.listdir( "." ) ):
if not os.path.isdir( filename ) or filename.endswith( ".build" ):
continue
path = os.path.relpath( filename )
if not active and start_at in ( filename, path ):
active = True
extra_flags = [
"expect_success",
"remove_output",
"module_mode",
"two_step_execution"
]
# The use of "__main__" in the test package gives a warning.
if filename == "sub_package":
extra_flags.append( "ignore_warnings" )
if active:
my_print( "Consider output of recursively compiled program:", path )
for filename_main in os.listdir( filename ):
if not os.path.isdir(os.path.join(filename,filename_main)):
continue
if filename_main not in ( "..", "." ):
break
else:
sys.exit(
"""\
Error, no package in dir '%s' found, incomplete test case.""" % filename
)
os.environ[ "NUITKA_EXTRA_OPTIONS" ] = \
"--recurse-to=%s" % os.path.basename(filename_main)
os.environ[ "NUITKA_EXTRA_OPTIONS" ] += \
" --output-dir=%s" % getTempDir()
compareWithCPython(
path = os.path.join( filename, filename_main ),
extra_flags = extra_flags,
search_mode = search_mode,
needs_2to3 = False
)
else:
my_print( "Skipping", filename )
Nuitka-0.5.0.1/tests/standalone/ 0000755 0001750 0001750 00000000000 12265271051 016633 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/standalone/run_all.py 0000755 0001750 0001750 00000024072 12265264105 020653 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import os, sys, shutil
# Find common code relative in file system. Not using packages for test stuff.
sys.path.insert(
0,
os.path.normpath(
os.path.join(
os.path.dirname(os.path.abspath(__file__)),
".."
)
)
)
from test_common import (
my_print,
setup,
hasModule,
compareWithCPython,
decideFilenameVersionSkip,
getRuntimeTraceOfLoadedFiles
)
python_version = setup(needs_io_encoding=True)
search_mode = len( sys.argv ) > 1 and sys.argv[1] == "search"
start_at = sys.argv[2] if len(sys.argv) > 2 else None
if start_at:
active = False
else:
active = True
for filename in sorted(os.listdir(".")):
if not filename.endswith(".py"):
continue
if not decideFilenameVersionSkip(filename):
continue
path = os.path.relpath(filename)
if not active and start_at in (filename, path):
active = True
extra_flags = [
"expect_success",
"standalone",
"remove_output"
]
if filename == "PySideUsing.py":
# Don't test on platforms not supported by current Debian testing, and
# which should be considered irrelevant by now.
if python_version.startswith(b"2.6") or \
python_version.startswith(b"3.2"):
my_print("Skipping", filename, "not relevant.")
continue
if not hasModule("PySide"):
my_print(
"Skipping", filename, "PySide not installed for",
python_version, "but test needs it."
)
continue
# For the warnings.
extra_flags.append( "ignore_stderr" )
if filename == "PyQtUsing.py":
# Don't test on platforms not supported by current Debian testing, and
# which should be considered irrelevant by now.
if python_version.startswith(b"2.6") or \
python_version.startswith(b"3.2"):
my_print("Skipping", filename, "not relevant.")
continue
if not hasModule("PyQt4"):
my_print(
"Skipping", filename, "PyQt4 not installed for",
python_version, "but test needs it."
)
continue
# For the warnings.
extra_flags.append( "ignore_stderr" )
if filename not in ("PySideUsing.py", "PyQtUsing.py"):
extra_flags += [
"no_site"
]
if active:
my_print("Consider output of recursively compiled program:", filename)
# First compare so we know the program behaves identical.
compareWithCPython(
path = filename,
extra_flags = extra_flags,
# Do not expect PySide to work yet, because it has that bug still
# where it won't call compiled functions as slots.
search_mode = search_mode and not filename == "PySideUsing.py",
needs_2to3 = False
)
# Second use strace on the result.
loaded_filenames = getRuntimeTraceOfLoadedFiles(
path = os.path.join(
filename[:-3] + ".dist",
filename[:-3] + ".exe"
)
)
current_dir = os.path.normpath(os.getcwd())
current_dir = os.path.normcase(current_dir)
illegal_access = False
for loaded_filename in loaded_filenames:
loaded_filename = os.path.normpath(loaded_filename)
loaded_filename = os.path.normcase(loaded_filename)
if loaded_filename.startswith(current_dir):
continue
if loaded_filename.startswith(os.path.abspath(current_dir)):
continue
if loaded_filename.startswith("/etc/"):
continue
if loaded_filename.startswith("/proc/"):
continue
if loaded_filename.startswith("/dev/"):
continue
if loaded_filename.startswith("/usr/lib/locale/"):
continue
if loaded_filename.startswith("/usr/share/locale/"):
continue
if loaded_filename.startswith("/lib/libdl.") or \
loaded_filename.startswith("/lib64/libdl."):
continue
if loaded_filename.startswith("/lib/libm.") or \
loaded_filename.startswith("/lib64/libm."):
continue
if loaded_filename.startswith("/lib/libz.") or \
loaded_filename.startswith("/lib64/libz."):
continue
if loaded_filename.startswith("/lib/libutil.") or \
loaded_filename.startswith("/lib64/libutil."):
continue
if loaded_filename.startswith("/lib/libpthread.") or \
loaded_filename.startswith("/lib64/libpthread."):
continue
# Loaded by C library potentially for DNS lookups.
if os.path.basename(loaded_filename).startswith("libnss_") or \
os.path.basename(loaded_filename).startswith("libnsl"):
continue
if loaded_filename.startswith("/home/") or \
loaded_filename.startswith("/data/") or \
loaded_filename in ("/home", "/data"):
continue
if os.path.basename(loaded_filename) == "gconv-modules.cache":
continue
if loaded_filename == "/usr/lib/python" + \
python_version[:3].decode() + \
"/dist-packages/PySide":
continue
loaded_basename = os.path.basename(loaded_filename).upper()
# Windows baseline DLLs
if loaded_basename in (
"SHELL32.DLL","USER32.DLL","KERNEL32.DLL",
"NTDLL.DLL", "NETUTILS.DLL", "LOGONCLI.DLL", "GDI32.DLL",
"RPCRT4.DLL", "ADVAPI32.DLL", "SSPICLI.DLL", "SECUR32.DLL",
"KERNELBASE.DLL", "WINBRAND.DLL", "DSROLE.DLL", "DNSAPI.DLL",
"SAMCLI.DLL", "WKSCLI.DLL", "SAMLIB.DLL", "WLDAP32.DLL",
"NTDSAPI.DLL", "CRYPTBASE.DLL", "W32TOPL", "WS2_32.DLL",
"SPPC.DLL", "MSSIGN32.DLL", "CERTCLI.DLL", "WEBSERVICES.DLL",
"AUTHZ.DLL", "CERTENROLL.DLL", "VAULTCLI.DLL", "REGAPI.DLL",
"BROWCLI.DLL", "WINNSI.DLL", "DHCPCSVC6.DLL", "PCWUM.DLL",
"CLBCATQ.DLL", "IMAGEHLP.DLL", "MSASN1.DLL", "DBGHELP.DLL",
"DEVOBJ.DLL", "DRVSTORE.DLL", "CABINET.DLL", "SCECLI.DLL",
"SPINF.DLL", "SPFILEQ.DLL", "GPAPI.DLL", "NETJOIN.DLL",
"W32TOPL.DLL", "NETBIOS.DLL", "DXGI.DLL", "DWRITE.DLL",
"D3D11.DLL", "WLANAPI.DLL", "WLANUTIL.DLL", "ONEX.DLL",
"EAPPPRXY.DLL", "MFPLAT.DLL", "AVRT.DLL", "ELSCORE.DLL",
"INETCOMM.DLL", "MSOERT2.DLL", "IEUI.DLL", "MSCTF.DLL",
"MSFEEDS.DLL", "UIAUTOMATIONCORE.DLL", "PSAPI.DLL",
"EFSADU.DLL", "MFC42U.DLL", "ODBC32.DLL", "OLEDLG.DLL",
"NETAPI32.DLL", "LINKINFO.DLL", "DUI70.DLL", "ADVPACK.DLL",
"NTSHRUI.DLL", "WINSPOOL.DRV", "EFSUTIL.DLL", "WINSCARD.DLL",
"SHDOCVW.DLL", "IEFRAME.DLL", "D2D1.DLL", "GDIPLUS.DLL",
"OCCACHE.DLL", "IEADVPACK.DLL", "MLANG.DLL", "MSI.DLL",
"MSHTML.DLL", "COMDLG32.DLL", "PRINTUI.DLL", "PUIAPI.DLL",
"ACLUI.DLL", "WTSAPI32.DLL", "FMS.DLL", "DFSCLI.DLL",
"HLINK.DLL", "MSRATING.DLL", "PRNTVPT.DLL", "IMGUTIL.DLL",
"MSLS31.DLL", "VERSION.DLL", "NORMALIZ.DLL", "IERTUTIL.DLL",
"WININET.DLL", "WINTRUST.DLL", "XMLLITE.DLL", "APPHELP.DLL",
"PROPSYS.DLL", "RSTRTMGR.DLL", "NCRYPT.DLL", "BCRYPT.DLL",
"MMDEVAPI.DLL", "MSILTCFG.DLL", "DEVMGR.DLL", "DEVRTL.DLL",
"NEWDEV.DLL", "VPNIKEAPI.DLL", "WINHTTP.DLL", "WEBIO.DLL",
"NSI.DLL", "DHCPCSVC.DLL", "CRYPTUI.DLL", "ESENT.DLL",
"DAVHLPR.DLL", "CSCAPI.DLL", "ATL.DLL", "OLEAUT32.DLL",
"SRVCLI.DLL", "RASDLG.DLL", "MPRAPI.DLL", "RTUTILS.DLL",
"RASMAN.DLL", "MPRMSG.DLL", "SLC.DLL", "CRYPTSP.DLL",
"RASAPI32.DLL", "TAPI32.DLL", "EAPPCFG.DLL", "NDFAPI.DLL",
"WDI.DLL", "COMCTL32.DLL", "UXTHEME.DLL", "IMM32.DLL",
"OLEACC.DLL", "WINMM.DLL", "WINDOWSCODECS.DLL", "DWMAPI.DLL",
"DUSER.DLL", "PROFAPI.DLL", "URLMON.DLL", "SHLWAPI.DLL",
"LPK.DLL", "USP10.DLL", "CFGMGR32.DLL", "MSIMG32.DLL",
"POWRPROF.DLL", "SETUPAPI.DLL", "WINSTA.DLL", "CRYPT32.DLL",
"IPHLPAPI.DLL", "MPR.DLL", "CREDUI.DLL", "NETPLWIZ.DLL",
"OLE32.DLL", "ACTIVEDS.DLL", "ADSLDPC.DLL", "USERENV.DLL"):
continue
# Win API can be assumed.
if loaded_basename.startswith("API-MS-WIN"):
continue
# MSVC run time DLLs, seem to sometimes come from system. TODO:
# clarify if that means we did it wrong.
if loaded_basename in ("MSVCRT.DLL", "MSVCR90.DLL"):
continue
my_print("Should not access '%s'." % loaded_filename)
illegal_access = True
if illegal_access:
sys.exit(1)
shutil.rmtree(filename[:-3] + ".dist")
else:
my_print("Skipping", filename)
Nuitka-0.5.0.1/tests/standalone/PyQtUsing.py 0000644 0001750 0001750 00000003150 12265264105 021111 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# This test is using signals and will only work if PySide properly accepts
# compiled functions as callables.
from __future__ import print_function
from PyQt4.QtCore import pyqtSlot, pyqtSignal, QObject, QMetaObject
class Communicate(QObject):
speak = pyqtSignal(int)
def __init__(self,name='',parent = None):
QObject.__init__(self,parent)
self.setObjectName(name)
class Speaker(QObject):
@pyqtSlot(int)
def on_communicator_speak(self, stuff):
print(stuff)
speaker = Speaker()
someone = Communicate(name='communicator',parent=speaker)
QMetaObject.connectSlotsByName(speaker)
print('The answer is:',end="")
# emit 'speak' signal
someone.speak.emit(42)
print('Slot should have made output by now.')
Nuitka-0.5.0.1/tests/standalone/ShlibUsing.py 0000644 0001750 0001750 00000001770 12265264105 021263 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
try:
import pyexpat
print(pyexpat.__doc__)
except ImportError:
print("Skipped, no pyexpat module installed.")
Nuitka-0.5.0.1/tests/standalone/Issue116_2.py 0000644 0001750 0001750 00000001634 12265264105 020754 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print('4B4159'.decode('hex'))
Nuitka-0.5.0.1/tests/standalone/PySideUsing.py 0000644 0001750 0001750 00000003131 12265264105 021410 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python test originally created or extracted from other peoples work. The
# parts from me are licensed as below. It is at least Free Softwar where
# it's copied from other people. In these cases, that will normally be
# indicated.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# This test is using signals and will only work if PySide properly accepts
# compiled functions as callables.
from __future__ import print_function
from PySide.QtCore import Slot, Signal, QObject, QMetaObject
class Communicate(QObject):
speak = Signal(int)
def __init__(self,name='',parent = None):
QObject.__init__(self,parent)
self.setObjectName(name)
class Speaker(QObject):
@Slot(int)
def on_communicator_speak(self, stuff):
print(stuff)
speaker = Speaker()
someone = Communicate(name='communicator',parent=speaker)
QMetaObject.connectSlotsByName(speaker)
print('The answer is:',end="")
# emit 'speak' signal
someone.speak.emit(42)
print('Slot should have made output by now.')
Nuitka-0.5.0.1/tests/syntax/ 0000755 0001750 0001750 00000000000 12265271051 016031 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/syntax/SyntaxError.py 0000644 0001750 0001750 00000001545 12265264105 020712 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Test the syntax error case:
def nested():
claxss ProxyBase(metaclass=ProxyType):
pass
Nuitka-0.5.0.1/tests/syntax/TryExceptAllNotLast.py 0000644 0001750 0001750 00000001527 12265264105 022277 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def f():
try:
raise A
except:
print "caught"
except A:
print "hit"
Nuitka-0.5.0.1/tests/syntax/ContinueWithoutLoop.py 0000644 0001750 0001750 00000001524 12265264105 022411 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def test_from_format(self):
if sys.version_info >= ( 3, 3 ):
continue
Nuitka-0.5.0.1/tests/syntax/ExecWithNesting2.py 0000644 0001750 0001750 00000001525 12265264105 021542 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def test_basic():
exec "def f(): pass"
def callback():
return self
Nuitka-0.5.0.1/tests/syntax/IndentationError.py 0000644 0001750 0001750 00000001437 12265264105 021700 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def someFunc():
a
b
Nuitka-0.5.0.1/tests/syntax/NonlocalForParameter3.py 0000644 0001750 0001750 00000001433 12265264105 022546 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def f(a):
nonlocal a
Nuitka-0.5.0.1/tests/syntax/ClosureDel2.py 0000644 0001750 0001750 00000001711 12265264105 020530 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def deletingClosure():
a = 1
def closureTaker():
return a
del a
try:
x = closureTaker()
except Exception as e:
print "Occured %r" % e
deletingClosure()
Nuitka-0.5.0.1/tests/syntax/run_all.py 0000755 0001750 0001750 00000004136 12265264105 020050 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import os, sys
# Find common code relative in file system. Not using packages for test stuff.
sys.path.insert(
0,
os.path.normpath(
os.path.join(
os.path.dirname( os.path.abspath( __file__ ) ),
".."
)
)
)
from test_common import (
my_print,
setup,
compareWithCPython
)
python_version = setup()
search_mode = len( sys.argv ) > 1 and sys.argv[1] == "search"
start_at = sys.argv[2] if len( sys.argv ) > 2 else None
if start_at:
active = False
else:
active = True
for filename in sorted( os.listdir( "." ) ):
if not filename.endswith( ".py" ) or filename == "run_all.py":
continue
path = filename
if not active and start_at in ( filename, path ):
active = True
# Some syntax errors are for Python3 only.
if filename == "Importing2.py" and python_version < b"3":
extra_flags = [ "remove_output" ]
elif filename == "GeneratorReturn2.py" and python_version >= b"3.3":
extra_flags = [ "remove_output" ]
else:
extra_flags = [ "expect_failure", "remove_output" ]
if active:
compareWithCPython(
path = path,
extra_flags = extra_flags,
search_mode = search_mode,
needs_2to3 = False
)
else:
my_print( "Skipping", filename )
Nuitka-0.5.0.1/tests/syntax/ClassReturn.py 0000644 0001750 0001750 00000001427 12265264105 020656 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
class X:
return 3
Nuitka-0.5.0.1/tests/syntax/MisplacedFutureImport.py 0000644 0001750 0001750 00000001514 12265264105 022675 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def f():
from __future__ import print_function
print( locals() )
Nuitka-0.5.0.1/tests/syntax/BreakWithoutLoop.py 0000644 0001750 0001750 00000001544 12265264105 021653 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# marker1
def test_from_format(self):
if sys.version_info >= ( 3, 3 ):
break
# marker2
Nuitka-0.5.0.1/tests/syntax/ModuleReturn.py 0000644 0001750 0001750 00000001441 12265264105 021032 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
lalala = 3
return
lelele = 7
Nuitka-0.5.0.1/tests/syntax/UnknownEncoding.py 0000644 0001750 0001750 00000001424 12265264105 021514 0 ustar hayen hayen 0000000 0000000 # coding: no-exist
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/tests/syntax/Importing2.py 0000644 0001750 0001750 00000001751 12265264105 020443 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def starImportFailure():
from doctest import *
try:
sys
print( "but it does not" )
except NameError:
print( "and it does" )
print( "Star import needs to respect __all__", starImportFailure() )
Nuitka-0.5.0.1/tests/syntax/DuplicateArgument.py 0000644 0001750 0001750 00000001427 12265264105 022026 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def f(a, a):
pass
Nuitka-0.5.0.1/tests/syntax/RelativeNonPackageImport.py 0000644 0001750 0001750 00000001530 12265264105 023301 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Not allowed without being a package, should raise ValueError
from . import whatever
Nuitka-0.5.0.1/tests/syntax/GlobalForParameter.py 0000644 0001750 0001750 00000001431 12265264105 022114 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def f(a):
global a
Nuitka-0.5.0.1/tests/syntax/StarImportExtra.py 0000644 0001750 0001750 00000001577 12265264105 021527 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
try:
from sys import not_there, *
except Exception as e:
print( "Star import with extra stuff not present gave", e )
Nuitka-0.5.0.1/tests/syntax/NonlocalNotFound3.py 0000644 0001750 0001750 00000001547 12265264105 021721 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def testNonlocal():
x = 0
y = 0
def f():
nonlocal z
f()
testNonlocal()
Nuitka-0.5.0.1/tests/syntax/YieldInModule.py 0000644 0001750 0001750 00000001446 12265264105 021115 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
x = (i for i in
(yield) if (yield))
Nuitka-0.5.0.1/tests/syntax/GeneratorReturn2.py 0000644 0001750 0001750 00000001561 12265264105 021620 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def g():
for a in range(3):
yield a
return 7
print( "Yielder with return value", list(g()) )
Nuitka-0.5.0.1/tests/syntax/ExecWithShortTuple2.py 0000644 0001750 0001750 00000001424 12265264105 022242 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
exec("print hey",)
Nuitka-0.5.0.1/tests/syntax/FutureBraces.py 0000644 0001750 0001750 00000001474 12265264105 021005 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import braces
print( "Look ma, braces." )
Nuitka-0.5.0.1/tests/syntax/LateFutureImport.py 0000644 0001750 0001750 00000001604 12265264105 021661 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Not allowed to do future imports that are not the first statements of the
# module.
a = 1
from __future__ import print_function
Nuitka-0.5.0.1/tests/reflected/ 0000755 0001750 0001750 00000000000 12265271051 016440 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/reflected/compile_itself.py 0000755 0001750 0001750 00000024206 12265264105 022021 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import os, sys, shutil, tempfile, time, difflib, subprocess
# Find common code relative in file system. Not using packages for test stuff.
sys.path.insert(
0,
os.path.normpath(
os.path.join(
os.path.dirname( os.path.abspath( __file__ ) ),
".."
)
)
)
from test_common import (
my_print,
setup,
getTempDir
)
python_version = setup()
# TODO: This ought to no longer be necessary, removing it could highlight bugs.
# No random hashing, it makes comparing outputs futile.
if "PYTHONHASHSEED" not in os.environ:
os.environ[ "PYTHONHASHSEED" ] = "0"
nuitka_main_path = os.path.join( "..", "..", "bin", "nuitka" )
tmp_dir = getTempDir()
# TODO: Could detect this more automatic.
PACKAGE_LIST = (
'nuitka',
'nuitka/nodes',
'nuitka/tree',
'nuitka/build',
'nuitka/freezer',
'nuitka/gui',
'nuitka/codegen',
'nuitka/codegen/templates',
'nuitka/optimizations',
'nuitka/finalizations',
)
def diffRecursive( dir1, dir2 ):
done = set()
for filename in os.listdir( dir1 ):
path1 = os.path.join( dir1, filename )
path2 = os.path.join( dir2, filename )
done.add( path1 )
# Skip these binary files of course.
if filename.endswith( ".o" ) or \
filename.endswith( ".os" ) or \
filename.endswith( ".obj" ):
continue
# Skip scons build database
if filename == ".sconsign.dblite":
continue
if not os.path.exists( path2 ):
sys.exit( "Only in %s: %s" % ( dir1, filename ))
if os.path.isdir( path1 ):
diffRecursive( path1, path2 )
elif os.path.isfile( path1 ):
fromdate = time.ctime( os.stat( path1 ).st_mtime )
todate = time.ctime( os.stat( path2 ).st_mtime )
diff = difflib.unified_diff(
a = open( path1, "rb" ).readlines(),
b = open( path2, "rb" ).readlines(),
fromfile = path1,
tofile = path2,
fromfiledate = fromdate,
tofiledate = todate,
n = 3
)
result = list( diff )
if result:
for line in result:
my_print( line )
sys.exit( 1 )
else:
assert False, path1
for filename in os.listdir( dir2 ):
path1 = os.path.join( dir1, filename )
path2 = os.path.join( dir2, filename )
if path1 in done:
continue
if not os.path.exists( path1 ):
sys.exit( "Only in %s: %s" % ( dir2, filename ))
def executePASS1():
my_print( "PASS 1: Compiling from compiler running from .py files." )
base_dir = os.path.join( "..", ".." )
for package in PACKAGE_LIST:
package = package.replace( "/", os.path.sep )
source_dir = os.path.join( base_dir, package )
target_dir = package
if os.path.exists( target_dir ):
shutil.rmtree( target_dir )
os.mkdir( target_dir )
for filename in os.listdir( target_dir ):
if filename.endswith( ".so" ):
path = os.path.join( target_dir, filename )
os.unlink( path )
for filename in sorted( os.listdir( source_dir ) ):
if not filename.endswith(".py"):
continue
if filename.startswith(".#"):
continue
path = os.path.join( source_dir, filename )
if filename != "__init__.py":
my_print( "Compiling", path )
command = [
os.environ[ "PYTHON" ],
nuitka_main_path,
"--module",
"--recurse-none",
"--output-dir=%s" % target_dir,
path
]
command += os.environ.get( "NUITKA_EXTRA_OPTIONS", "" ).split()
result = subprocess.call(
command
)
if result != 0:
sys.exit( result )
else:
shutil.copyfile( path, os.path.join( target_dir, filename ) )
my_print( "Compiling", nuitka_main_path )
shutil.copyfile( nuitka_main_path, "nuitka.py" )
command = [
os.environ[ "PYTHON" ],
nuitka_main_path,
"--recurse-none",
"--output-dir=.",
"nuitka.py"
]
command += os.environ.get( "NUITKA_EXTRA_OPTIONS", "" ).split()
result = subprocess.call(
command
)
if result != 0:
sys.exit( result )
scons_inline_copy_path = os.path.join( base_dir, "nuitka", "build", "inline_copy" )
if os.path.exists( scons_inline_copy_path ):
shutil.copytree(
scons_inline_copy_path,
os.path.join( "nuitka", "build", "inline_copy" )
)
shutil.copy(
os.path.join( base_dir, "nuitka", "build", "SingleExe.scons" ),
os.path.join( "nuitka", "build", "SingleExe.scons" )
)
shutil.copytree(
os.path.join( base_dir, "nuitka", "build", "static_src" ),
os.path.join( "nuitka", "build", "static_src" )
)
shutil.copytree(
os.path.join( base_dir, "nuitka", "build", "include" ),
os.path.join( "nuitka", "build", "include" )
)
def compileAndCompareWith( nuitka ):
base_dir = os.path.join( "..", ".." )
for package in PACKAGE_LIST:
package = package.replace( "/", os.path.sep )
source_dir = os.path.join( base_dir, package )
for filename in sorted( os.listdir( source_dir ) ):
if not filename.endswith(".py"):
continue
if filename.startswith(".#"):
continue
path = os.path.join( source_dir, filename )
if filename != "__init__.py":
my_print( "Compiling", path )
target = filename.replace( ".py", ".build" )
target_dir = os.path.join( tmp_dir, target )
if os.path.exists( target_dir ):
shutil.rmtree( target_dir )
command = [
nuitka,
"--module",
"--recurse-none",
"--output-dir=%s"% tmp_dir,
path
]
command += os.environ.get( "NUITKA_EXTRA_OPTIONS", "" ).split()
result = subprocess.call(
command
)
if result != 0:
sys.exit( result )
diffRecursive( os.path.join( package, target ), target_dir )
shutil.rmtree(target_dir)
if os.name == "nt":
target_filename = filename.replace(".py", ".pyd")
else:
target_filename = filename.replace(".py", ".so")
os.unlink(os.path.join(tmp_dir, target_filename))
def executePASS2():
my_print( "PASS 2: Compiling from compiler running from .exe and many .so files." )
# Windows will load the compiled modules (pyd) only from PYTHONPATH, so we
# have to add it.
if os.name == "nt":
os.environ[ "PYTHONPATH" ] = ":".join( PACKAGE_LIST )
compileAndCompareWith( os.path.join( ".", "nuitka.exe" ) )
# Undo the damage from above.
if os.name == "nt":
del os.environ[ "PYTHONPATH" ]
my_print( "OK." )
def executePASS3():
my_print( "PASS 3: Compiling from compiler running from .py files to single .exe." )
exe_path = os.path.join( tmp_dir, "nuitka.exe" )
if os.path.exists( exe_path ):
os.unlink( exe_path )
build_path = os.path.join( tmp_dir, "nuitka.build" )
if os.path.exists( build_path ):
shutil.rmtree( build_path )
path = os.path.join( "..", "..", "bin", "nuitka" )
my_print( "Compiling", path )
command = [
os.environ[ "PYTHON" ],
nuitka_main_path,
path,
"--output-dir=%s" % tmp_dir,
"--exe",
"--recurse-all"
]
result = subprocess.call(
command
)
if result != 0:
sys.exit( result )
shutil.rmtree( build_path )
my_print( "OK." )
def executePASS4():
my_print( "PASS 4: Compiling the compiler running from single exe" )
exe_path = os.path.join( tmp_dir, "nuitka.exe" )
compileAndCompareWith( exe_path )
my_print( "OK." )
def executePASS5():
my_print( "PASS 5: Compiling the compiler 'nuitka' package to a single '.so' file." )
path = os.path.join( "..", "..", "nuitka" )
command = [
os.environ[ "PYTHON" ],
nuitka_main_path,
path,
"--output-dir=%s" % tmp_dir,
"--recurse-all",
"--recurse-dir=%s" % path,
"--module",
path
]
result = subprocess.call(
command
)
if result != 0:
sys.exit( result )
os.unlink( os.path.join( tmp_dir, "nuitka.so" ) )
shutil.rmtree( os.path.join( tmp_dir, "nuitka.build" ) )
cross_compilation = "--windows-target" in os.environ.get( "NUITKA_EXTRA_OPTIONS", "" )
executePASS1()
if cross_compilation:
my_print( "PASS 2: Skipped for cross-compilation case." )
else:
executePASS2()
executePASS3()
if cross_compilation:
my_print( "PASS 4: Skipped for cross-compilation case." )
else:
executePASS4()
shutil.rmtree( "nuitka" )
executePASS5()
os.unlink(os.path.join(tmp_dir, "nuitka.exe" ))
os.rmdir(tmp_dir)
Nuitka-0.5.0.1/tests/test_common.py 0000644 0001750 0001750 00000023662 12265264105 017417 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import print_function
import os, sys, subprocess, tempfile, atexit, shutil, re
# Make sure we flush after every print, the "-u" option does more than that
# and this is easy enough.
def my_print(*args, **kwargs):
print(*args, **kwargs)
sys.stdout.flush()
def check_output(*popenargs, **kwargs):
if 'stdout' in kwargs:
raise ValueError('stdout argument not allowed, it will be overridden.')
process = subprocess.Popen(
stdout = subprocess.PIPE,
*popenargs,
**kwargs
)
output, _unused_err = process.communicate()
retcode = process.poll()
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
raise subprocess.CalledProcessError( retcode, cmd, output=output )
return output
def setup(needs_io_encoding = False):
# Go its own directory, to have it easy with path knowledge.
os.chdir(
os.path.dirname(
os.path.abspath( sys.modules[ "__main__" ].__file__ )
)
)
if "PYTHON" not in os.environ:
os.environ["PYTHON"] = sys.executable
if needs_io_encoding and "PYTHONIOENCODING" not in os.environ:
os.environ["PYTHONIOENCODING"] = "utf-8"
version_output = check_output(
(
os.environ["PYTHON"],
"-c",
"""\
import sys, os;\
print(".".join(str(s) for s in list(sys.version_info)[:3]));\
print(("x86_64" if "AMD64" in sys.version else "x86") if os.name=="nt" else os.uname()[4]);\
""",
),
stderr = subprocess.STDOUT
)
global python_version, python_arch
python_version = version_output.split(b"\n")[0].strip()
python_arch = version_output.split(b"\n")[1].strip()
my_print("Using concrete python", python_version, "on", python_arch)
assert type( python_version ) is bytes
return python_version
tmp_dir = None
def getTempDir():
# Create a temporary directory to work in, automatically remove it in case
# it is empty in the end.
global tmp_dir
if tmp_dir is None:
tmp_dir = tempfile.mkdtemp(
prefix = os.path.basename(
os.path.dirname(
os.path.abspath( sys.modules[ "__main__" ].__file__ )
)
) + "-",
dir = tempfile.gettempdir() if
not os.path.exists( "/var/tmp" ) else
"/var/tmp"
)
def removeTempDir():
try:
os.rmdir(tmp_dir)
except OSError:
pass
atexit.register(removeTempDir)
return tmp_dir
def convertUsing2to3( path ):
filename = os.path.basename( path )
new_path = os.path.join( getTempDir(), filename )
shutil.copy( path, new_path )
# On Windows, we cannot rely on 2to3 to be in the path.
if os.name == "nt":
command = [
sys.executable,
os.path.join(
os.path.dirname( sys.executable ),
"Tools/Scripts/2to3.py"
)
]
else:
command = [ "2to3" ]
command += [
"-w",
"-n",
"--no-diffs",
new_path
]
check_output(
command,
stderr = open( os.devnull, "w" ),
)
return new_path
def decideFilenameVersionSkip(filename):
assert type(filename) is str
assert type(python_version) is bytes
# Skip runner scripts by default.
if filename.startswith("run_"):
return False
# Skip tests that require Python 2.7 at least.
if filename.endswith("27.py") and python_version.startswith(b"2.6"):
return False
if filename.endswith("_2.py") and python_version.startswith(b"3"):
return False
# Skip tests that require Python 3.2 at least.
if filename.endswith("32.py") and not python_version.startswith(b"3"):
return False
# Skip tests that require Python 3.3 at least.
if filename.endswith("33.py") and not python_version.startswith(b"3.3"):
return False
return True
def compareWithCPython(path, extra_flags, search_mode, needs_2to3):
# Apply 2to3 conversion if necessary.
if needs_2to3:
path = convertUsing2to3(path)
command = [
sys.executable,
os.path.join("..", "..", "bin", "compare_with_cpython"),
path,
"silent"
]
command += extra_flags
try:
result = subprocess.call(
command
)
except KeyboardInterrupt:
result = 2
if result != 0 and result != 2 and search_mode:
my_print("Error exit!", result)
sys.exit(result)
if needs_2to3:
os.unlink(path)
if result == 2:
sys.stderr.write("Interruped, with CTRL-C\n")
sys.exit(2)
def hasDebugPython():
global python_version
# On Debian systems, these work.
debug_python = os.path.join("/usr/bin/", os.environ["PYTHON"] + "-dbg")
if os.path.exists(debug_python):
return True
# For self compiled Python, if it's the one also executing the runner, lets
# use it.
if sys.executable == os.environ["PYTHON"] and \
hasattr(sys, "gettotalrefcount"):
return True
# Otherwise no.
return False
def getArchitecture():
if os.name == "nt":
if "AMD64" in sys.version:
return "x86_64"
else:
return "x86"
else:
return os.uname()[4]
def getDependsExePath():
if "APPDATA" not in os.environ:
sys.exit("Error, standalone mode cannot find 'APPDATA' environment.")
nuitka_app_dir = os.path.join(os.environ["APPDATA"],"nuitka")
depends_dir = os.path.join(
nuitka_app_dir,
python_arch,
)
depends_exe = os.path.join(
depends_dir,
"depends.exe"
)
assert os.path.exists(depends_exe), depends_exe
return depends_exe
def getRuntimeTraceOfLoadedFiles(path,trace_error=True):
""" Returns the files loaded when executing a binary. """
result = []
if os.name == "posix":
args = (
"strace",
"-e", "file",
"-s4096", # Some paths are truncated otherwise.
path
)
process = subprocess.Popen(
args = args,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
stdout_strace, stderr_strace = process.communicate()
open(path+".strace","wb").write(stderr_strace)
for line in stderr_strace.split(b"\n"):
if process.returncode != 0 and trace_error:
my_print(line)
if not line:
continue
# Don't consider files not found. The "site" module checks lots
# of things.
if b"ENOENT" in line:
continue
# Allow stats on the python binary, and stuff pointing to the
# standard library, just not uses of it. It will search there
# for stuff.
if line.startswith(b"lstat(") or \
line.startswith(b"stat(") or \
line.startswith(b"readlink("):
filename = line[line.find(b"(")+2:line.find(b", ")-1]
if filename in (b"/usr", b"/usr/bin"):
continue
if filename == b"/usr/bin/python" + python_version[:3]:
continue
if filename in (b"/usr/bin/python", b"/usr/bin/python2",
b"/usr/bin/python3"):
continue
result.extend(
os.path.abspath(match)
for match in
re.findall(b'"(.*?)"', line)
)
elif os.name == "nt":
subprocess.call(
(
getDependsExePath(),
"-c",
"-ot%s" % path + ".depends",
"-f1",
"-pa1",
"-ps1",
"-pp0",
"-pl1",
path
)
)
inside = False
for line in open(path + ".depends"):
if "| Module Dependency Tree |" in line:
inside = True
continue
if not inside:
continue
if "| Module List |" in line:
break
if "]" not in line:
continue
# Skip missing DLLs, apparently not needed anyway.
if "?" in line[:line.find("]")]:
continue
dll_filename = line[line.find("]")+2:-1]
assert os.path.isfile(dll_filename), dll_filename
# The executable itself is of course excempted.
if os.path.normcase(dll_filename) == \
os.path.normcase(os.path.abspath(path)):
continue
dll_filename = os.path.normcase(dll_filename)
result.append(dll_filename)
os.unlink(path + ".depends")
result = list(sorted(set(result)))
if sys.version.startswith("3"):
result = [s.decode("utf-8") for s in result]
return result
def hasModule(module_name):
result = subprocess.call(
(
os.environ["PYTHON"],
"-c"
"import %s" % module_name
),
stdout = open(os.devnull,"w"),
stderr = subprocess.STDOUT
)
return result == 0
Nuitka-0.5.0.1/tests/basics/ 0000755 0001750 0001750 00000000000 12265271051 015747 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/basics/Printing.py 0000644 0001750 0001750 00000002565 12265264105 020125 0 ustar hayen hayen 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# All of these should be identical with correct software behaviour.
print "Output with newline."
print "Output", "with", "newline."
print "Output trailing spaces ", "with ", "newline."
print "Output ",
print "with ",
print "newline."
print "Output\twith tab"
print "Output\t",
print "with tab"
# These ones gave errors with previos literal bugs:
print "changed 2"
print "foo%sbar%sfred%sbob?????"
a = "partial print"
# b doesn't exist
try:
print a, b
except Exception, e:
print "then occured", repr(e)
print "No newline at the end",
x = 1
print """
New line is no soft space, is it
""", x
Nuitka-0.5.0.1/tests/basics/Crashers.py 0000644 0001750 0001750 00000002672 12265264105 020104 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import random
def getrandom():
return random.random()
def optimizerCrashIssue13():
try:
print "Something with side effects that might raise."
except Exception,x:
print "Caught it"
raise
print "Should not reach this"
raise
# Just so it won't be optimized away entirely.
optimizerCrashIssue13()
def codegeneratorCrashIssue15():
f = float( "nan" )
g = getrandom() # Prevent optimization of nan-constant
return f+g
# Just so it won't be optimized away entirely.
codegeneratorCrashIssue15()
def codegeneratorCrashIssue30():
f = getrandom() # Prevent optimization
f # Will be optimized way in later versions of Nuitka.
Nuitka-0.5.0.1/tests/basics/HelloWorld.py 0000644 0001750 0001750 00000002407 12265264105 020401 0 ustar hayen hayen 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print "Hello World from Module main Code"
def printHelloWorld():
print "Hello World from Function main Code"
print printHelloWorld
printHelloWorld()
def printHelloWorld2(arg):
print arg
print printHelloWorld2
printHelloWorld2( "Hello World from Function positional argument" )
printHelloWorld2( arg = "Hello World from Function keyword argument" )
def printHelloWorld3(arg = "Hello World from Function default argument"):
print arg
print printHelloWorld3
printHelloWorld3() Nuitka-0.5.0.1/tests/basics/run_xml.py 0000755 0001750 0001750 00000006323 12265264105 020016 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import print_function
import os, sys, subprocess, tempfile, shutil
# Go its own directory, to have it easy with path knowledge.
nuitka1 = sys.argv[1]
nuitka2 = sys.argv[2]
search_mode = len( sys.argv ) > 3 and sys.argv[3] == "search"
start_at = sys.argv[4] if len( sys.argv ) > 4 else None
if start_at:
active = False
else:
active = True
def check_output(*popenargs, **kwargs):
from subprocess import Popen, PIPE, CalledProcessError
if 'stdout' in kwargs:
raise ValueError('stdout argument not allowed, it will be overridden.')
process = Popen(stdout=PIPE, *popenargs, **kwargs)
output, unused_err = process.communicate()
retcode = process.poll()
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
raise CalledProcessError(retcode, cmd, output=output)
return output
my_dir = os.path.dirname( os.path.abspath( __file__ ) )
for filename in sorted( os.listdir( my_dir ) ):
if not filename.endswith( ".py" ) or filename.startswith( "run_" ):
continue
path = os.path.relpath( os.path.join( my_dir, filename ) )
if not active and start_at in ( filename, path ):
active = True
if active:
# TODO: Reactivate Python3 support here.
if False:
new_path = os.path.join( tempfile.gettempdir(), filename )
shutil.copy( path, new_path )
path = new_path
# On Windows, we cannot rely on 2to3 to be in the path.
if os.name == "nt":
command = sys.executable + " " + os.path.join( os.path.dirname( sys.executable ), "Tools/Scripts/2to3.py" )
else:
command = "2to3"
result = subprocess.call(
command + " -w -n --no-diffs " + path,
stderr = open( "/dev/null", "w" ),
shell = True
)
command = "%s %s '%s' '%s' %s" % (
sys.executable,
os.path.join( my_dir, "..", "..", "bin", "compare_with_xml" ),
nuitka1,
nuitka2,
path,
)
result = subprocess.call(
command,
shell = True
)
if result == 2:
sys.stderr.write( "Interruped, with CTRL-C\n" )
sys.exit( 2 )
if result != 0 and search_mode:
print("Error exit!", result)
sys.exit( result )
else:
print("Skipping", filename)
Nuitka-0.5.0.1/tests/basics/Varargs.py 0000644 0001750 0001750 00000004120 12265264105 017725 0 ustar hayen hayen 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def plain_list_dict_args_function(plain, *arg_list, **arg_dict):
print "plain", plain, "arg_list", arg_list, "arg_dict", arg_dict
def plain_list_args_function(plain, *arg_list):
print plain, arg_list
def plain_dict_args_function(plain, **arg_dict):
print plain, arg_dict
print "Function with plain arg and varargs dict:"
plain_dict_args_function( 1, a = 2, b = 3, c = 4 )
plain_dict_args_function( 1 )
print "Function with plain arg and varargs list:"
plain_list_args_function( 1, 2, 3, 4 )
plain_list_args_function( 1 )
print "Function with plain arg, varargs list and varargs dict:"
plain_list_dict_args_function( 1, 2, z = 3 )
plain_list_dict_args_function( 1, 2, 3 )
plain_list_dict_args_function( 1, a = 2, b = 3, c = 4 )
def list_dict_args_function(*arg_list, **arg_dict):
print arg_list, arg_dict
def list_args_function(*arg_list):
print arg_list
def dict_args_function(**arg_dict):
print arg_dict
print "Function with plain arg and varargs dict:"
dict_args_function( a = 2, b = 3, c = 4 )
dict_args_function()
print "Function with plain arg and varargs list:"
list_args_function( 2, 3, 4 )
list_args_function()
print "Function with plain arg, varargs list and varargs dict:"
list_dict_args_function( 2, z = 3 )
list_dict_args_function( 2, 3 )
list_dict_args_function( a = 2, b = 3, c = 4 )
Nuitka-0.5.0.1/tests/basics/Decorators.py 0000644 0001750 0001750 00000003054 12265264105 020432 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def decorator1(f):
print "Executing decorator 1"
def deco_f():
return f() + 2
return deco_f
def decorator2(f):
print "Executing decorator 2"
def deco_f():
return f() * 2
return deco_f
# Result of function now depends on correct order of applying the decorators
@decorator1
@decorator2
def function1():
return 3
print function1()
def deco_returner1():
print "Executing decorator returner D1"
return decorator1
def deco_returner2():
print "Executing decorator returner D2"
return decorator2
@deco_returner1()
@deco_returner2()
def function2():
return 3
print function2()
# Same as function2, but without decorator syntax.
def function3():
return 3
function3 = deco_returner1()( deco_returner2()( function3 ) )
print function3()
Nuitka-0.5.0.1/tests/basics/TryReturnFinally.py 0000644 0001750 0001750 00000001763 12265264105 021627 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def returnInTried():
try:
return 8
finally:
print "returnInTried",
def returnInFinally():
try:
print "returnInFinally tried",
finally:
return 9
print returnInTried()
print returnInFinally()
Nuitka-0.5.0.1/tests/basics/BuiltinsTest.py 0000644 0001750 0001750 00000017041 12265264246 020765 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def someFunctionWritingLocals():
x = 1
r = locals()
# This is without effect on r. It doesn't mention y at all
y = 2
# This adds z to the locals, but only that.
r[ "z" ] = 3
del x
try:
z
except Exception, e:
print "Accessing z writing to locals gives Exception", e
return r, y
def someFunctionWritingLocalsContainingExec():
x = 1
r = locals()
# This is without effect on r. It doesn't mention y at all
y = 2
# This adds z to the locals, but only that.
r[ "z" ] = 3
try:
z
except Exception, e:
print "Accessing z writing to locals in exec function gives Exception", e
return r, y
# Note: This exec is dead code, and still changes the behaviour of
# CPython, because it detects exec during parse already.
exec ""
print "Testing locals():"
print someFunctionWritingLocals()
print someFunctionWritingLocalsContainingExec()
def displayDict(d):
d = dict(d)
if "__loader__" in d:
d[ "__loader__" ] = ""
return repr( d )
print "Vars on module level", displayDict( vars() )
module_locals = locals()
# Patch away __file__ path in a hard to detect way. This will make sure, repeated calls to
# locals really get the same dictionary.
import os
module_locals[ "__file__" ] = os.path.basename( module_locals[ "__file__" ] )
del module_locals
print "Use of locals on the module level", displayDict( locals() )
def someFunctionUsingGlobals():
g = globals()
g[ "hallo" ] = "du"
global hallo
print "hallo", hallo
print "Testing globals():"
someFunctionUsingGlobals()
print "Testing dir():"
print "Module dir",
def someFunctionUsingDir():
x = someFunctionUsingGlobals()
print "Function dir", dir()
someFunctionUsingDir()
print "Making a new type, with type() and 3 args:",
new_class = type("Name", (object, ), {})
print new_class, new_class()
print "None has type", type(None)
print "Constant ranges", range( 2 ), range( 1, 6 ), range( 3, 0, -1 ), range( 3, 8, 2 ), range(5, -5, -3)
print "Border cases", range(0), range(-1), range( -1, 1 )
print "Corner case large negative value", range(-2**100)
print "Corner case with large start/end values in small range", range(2**100,2**100+2)
try:
print "Range with 0 step gives:",
print range( 3, 8, 0 )
except ValueError, e:
print e
try:
print "Range with float:",
print range(1.0)
except TypeError, e:
print "Gives exception:", e
try:
print "Empty range call",
print range()
except TypeError, e:
print "Gives exception:", e
print "List from iterable", list( "abc" ), list()
print "List from sequence", list( sequence = (0, 1, 2) )
print "Tuple from iterable", tuple( "cda" ), tuple()
print "Tuple from sequence", tuple( sequence = (0, 1, 2) )
print "Dictionary from iterable and keywords", dict( ( "ab", ( 1, 2 ) ), f = 1, g = 1 )
print "More constant dictionaries", {'two': 2, 'one': 1}, {}, dict()
g = {'two': 2, 'one': 1}
print "Variable dictionary", dict( g )
print "Found during optimization", dict( dict( {'le': 2, 'la': 1} ), fu = 3 ), dict( named = dict( {'le': 2, 'la': 1} ) )
print "Floats from constants", float( "3.0" ), float( x = 9.0 ), float()
print "Found during optimization", float( float( "3.2" ) ), float( x = float( 11.0 ) )
print "Strs from constants", str( "3.3" ), str( object = 9.1 ), str()
print "Found during optimization", str( float( "3.3" ) ), str( object = float( 12.0 ) )
print "Bools from constants", bool( "3.3" ), bool( x = 9.1 ), bool(0), bool()
print "Found during optimization", bool( float( "3.3" ) ), bool( x = float( 0.0 ) )
print "Ints from constants", int( "3" ), int( x = "9" ), int( "f", 16 ), int( x = "e", base = 16 ), int( "0101", base = 2 ), int(0), int()
print "Found during optimization", int( int( "3" ) ), int( x = int( 0.0 ) )
try:
print "Int with only base", int( base = 2 ),
except Exception as e:
print "Caused", repr(e)
else:
print "Worked"
print "Oct from constants", oct( 467 ), oct( 0 )
print "Found during optimization", oct( int( "3" ) )
print "Hex from constants", hex( 467 ), hex( 0 )
print "Found during optimization", hex( int( "3" ) )
print "Bin from constants", bin( 467 ), bin( 0 )
print "Found during optimization", bin( int( "3" ) )
try:
int( 1,2,3 )
except Exception, e:
print "Too many args gave", repr(e)
try:
int( y = 1 )
except Exception, e:
print "Wrong arg", repr(e)
f = 3
print "Unoptimized call of int", int( "0" * f, base = 16 )
d = { "x" : "12", "base" : 8 }
print "Dict call of int", int( **d )
try:
print chr()
except Exception, e:
print "Disallowed without args", repr(e)
try:
print ord()
except Exception, e:
print "Disallowed without args", repr(e)
try:
print ord( s = 1 )
except Exception, e:
print "Disallowed keyword args", repr(e)
try:
print ord( 1, 2 )
except Exception, e:
print "Too many plain args", repr(e)
try:
print ord( 1, s = 2 )
except Exception, e:
print "Too many args, some keywords", repr(e)
try:
print str( "1", offer = 2 )
except Exception, e:
print "Too many args, some keywords", repr(e)
# TODO: This is calls, not really builtins.
a = 2
print "Can optimize the star list argness away", int(*(a,)),
print "Can optimize the empty star list arg away", int(*tuple()),
print "Can optimize the empty star dict arg away", long(**dict())
print "Dict building with keyword arguments", dict(), dict( a = f )
print "Dictionary entirely from constant args", dict(q='Guido', w='van', e='Rossum', r='invented', t='Python', y='')
a = 5
print "Instance check recognises", isinstance( a, int )
try:
print "Instance check with too many arguments", isinstance( a, long, int )
except Exception, e:
print "Too many args", repr(e)
try:
print "Instance check with too many arguments", isinstance( a )
except Exception, e:
print "Too few args", repr(e)
def usingIterToCheckIterable(a):
try:
iter(a)
except TypeError:
print "not iterable"
else:
print "ok"
usingIterToCheckIterable(1)
print "Nested constant, dict inside a list, referencing a built-in compile time constant",
print( [dict(type=int)] )
print "nan and -nan sign checks:"
from math import copysign
print copysign(1.0, float('nan'))
print copysign(1.0, float('-nan'))
print "Using != to detect nan floats:"
a = float("nan")
if a != a:
print("is nan")
else:
print("isn't nan")
class CustomStr(str): pass
class CustomBytes(bytes): pass
class CustomByteArray(bytearray): pass
values = [
b'100',
bytearray(b'100'),
CustomStr('100'),
CustomBytes(b'100'),
CustomByteArray(b'100')
]
for x in values:
try:
print "int", repr(x), int(x), int(x,2)
except TypeError as e:
print "caught", repr(e)
try:
print "long", repr(x), long(x), long(x,2)
except TypeError as e:
print "caught", repr(e)
Nuitka-0.5.0.1/tests/basics/Classes.py 0000644 0001750 0001750 00000007302 12265264105 017722 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
class SimpleClass:
""" The class documentation."""
# TODO: Doesn't work with Python3, because we don't yet make our own dict visible.
# print locals()
print str( locals() ).replace( "'__locals__': {...}, ", "" )
class_var = 1
def __init__(self, init_parameter):
self.x = init_parameter
def normal_method(self, arg1, arg2):
self.arg1 = arg1
self.arg2 = arg2
return self.arg1, self.arg2, self.x, self
@staticmethod
def static_method():
return "something"
print "Simple class:", SimpleClass
print "Lives in", SimpleClass.__module__
print "Documentation", SimpleClass.__doc__
print "Instantiate simple class:", SimpleClass(14)
print "Call simple class normal method:", SimpleClass( 11 ).normal_method(1, 2)
print "Call simple class static method:", SimpleClass(11).static_method()
class MetaClass(type):
def __init__(cls, name, bases, dictionary):
print "MetaClass is called."
cls.addedin = 5
print MetaClass
class ComplexClass:
__metaclass__ = MetaClass
print ComplexClass, dir( ComplexClass )
print ComplexClass, hasattr( ComplexClass, "addedin" ) and ComplexClass.addedin
def function():
x = 1
class DynamicClass:
y = x
return DynamicClass
x = 2
print function(), function().y
def strangeClassBehaviour():
class StrangeClass(object):
count = 0
def __new__(cls):
print "__new__"
cls.count += 1
return object.__new__(cls)
def __del__(self):
print "__del__"
cls = self.__class__
cls.count -= 1
assert cls.count >= 0
x = StrangeClass()
return x.count
print "Strange class with __new__ and __del__ overloads", strangeClassBehaviour()
class ClosureLocalizer:
function = function
def deco(f):
return f
@deco
def x():
pass
print "Class with a name from module level renamed to local", ClosureLocalizer.function
print "Class with decorator"
def classdecorator(cls):
print "cls decorator", cls.addedin
return cls
@classdecorator
class MyClass:
__metaclass__ = MetaClass
print "Class that updates its locals:",
class DictUpdating:
a = 1
locals().update( { "b" : 2 } )
for f in range(6):
locals()[ "test_%s" % f ] = f
print "Changed values", DictUpdating.b, DictUpdating.test_4
def functionThatOffersClosureToPassThroughClass(x):
class Foo:
global x
x = 1
def __call__(self, y):
return x + y
return Foo()
print functionThatOffersClosureToPassThroughClass(6)(2),
print x
class NameCollisionClosure:
def x(self):
return x
print NameCollisionClosure, NameCollisionClosure().x()
class ClassesWithNestedClass:
class NestedClass(object):
def getDict(self):
return { 'a':2 }
print ClassesWithNestedClass, ClassesWithNestedClass().NestedClass, ClassesWithNestedClass().NestedClass().getDict()
Nuitka-0.5.0.1/tests/basics/YieldFrom33.py 0000644 0001750 0001750 00000006265 12265264105 020374 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def g():
for a in range(3):
yield a
return 7
def h():
yield 4
yield 5
def f():
print( "Yielded from returner", (yield from g()) )
print( "Yielded from non-return value", (yield from h()) )
print( "Result", list( f() ) )
print( "Yielder with return value", list(g()) )
# This will raise when looking up any attribute.
class Broken:
def __iter__(self):
return self
def __next__(self):
return 1
def __getattr__(self, attr):
1/0
def test_broken_getattr_handling():
def g():
yield from Broken()
print( "Next with send: ", end = "" )
try:
gi = g()
next(gi)
gi.send(1)
except Exception as e:
print( "Caught", repr(e) )
print( "Next with throw: ", end = "" )
try:
gi = g()
next(gi)
gi.throw(AttributeError)
except Exception as e:
print( "Caught", repr(e) )
print( "Next with close: ", end = "" )
try:
gi = g()
next(gi)
gi.close()
print( "All good" )
except Exception as e:
print( "Caught", repr(e) )
test_broken_getattr_handling()
def test_throw_catched_subgenerator_handling():
def g1():
try:
print("Starting g1")
yield "g1 ham"
yield from g2()
yield "g1 eggs"
finally:
print("Finishing g1")
def g2():
try:
print("Starting g2")
yield "g2 spam"
yield "g2 more spam"
except LunchError:
print("Caught LunchError in g2")
yield "g2 lunch saved"
yield "g2 yet more spam"
class LunchError(Exception):
pass
g = g1()
for i in range(2):
x = next(g)
print("Yielded %s" % (x,))
e = LunchError("tomato ejected")
print( "Throw returned", g.throw(e) )
print( "Sub thrown" )
for x in g:
print("Yielded %s" % (x,))
test_throw_catched_subgenerator_handling()
def give_cpython_generator():
# TODO: This relies on eval not being inlined, which will become untrue.
return eval( "( x for x in range(3) )" )
def gen_compiled():
yield from give_cpython_generator()
yield from range(7)
print( list( gen_compiled() ) )
Nuitka-0.5.0.1/tests/basics/InplaceOperations.py 0000644 0001750 0001750 00000002313 12265264105 021741 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
x = 1
x += 2
print "Plain inplace:", x
z = [ 1, 2, 3 ]
z[1] += 5
print "List inplace:", z[1]
h = { "a" : 3 }
h["a"] += 2
print "Dict inplace:", h["a"]
class B:
a = 1
B.a += 2
print "Class attribute inplace:", B.a
h = [ 1, 2, 3, 4 ]
h[1:2] += (2,3)
print "List sclice inplace [x:y]", h
h[:1] += (9,9)
print "List sclice inplace [:y]", h
h[2:] += (6,6)
print "List sclice inplace [y:]", h
h[:] += (5,5,5)
print "List sclice inplace [:]", h
Nuitka-0.5.0.1/tests/basics/Looping.py 0000644 0001750 0001750 00000006060 12265264105 017734 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def cond():
return False
def loopingFunction(a = 1*2):
c = []
f = [ c, a ]
for a in range(6 or 8):
for b in range(8):
if a == b:
c.append( (a,b,True) )
elif a < b:
c.append( (b,a,False) )
else:
c.append( (a,b,False) )
if a != b:
z = 1
else:
z = 0
if z == 0:
continue
if z == 1 and b == 6:
break
if a == b:
z = 0
print c
f = 1
while f < (10 or 8):
m = 1
f += 1
print "m=", m
x = [u for u in range(8)]
x = [(u,v) for (u,v) in zip(range(8),reversed(range(8))) ]
print x
x = [(u if u%2==0 else 0) for u in range(10)]
print x
x = [(u if u%2==0 else 0) for u in (a if cond() else range(9))]
print x
y = [ [ 3+ (l if l else -1) for l in [m,m+1] ] for m in [f for f in range(2)] ]
print "f=", f
print "y=", y
if x:
l = "YES"
else:
l = "NO"
if x:
l = "yes"
else:
if True:
l = "no"
print "Triple and chain"
if m and l and f:
print "OK"
print "Triple or chain"
if m or l or f:
print "Okey"
print "Nested if not chain"
if not m:
if not l:
print "ok"
print "Braced if not chain with 'or'"
if not (m or l):
print "oki"
print "Braced if not chain with 'and'"
if not (m and l):
print "oki"
d=1
print "Nested if chain with outer else"
if a:
if b or c:
if d:
print "inside nest"
else:
print "outer else"
print x
while False:
pass
else:
print "Executed else branch for False condition while loop"
while True:
break
else:
print "Executed else branch for True condition while loop"
for x in range( 7 ):
pass
else:
print "Executed else branch for no break for loop"
for x in range( 7 ):
break
else:
print "Executed else branch despite break in for loop"
x = iter( range(5) )
while next( x ):
pass
else:
print "Executed else branch of while loop without break"
loopingFunction()
Nuitka-0.5.0.1/tests/basics/Branching.py 0000644 0001750 0001750 00000006745 12265264105 020232 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Some random branching to cover most cases """
def branchingFunction(a, b, c):
print "branchingFunction:", a, b, c
print "a or b", a or b
print "a and b", a and b
print "not a", not a
print "not b", not b
print "Simple branch with both branches"
if a:
l = "YES"
else:
l = "NO"
print a, "->", l
print "Simple not branch with both branches"
if not a:
l = "YES"
else:
l = "NO"
print not a, "->", l
print "Simple branch with a nested branch in else path"
if a:
m = "yes"
else:
if True:
m = "no"
print a, "->", m
print "Triple and chain"
v = "NO"
if a and b and c:
v = "YES"
print a, b, c, "->", v
print "Triple or chain"
k = "NO"
if a or b or c:
k = "YES"
print a, b, c, "->", k
print "Nested if not chain"
p = "NO"
if not a:
if not b:
p = "YES"
print "not a, not b", not a, not b, "->", p
print "or condition in braces:"
q = "NO"
if (a or b):
q = "YES"
print "(a or b) ->", q
print "Braced if not with two 'or'"
if not (a or b or c):
q = "YES"
else:
q = "NO"
print "not (a or b or c)", q
print "Braced if not with one 'or'"
q = "NO"
if not (b or b):
q = "YES"
print "not (b or b)", q
print "Expression a or b", a or b
print "Expression not(a or b)", not(a or b)
print "Expression a and (b+5)", a and (b+5)
print "Expression (b if b else 2)", (b if b else 2)
print "Expression (a and (b if b else 2))", (a and (b if b else 2))
print "Braced if not chain with 'and' and conditional expression"
if not (a and (b if b else 2)):
print "oki"
print "Nested if chain with outer else"
d=1
if a:
if b or c:
if d:
print "inside nest"
else:
print "outer else"
print "Complex conditional expression"
v = (3 if a-1 else 0) or \
(b or (c*2 if c else 6) if b-1 else a and b and c)
print v
if True:
print "Predictable branch taken"
branchingFunction(1,0,3)
x = 3
def optimizationVictim():
if x:
pass
else:
pass
if x:
pass
pass
optimizationVictim()
def dontOptimizeSideEffects():
print "Lets see, if conditional expression in known true values are correctly handled",
def returnTrue():
print "'returnTrue' should be called",
return True
def returnFalse():
print "'returnFalse' should not be called",
return False
if ( returnTrue() or returnFalse(), ):
print "Taken branch as expected"
else:
print "Bad2"
dontOptimizeSideEffects()
Nuitka-0.5.0.1/tests/basics/Referencing.py 0000644 0001750 0001750 00000031415 12265264105 020556 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys, gc
if not hasattr( sys, "gettotalrefcount" ):
print( "Warning, using non-debug Python makes this test ineffective." )
sys.gettotalrefcount = lambda : 0
gc.disable()
def simpleFunction1():
return 1
def simpleFunction2():
y = 3 * x
y = 3
y = 2
return x*2
def simpleFunction3():
def contained():
return x
return contained
def simpleFunction4():
y = 1
def contained():
return y
return contained
def simpleFunction5(a = 1*2):
c = 1
f = [ a, a + c ]
def simpleFunction6():
for b in range(6):
pass
for c in (1, 2, 3, 4, 5, 6):
pass
def simpleFunction7(b = 1):
for b in range(6):
pass
def simpleFunction8():
c = []
c.append( x )
def simpleFunction9(a = 1*2):
if a == a:
pass
u = None
def simpleFunction10(a = 1*2):
x = [u for u in range(8)]
def simpleFunction11():
f = 1
while f < 8:
f += 1
v = None
def simpleFunction12():
a = [ (u,v) for (u,v) in zip(range(8),range(8)) ]
def cond():
return 1
def simpleFunction13(a = 1*2):
pass
def simpleFunction14p(x):
try:
simpleFunction14p(1,1)
except TypeError, e:
pass
try:
simpleFunction14p(1,1)
except TypeError:
pass
def simpleFunction14():
simpleFunction14p( 3 )
def simpleFunction15p(x):
try:
try:
x += 1
finally:
try:
x *= 1
finally:
z = 1
except:
pass
def simpleFunction15():
simpleFunction15p( [ 1 ] )
def simpleFunction16():
class EmptyClass:
pass
return EmptyClass
def simpleFunction17():
class EmptyObjectClass:
pass
return EmptyObjectClass()
def simpleFunction18():
closured = 1
class NonEmptyClass:
def __init__(self, a, b):
self.a = a
self.b = b
inside = closured
return NonEmptyClass( 133, 135 )
def simpleFunction19():
lam = lambda l : l+1
return lam( 9 ), lam
def simpleFunction20():
try:
a = []
a[1]
except IndexError, e:
pass
def simpleFunction21():
class EmptyBaseClass:
def base(self):
return 3
class EmptyObjectClass(EmptyBaseClass):
pass
result = EmptyObjectClass()
c = result.base()
return result
def simpleFunction22():
return True is False and False is not None
def simpleFunction23():
not 2
def simpleFunction24p(x):
pass
def simpleFunction24():
simpleFunction24p( x = 3 )
def simpleFunction25():
class X:
f = 1
def inplace_adder(b):
X.f += b
return inplace_adder( 6**8 )
def simpleFunction26():
class X:
f = [ 5 ]
def inplace_adder(b):
X.f += b
return inplace_adder( [ 1, 2 ] )
def simpleFunction27():
a = { "g": 8 }
def inplace_adder(b):
a[ "g" ] += b
return inplace_adder( 3 )
def simpleFunction28():
a = { "g": [ 8 ], "h": 2 }
def inplace_adder(b):
a[ "g" ] += b
return inplace_adder( [ 3, 5 ] )
def simpleFunction29():
return "3" in "7"
def simpleFunction30():
def generatorFunction():
yield 1
yield 2
yield 3
def simpleFunction31():
def generatorFunction():
yield 1
yield 2
yield 3
a = []
for y in generatorFunction():
a.append( y )
for z in generatorFunction():
a.append( z )
def simpleFunction32():
def generatorFunction():
yield 1
gen = generatorFunction()
gen.next()
def simpleFunction33():
def generatorFunction():
a = 1
yield a
a = []
for y in generatorFunction():
a.append( y )
def simpleFunction34():
try:
raise ValueError
except:
pass
def simpleFunction35():
try:
raise ValueError(1,2,3)
except:
pass
def simpleFunction36():
try:
raise TypeError, (3,x,x,x)
except TypeError:
pass
def simpleFunction37():
l = [ 1, 2, 3 ]
try:
a, b = l
except ValueError:
pass
def simpleFunction38():
class Base:
pass
class Parent(Base):
pass
def simpleFunction39():
class Parent(object):
pass
def simpleFunction40():
def myGenerator():
yield 1
myGenerator()
def simpleFunction41():
a = b = 2
def simpleFunction42():
a = b = 2 * x
def simpleFunction43():
class D:
pass
a = D()
a.b = 1
def simpleFunction44():
def nested_args_function((a,b), c):
return a, b, c
nested_args_function( ( 1, 2 ), 3 )
def simpleFunction45():
def nested_args_function((a,b), c):
return a, b, c
try:
nested_args_function( ( 1, ), 3 )
except ValueError:
pass
def simpleFunction46():
def nested_args_function((a,b), c):
return a, b, c
try:
nested_args_function( ( 1, 2, 3 ), 3 )
except ValueError:
pass
def simpleFunction47():
def reraisy():
def raisingFunction():
raise ValueError(3)
def reraiser():
raise
try:
raisingFunction()
except:
reraiser()
try:
reraisy()
except:
pass
def simpleFunction48():
class BlockExceptions:
def __enter__(self):
pass
def __exit__( self, exc, val, tb):
return True
with BlockExceptions():
raise ValueError()
template = "lala %s lala"
def simpleFunction49():
c = 3
d = 4
a = x, y = b,e = (c,d)
b = range(10)
def simpleFunction50():
def getF():
def f():
for i in b:
yield i
return f
f = getF()
for x in range( 2 ):
r = list( f() )
def simpleFunction51():
g = ( x for x in range(9) )
try:
g.throw( ValueError, 9 )
except ValueError, e:
pass
def simpleFunction52():
g = ( x for x in range(9) )
try:
g.throw( ValueError( 9 ) )
except ValueError, e:
pass
def simpleFunction53():
g = ( x for x in range(9) )
try:
g.send( 9 )
except TypeError, e:
pass
def simpleFunction54():
g = ( x for x in range(9) )
g.next()
try:
g.send( 9 )
except TypeError, e:
pass
def simpleFunction55():
g = ( x for x in range(9) )
try:
g.close()
except ValueError, e:
pass
def simpleFunction56():
def f():
f()
try:
f()
except RuntimeError:
pass
def simpleFunction57():
x = 1
y = 2
def f( a = x, b = y):
return a, b
f()
f(2)
f(3,4)
def simpleFunction58():
a = 3
b = 5
try:
a = a * 2
return a
finally:
a / b
def simpleFunction59():
a = 3
b = 5
try:
a = a * 2
return a
finally:
return a / b
def simpleFunction60():
try:
raise ValueError(1,2,3), ValueError(1,2,3)
except Exception:
pass
def simpleFunction61():
try:
raise ValueError, 2, None
except Exception:
pass
def simpleFunction62():
try:
raise ValueError, 2, 3
except Exception:
pass
class X:
def __del__(self):
# Super used to reference leak.
x = super()
raise ValueError, ValueError(1)
def simpleFunction63():
def superUser():
X()
try:
superUser()
except Exception:
pass
def simpleFunction64():
x = 2
y = 3
z = eval( "x * y" )
def simpleFunction65():
import array
a = array.array("b", "")
assert a == eval(repr(a), {"array": array.array})
d = {
"x" : 2,
"y" : 3
}
z = eval( repr(d), d )
def simpleFunction66():
import types
return type(simpleFunction65) == types.FunctionType
x = 17
m1 = {}
m2 = {}
def snapObjRefCntMap(before):
if before:
global m1
m = m1
else:
global m2
m = m2
for x in gc.get_objects():
if x is m1:
continue
if x is m2:
continue
m[ str( x ) ] = sys.getrefcount( x )
def checkReferenceCount(checked_function, max_rounds = 10):
assert sys.exc_info() == ( None, None, None ), sys.exc_info()
print checked_function.func_name + ":",
ref_count1 = 17
ref_count2 = 17
explain = False
for count in range( max_rounds ):
x1 = 0
x2 = 0
gc.collect()
ref_count1 = sys.gettotalrefcount()
if explain and count == max_rounds - 1:
snapObjRefCntMap( True )
checked_function()
assert sys.exc_info() == ( None, None, None ), sys.exc_info()
gc.collect()
if explain and count == max_rounds - 1:
snapObjRefCntMap( False )
ref_count2 = sys.gettotalrefcount()
if ref_count1 == ref_count2:
print "PASSED"
break
# print count, ref_count1, ref_count2
else:
print "FAILED", ref_count1, ref_count2, "leaked", ref_count2 - ref_count1
if explain:
assert m1
assert m2
for key in m1.keys():
if key not in m2:
print "*" * 80
print key
elif m1[key] != m2[key]:
print "*" * 80
print key
else:
pass
# print m1[key]
assert sys.exc_info() == ( None, None, None ), sys.exc_info()
gc.collect()
checkReferenceCount( simpleFunction1 )
checkReferenceCount( simpleFunction2 )
checkReferenceCount( simpleFunction3 )
checkReferenceCount( simpleFunction4 )
checkReferenceCount( simpleFunction5 )
checkReferenceCount( simpleFunction6 )
checkReferenceCount( simpleFunction7 )
checkReferenceCount( simpleFunction8 )
checkReferenceCount( simpleFunction9 )
checkReferenceCount( simpleFunction10 )
checkReferenceCount( simpleFunction11 )
checkReferenceCount( simpleFunction12 )
checkReferenceCount( simpleFunction13 )
checkReferenceCount( simpleFunction14 )
checkReferenceCount( simpleFunction15 )
checkReferenceCount( simpleFunction16 )
checkReferenceCount( simpleFunction17 )
checkReferenceCount( simpleFunction18 )
checkReferenceCount( simpleFunction19 )
checkReferenceCount( simpleFunction20 )
checkReferenceCount( simpleFunction21 )
checkReferenceCount( simpleFunction22 )
checkReferenceCount( simpleFunction23 )
checkReferenceCount( simpleFunction24 )
checkReferenceCount( simpleFunction25 )
checkReferenceCount( simpleFunction26 )
checkReferenceCount( simpleFunction27 )
checkReferenceCount( simpleFunction28 )
checkReferenceCount( simpleFunction29 )
checkReferenceCount( simpleFunction30 )
checkReferenceCount( simpleFunction31 )
checkReferenceCount( simpleFunction32 )
checkReferenceCount( simpleFunction33 )
checkReferenceCount( simpleFunction34 )
checkReferenceCount( simpleFunction35 )
checkReferenceCount( simpleFunction36 )
checkReferenceCount( simpleFunction37 )
checkReferenceCount( simpleFunction38 )
checkReferenceCount( simpleFunction39 )
checkReferenceCount( simpleFunction40 )
checkReferenceCount( simpleFunction41 )
checkReferenceCount( simpleFunction42 )
checkReferenceCount( simpleFunction43 )
checkReferenceCount( simpleFunction44 )
checkReferenceCount( simpleFunction45 )
checkReferenceCount( simpleFunction46 )
checkReferenceCount( simpleFunction47 )
checkReferenceCount( simpleFunction48 )
checkReferenceCount( simpleFunction49 )
checkReferenceCount( simpleFunction50 )
checkReferenceCount( simpleFunction51 )
checkReferenceCount( simpleFunction52 )
checkReferenceCount( simpleFunction53 )
checkReferenceCount( simpleFunction54 )
checkReferenceCount( simpleFunction55 )
# TODO: The function taking a closure of itself, causes a reference leak, that
# we accept for now.
# checkReferenceCount( simpleFunction56 )
checkReferenceCount( simpleFunction57 )
checkReferenceCount( simpleFunction58 )
checkReferenceCount( simpleFunction59 )
checkReferenceCount( simpleFunction60 )
checkReferenceCount( simpleFunction61 )
checkReferenceCount( simpleFunction62 )
# Avoid unraisable output.
old_stderr = sys.stderr
try:
sys.stderr = open( "/dev/null", "wb" )
except Exception: # Windows
checkReferenceCount( simpleFunction63 )
else:
checkReferenceCount( simpleFunction63 )
new_stderr = sys.stderr
sys.stderr = old_stderr
new_stderr.close()
checkReferenceCount( simpleFunction64 )
checkReferenceCount( simpleFunction65 )
checkReferenceCount( simpleFunction66 )
Nuitka-0.5.0.1/tests/basics/PrintFuture.py 0000644 0001750 0001750 00000001544 12265264105 020616 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import print_function
print( "hallo welt", end = "," )
print( "this is the end" )
Nuitka-0.5.0.1/tests/basics/GlobalStatement.py 0000644 0001750 0001750 00000007124 12265264105 021414 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys
x = 2
def someFunction1():
x = 3
return x
def someFunction2():
global x
x = 4
return x
def someFunction3():
return x
def someNestedGlobalUser1():
z = 1
# Nested function that uses a global z doesn't affect the local variable z at all.
def setZ():
global z
z = 3
setZ()
return z
def someNestedGlobalUser2():
z = 1
# Nested function that uses a global z doesn't affect the local variable z at
# all. This doesn't change if it's done inside an exec block.
exec """
def setZ():
global z
z = 3
setZ()
"""
return z
def someNestedGlobalUser3a():
# Nested function that uses a exec variable scope z and a global z, changes z to be
# the global one only. We verify that by looking at locals. This means that the global
# statement inside the function of exec changes the effect of the z.
exec """
z = 1
def setZ():
global z
z = 3
setZ()
"""
return z, locals().keys() == [ "setZ" ]
def someNestedGlobalUser3b():
# Nested function that uses a exec variable scope z and a global z, changes
# z to be the global one only. We verify that by looking at locals.
exec """
z = 1
"""
if sys.version_info[0] < 3:
return z, locals().keys() == [ "z" ]
else:
return locals().keys() == []
def someNestedGlobalUser4():
z = 1
# This one proves that the local variable z is entirely ignored, and that the global z
# has the value 2 inside setZ().
exec """
z = 2
def setZ():
global z
z = 3*z
setZ()
"""
return z
def someNestedGlobalUser5():
z = 1
# Without a global statement, z affects the local variable z.
exec """
z = 3
"""
return z
def someNestedGlobalUser6():
# Without a global statement, a local variable z is created.
exec """
z = 7
"""
return z
print "Function that shadows a global variable with a local variable"
print someFunction1()
print "Function that accesses and changes a global variable declared with a global statement"
print someFunction2()
print "Function that uses a global variable"
print someFunction3()
print "Functions that uses a global variable in a nested function in various ways:"
print someNestedGlobalUser1, someNestedGlobalUser1()
del z
print someNestedGlobalUser2, someNestedGlobalUser2()
del z
print someNestedGlobalUser3a, someNestedGlobalUser3a()
del z
print someNestedGlobalUser3b, someNestedGlobalUser3b()
print someNestedGlobalUser4, ( someNestedGlobalUser4(), z )
del z
print someNestedGlobalUser5, someNestedGlobalUser5()
z = 9
print someNestedGlobalUser6, ( someNestedGlobalUser6(), z )
x = 7
def f():
x = 1
def g():
global x
def i():
def h():
return x
return h()
return i()
return g()
print f()
global global_already
global_already = 1
Nuitka-0.5.0.1/tests/basics/OrderChecks.py 0000644 0001750 0001750 00000022755 12265264105 020532 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def dictOrderCheck():
def key1():
print "key1 called"
return 1
def key2():
print "key2 called"
return 2
def value1():
print "value1 called"
return 11
def value2():
print "value2 called"
return 22
print "Checking order of calls in dictionary creation from callables:"
print { key1() : value1(), key2() : value2() }
def listOrderCheck():
def value1():
print "value1 called"
return 11
def value2():
print "value2 called"
return 22
print [ value1(), value2() ]
def sliceOrderCheck():
d = range(10)
def lvalue():
print "lvalue",
return d
def rvalue():
print "rvalue",
return range(2)
def rvalue4():
print "rvalue",
return range(4)
def low():
print "low",
return 0
def high():
print "high",
return 4
def step():
print "step",
return 2
print "Complex slice lookup:", lvalue()[ low() : high() : step() ]
print "Complex slice assignment:",
lvalue()[ low() : high() : step() ] = rvalue()
print d
print "Complex slice del:",
del lvalue()[ low() : high() : step() ]
print d
print "Complex inplace slice operation",
# TODO: This gives an error in CPython, but not in Nuitka.
# lvalue()[ low() : high() : step() ] += rvalue()
print d
d = range(10)
print "Simple slice lookup", lvalue()[ low() : high() ]
print "Simple slice assignment",
lvalue()[ 3 + low() : 3 + high() ] = rvalue()
print d
print "Simple slice del",
del lvalue()[ 3 + low() : 3 + high() ]
print d
print "Simple inplace slice operation",
lvalue()[ low() : high() ] += rvalue4()
print d
def subscriptOrderCheck():
d={}
def lvalue():
print "lvalue",
return d
def rvalue():
print "rvalue",
return 2
def subscript():
print "subscript",
return 1
print "Assigning subscript:"
lvalue()[ subscript() ] = rvalue()
print d
print "Lookup subscript:"
print lvalue()[ subscript() ]
print "Deleting subscript:"
del lvalue()[ subscript() ]
print d
def attributeOrderCheck():
def lvalue():
print "lvalue",
return lvalue
def rvalue():
print "rvalue",
return 2
print "Attribute assigment order:"
lvalue().xxx = rvalue()
print lvalue.xxx
try:
zzz.xxx = yyy
except Exception as e:
print "Caught", repr(e)
def compareOrderCheck():
def lvalue():
print "lvalue",
return 1
def rvalue():
print "rvalue",
return 2
print "Comparisons:"
print "==", lvalue() == rvalue()
print "<=", lvalue() <= rvalue()
print ">=", lvalue() >= rvalue()
print "!=", lvalue() != rvalue()
print ">", lvalue() > rvalue()
print "<", lvalue() < rvalue()
print "Comparison used in bool context:"
print "==", "yes" if lvalue() == rvalue() else "no"
print "<=", "yes" if lvalue() <= rvalue() else "no"
print ">=", "yes" if lvalue() >= rvalue() else "no"
print "!=", "yes" if lvalue() != rvalue() else "no"
print ">", "yes" if lvalue() > rvalue() else "no"
print "<", "yes" if lvalue() < rvalue() else "no"
def operatorOrderCheck():
def left():
print "left",
return 1
def middle():
print "middle",
return 3
def right():
print "right",
return 2
print "Operations:"
print "+", left() + middle() + right()
print "-", left() - middle() - right()
print "*", left() * middle() * right()
print "/", left() / middle() / right()
print "%", left() % middle() % right()
print "**", left() ** middle() ** right()
def generatorOrderCheck():
def default1():
print "default1",
return 1
def default2():
print "default2",
return 2
def default3():
print "default3",
return 3
def generator(a = default1(), b = default2(), c = default3()):
yield a
yield b
yield c
print list( generator() )
def classOrderCheck():
print "Checking order of class constructions:"
class B1:
pass
class B2:
pass
def base1():
print "base1",
return B1
def base2():
print "base2",
return B2
def deco1(cls):
print "deco1",
return cls
def deco2(cls):
print "deco2",
return B2
@deco2
@deco1
class X(base1(), base2()):
print "class body",
print
def inOrderCheck():
print "Checking order of in operator:"
def container():
print "container",
return [ 3 ]
def searched():
print "searched",
return 3
print searched() in container()
print searched() not in container()
def unpackOrderCheck():
class Iterable:
def __init__(self):
self.consumed = 2
def __iter__(self):
return Iterable()
def __del__(self):
print "Deleted with", self.consumed
def next(self):
print "Next with", self.consumed
if self.consumed:
self.consumed -=1
else:
raise StopIteration
return self.consumed
iterable = Iterable()
try:
x, y = a, b = Iterable()
except Exception as e:
print "Caught", repr(e)
def superOrderCheck():
try:
super( zzz, xxx )
except Exception as e:
print "Caught super 2", repr(e)
def isinstanceOrderCheck():
try:
isinstance( zzz, xxx )
except Exception as e:
print "Caught isinstance 2", repr(e)
def rangeOrderCheck():
try:
range( zzz, yyy, xxx )
except Exception as e:
print "Caught range 3", repr(e)
try:
range( zzz, xxx )
except Exception as e:
print "Caught range 2", repr(e)
def importOrderCheck():
def name():
print "name",
def globals():
print "globals",
def locals():
print "locals",
def fromlist():
print "fromlist",
def level():
print "level",
try:
print "__import__ builtin:"
__import__( name(), globals(), locals(), fromlist(), level() )
except Exception as e:
print "Caught __import__", repr(e)
def hasattrOrderCheck():
try:
hasattr( zzz, yyy )
except Exception as e:
print "Caught hasattr", repr(e)
def getattrOrderCheck():
try:
getattr( zzz, yyy )
except Exception as e:
print "Caught getattr 2", repr(e)
try:
getattr( zzz, yyy, xxx )
except Exception as e:
print "Caught getattr 3", repr(e)
def typeOrderCheck():
try:
type( zzz, yyy, xxx )
except Exception as e:
print "Caught type 3", repr(e)
def iterOrderCheck():
try:
iter( zzz, xxx )
except Exception as e:
print "Caught iter 2", repr(e)
def openOrderCheck():
try:
open( zzz, yyy, xxx )
except Exception as e:
print "Caught open 3", repr(e)
def unicodeOrderCheck():
try:
unicode( zzz, yyy, xxx )
except Exception as e:
print "Caught unicode", repr(e)
def longOrderCheck():
try:
long( zzz, xxx )
except Exception as e:
print "Caught long 2", repr(e)
def intOrderCheck():
try:
int( zzz, xxx )
except Exception as e:
print "Caught int", repr(e)
def nextOrderCheck():
try:
next( zzz, xxx )
except Exception as e:
print "Caught next 2", repr(e)
def raiseOrderCheck():
print "Checking order of raises:"
def exception_type():
print "exception_type",
return ValueError
def exception_value():
print "exception_value",
return 1
def exception_tb():
print "exception_value",
return None
print "3 args",
try:
raise exception_type(), exception_value(), exception_tb()
except Exception as e:
print "caught", repr(e)
print "2 args",
try:
raise exception_type(), exception_value()
except Exception as e:
print "caught", repr(e)
print "1 args",
try:
raise exception_type()
except Exception as e:
print "caught", repr(e)
dictOrderCheck()
listOrderCheck()
subscriptOrderCheck()
attributeOrderCheck()
operatorOrderCheck()
compareOrderCheck()
sliceOrderCheck()
generatorOrderCheck()
classOrderCheck()
inOrderCheck()
unpackOrderCheck()
superOrderCheck()
isinstanceOrderCheck()
rangeOrderCheck()
importOrderCheck()
hasattrOrderCheck()
getattrOrderCheck()
typeOrderCheck()
iterOrderCheck()
openOrderCheck()
unicodeOrderCheck()
nextOrderCheck()
longOrderCheck()
intOrderCheck()
raiseOrderCheck()
Nuitka-0.5.0.1/tests/basics/ListContractions.py 0000644 0001750 0001750 00000004455 12265264105 021635 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print "List contraction on the module level:"
x = [(u if u%2==0 else 0) for u in range(10)]
print x
print "List contraction on the function level:"
def someFunction():
x = [(u if u%2==0 else 0) for u in range(10)]
print x
someFunction()
print "List contractions with no, 1 one 2 conditions:"
def otherFunction():
print [ x for x in range(8) ]
print [ x for x in range(8) if x % 2 == 1 ]
print [ x for x in range(8) if x % 2 == 1 if x > 4 ]
otherFunction()
print "Complex list contractions with more than one for:"
def complexContractions():
print [ (x,y) for x in range(3) for y in range(5) ]
seq = range(3)
res = [(i, j, k) for i in iter(seq) for j in iter(seq) for k in iter(seq)]
print res
complexContractions()
print "Contraction for 2 fors and one final if refering to first for:"
def trickyContraction():
class Range:
def __init__(self, value):
self.value = value
def __iter__(self):
print "Giving range iter to", self.value
return iter( range( self.value ))
def Cond(y):
print "Checking against", y
return y == 1
r = [ (x,z,y) for x in Range(3) for z in Range(2) for y in Range(4) if Cond(y) ]
print "result", r
trickyContraction()
def lambdaWithcontraction(x):
l = lambda x : [ z for z in range(x) ]
r = l(x)
print locals()
lambdaWithcontraction( 3 )
print "Contraction that gets a del on the iterator variable:",
def allowedDelOnIteratorVariable(z):
x = 2
del x
return [ x*z for x in range(z) ]
print allowedDelOnIteratorVariable( 3 )
Nuitka-0.5.0.1/tests/basics/Inspection.py 0000644 0001750 0001750 00000005703 12265264105 020443 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import inspect, types, sys
def compiledFunction():
pass
assert inspect.isfunction( compiledFunction ) is True
assert isinstance( compiledFunction, types.FunctionType )
assert isinstance( compiledFunction, ( int, types.FunctionType ) )
# Even this works.
assert type( compiledFunction ) == types.FunctionType
class compiledClass:
def compiledMethod(self):
pass
assert inspect.isfunction( compiledClass ) is False
assert isinstance( compiledClass, types.FunctionType ) is False
assert inspect.ismethod( compiledFunction ) is False
assert inspect.ismethod( compiledClass ) is False
assert inspect.ismethod( compiledClass.compiledMethod ) == ( sys.version_info < ( 3, ) )
assert inspect.ismethod( compiledClass().compiledMethod ) is True
assert bool( type( compiledClass.compiledMethod ) == types.MethodType ) == ( sys.version_info < ( 3, ) )
def compiledGenerator():
yield 1
assert inspect.isfunction( compiledGenerator ) is True
assert inspect.isgeneratorfunction( compiledGenerator ) is True
assert isinstance( compiledGenerator(), types.GeneratorType ) is True
assert type( compiledGenerator() ) == types.GeneratorType
assert isinstance( compiledGenerator, types.GeneratorType ) is False
assert inspect.ismethod( compiledGenerator() ) is False
assert inspect.isfunction( compiledGenerator() ) is False
assert inspect.isgenerator( compiledFunction ) is False
assert inspect.isgenerator( compiledGenerator ) is False
assert inspect.isgenerator( compiledGenerator() ) is True
def someFunction():
assert inspect.isframe( sys._getframe() )
print inspect.getframeinfo( sys._getframe() )
someFunction()
import sys
class C:
print "Class locals", str( sys._getframe().f_locals ).replace( ", '__locals__': {...}", "" ).replace( "'__qualname__': 'C', ", "" )
print "Class flags", sys._getframe().f_code.co_flags | 64
def f():
print "Func locals", sys._getframe().f_locals
print "Func flags", sys._getframe().f_code.co_flags | 64
f()
def displayDict(d):
d = dict(d)
if "__loader__" in d:
d[ "__loader__" ] = ""
return repr( d )
print "Module frame locals", displayDict( sys._getframe().f_locals )
print "Module flags", sys._getframe().f_code.co_flags | 64
Nuitka-0.5.0.1/tests/basics/Recursion.py 0000644 0001750 0001750 00000001550 12265264105 020275 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
count = 0
def recurse():
global count
count += 1
if count < 50:
recurse()
recurse()
Nuitka-0.5.0.1/tests/basics/ParameterErrors32.py 0000644 0001750 0001750 00000003605 12265264105 021611 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def kwfunc(a, *, k):
pass
print( "Call function with mixed arguments with too wrong keyword argument." )
try:
kwfunc( k = 3, b = 5 )
except TypeError as e:
print( repr(e) )
print( "Call function with mixed arguments with too little positional arguments." )
try:
kwfunc( k = 3 )
except TypeError as e:
print( repr(e) )
print( "Call function with mixed arguments with too little position arguments." )
try:
kwfunc( 3 )
except TypeError as e:
print( repr(e) )
print( "Call function with mixed arguments with too many position arguments." )
try:
kwfunc( 1,2,k=3 )
except TypeError as e:
print( repr(e) )
def kwfuncdefaulted(a, b = None, *, c = None):
pass
print( "Call function with mixed arguments and defaults but too many position arguments." )
try:
kwfuncdefaulted(1, 2, 3)
except TypeError as e:
print( repr(e) )
def kwfunc2(a, *, k, l, m):
pass
print( "Call function with mixed arguments with too little positional and keyword-only arguments." )
try:
kwfunc2( 1, l = 2 )
except TypeError as e:
print( repr(e) )
try:
kwfunc2( 1 )
except TypeError as e:
print( repr(e) )
Nuitka-0.5.0.1/tests/basics/MainPrograms.py 0000644 0001750 0001750 00000002370 12265264105 020724 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print "Module name is", __name__
class SomeClass:
pass
print "Class inside names it as", repr( SomeClass.__module__ )
if __name__ == "__main__":
print "Executed as __main__"
import sys, os
# The sys.argv[0] might contain .exe, .py or no suffix at all. Remove it, so the diff
# is more acceptable.
args = sys.argv[:]
args[0] = os.path.basename( args[0] ).replace( ".exe", ".py" ).replace( ".py", "" )
print "Arguments were", args
print "Flags are", sys.flags
Nuitka-0.5.0.1/tests/basics/Asserts.py 0000644 0001750 0001750 00000002537 12265264105 017756 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def testAssert1():
assert False
return 1
def testAssert2():
assert True
return 1
def testAssert3():
assert False, "argument"
return 1
try:
print "Function that will assert."
testAssert1()
print "No exception."
except Exception, e:
print "Raised", type(e), e
try:
print "Function that will not assert."
testAssert2()
print "No exception."
except Exception, e:
print "Raised", type(e), e
try:
print "Function that will assert with argument."
testAssert3()
print "No exception."
except Exception, e:
print "Raised", type(e), e
Nuitka-0.5.0.1/tests/basics/DefaultParameters.py 0000644 0001750 0001750 00000004107 12265264105 021735 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
module_level = 1
def defaultValueTest1(no_default, some_default_constant = 1):
return some_default_constant
def defaultValueTest2(no_default, some_default_computed = module_level*2):
local_var = no_default = "1"
return local_var, some_default_computed
def defaultValueTest3( no_default, func_defaulted = defaultValueTest1(module_level)):
return [ func_defaulted for i in range(8) ]
def defaultValueTest4( no_default, funced_defaulted = lambda x: x**2):
c = 1
d = 1
return ( i+c+d for i in range(8) )
def defaultValueTest5( no_default, tuple_defaulted = (1,2,3)):
return tuple_defaulted
def defaultValueTest6( no_default, list_defaulted = [1,2,3]):
list_defaulted.append(5)
return list_defaulted
print defaultValueTest1("ignored")
# The change of the default variable doesn't influence the default
# parameter of defaultValueTest2, that means it's also calculated
# at the time the function is defined.
module_level = 7
print defaultValueTest2("also ignored")
print defaultValueTest3("nono not again")
print list( defaultValueTest4("unused") )
print defaultValueTest5("unused")
print defaultValueTest6("unused"),
print defaultValueTest6("unused")
print defaultValueTest6.func_defaults
defaultValueTest6.func_defaults = ([1,2,3],)
print defaultValueTest6.func_defaults
print defaultValueTest6(1)
Nuitka-0.5.0.1/tests/basics/FunctionObjects.py 0000644 0001750 0001750 00000003112 12265264105 021417 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def func(arg1, arg2, arg3, **star):
""" Some documentation. """
pass
print "Starting out: func, func_name:", func, func.func_name
print "Changing its name:"
func.func_name = "renamed"
print "With new name: func, func_name:", func, func.func_name
print "Documentation initially:", func.__doc__
print "Changing its doc:"
func.__doc__ = "changed doc" + chr(0) + " with 0 character"
print "Documentation updated:", repr( func.__doc__ )
print "Setting its dict"
func.my_value = "attached value"
print "Reading its dict", func.my_value
print "func_code", func.func_code, func.func_code.co_argcount
print dir( func.func_code )
def func2(arg1, arg2 = "default_arg2", arg3 = "default_arg3"):
x = 1
return x
print "func_defaults", func2.__defaults__, func2.func_defaults
print "function varnames", func2.__code__.co_varnames
Nuitka-0.5.0.1/tests/basics/run_all.py 0000755 0001750 0001750 00000006704 12265264105 017771 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import os, sys
# Find common code relative in file system. Not using packages for test stuff.
sys.path.insert(
0,
os.path.normpath(
os.path.join(
os.path.dirname( os.path.abspath( __file__ ) ),
".."
)
)
)
from test_common import (
my_print,
setup,
convertUsing2to3,
decideFilenameVersionSkip,
compareWithCPython,
hasDebugPython
)
python_version = setup(needs_io_encoding = True)
search_mode = len( sys.argv ) > 1 and sys.argv[1] == "search"
start_at = sys.argv[2] if len( sys.argv ) > 2 else None
if start_at:
active = False
else:
active = True
# Create large constants test on the fly, if it's not there, not going to
# add it to release archives for no good reason.
if not os.path.exists( "BigConstants.py" ):
with open( "BigConstants.py", "w" ) as output:
output.write( "# Automatically generated test, not part of releases or git.\n\n" )
output.write( "print( '%s' )\n" % ( "1234" * 17000 ) )
# Now run all the tests in this directory.
for filename in sorted( os.listdir( "." ) ):
if not filename.endswith( ".py" ):
continue
if not decideFilenameVersionSkip( filename ):
continue
# The overflow functions test gives syntax error on Python 3.x and will be
# skiped as well.
if filename == "OverflowFunctions.py" and python_version.startswith( b"3" ):
continue
path = filename
if not active and start_at in ( filename, path ):
active = True
extra_flags = [ "expect_success", "remove_output" ]
# This test should be run with the debug Python, and makes outputs to
# standard error that might be ignored.
if filename.startswith( "Referencing" ):
extra_flags.append( "ignore_stderr" )
extra_flags.append( "python_debug" )
# This tests warns about __import__() used.
if filename == "OrderChecks.py":
extra_flags.append( "ignore_stderr" )
# TODO: Nuitka does not give output for ignored exception in dtor, this is
# not fully compatible and potentially in error.
if filename == "YieldFrom33.py":
extra_flags.append( "ignore_stderr" )
if active:
if filename.startswith( "Referencing" ) and not hasDebugPython():
my_print( "Skipped (no debug Python)" )
continue
needs_2to3 = python_version.startswith( b"3" ) and \
not filename.endswith( "32.py" ) and \
not filename.endswith( "33.py" )
compareWithCPython(
path = path,
extra_flags = extra_flags,
search_mode = search_mode,
needs_2to3 = needs_2to3
)
else:
my_print( "Skipping", filename )
Nuitka-0.5.0.1/tests/basics/Operators.py 0000644 0001750 0001750 00000003555 12265264105 020311 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
a = 3
b = 7
c = [ 7, 8 ]
d = 15
print "+", a + b
print "-", a - b
print "*", a * b
print "/", a / b
print "//", a // b
print "%", b % a
print "& (2)", a & b
print "| (2)", a | b
print "& (3)", a & b & d
print "| (3)", a | b | d
print "^ (2)", a ^ b
print "^ (3)", a ^ b ^ d
print "**", a ** b
print "<<", a << b
print ">>", b >> a
print "in", b in c
print "not in", b not in c
print "<", a < b
print ">", a > b
print "==", a == b
print "<=", a <= b
print ">=", a >= b
print "!=", a != b
print "is", a is b
print "is not", a is not b
print "~", ~ b
print "-", - b
print "+", + b
l = { ( "a", "c" ) : "a,c", "b" : 2, "c" : 3, "d" : 4 }
l[ "l", ] = "6"
print "Extended slicing:"
print "Should be a,c:", l[ "a", "c" ]
print "Short form of extended slicing:"
d = {}
# d[1] = 1
d[1,] = 2
d[1,2] = 3
d[1,2,3] = 4
L = list(d)
L.sort()
print L
s = "Some information"
ss = s[-1]
print "Constant subscript of string", ss
print "Repr"
print `L`, `ss`
print `0L`
print repr(L), repr(ss)
print repr(3L)
print "Slicing on a list:"
l = [ 1, 3, 5, 7, 11, 13, 17 ]
print l[None:None]
n = None
print l[n:n]
print l[3:n]
print l[n:3]
Nuitka-0.5.0.1/tests/basics/ExceptionRaising32.py 0000644 0001750 0001750 00000001677 12265264105 021756 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def raisy():
raise ValueError() from None
try:
print( "Raising exception in a function 'from None'" )
raisy()
except (ValueError,TypeError) as e:
print( "Caught as", repr(e) )
Nuitka-0.5.0.1/tests/basics/Slots.py 0000644 0001750 0001750 00000003114 12265270513 017426 0 ustar hayen hayen 0000000 0000000 # Copyright 2014, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
class W1(object):
def __init__(self):
self.__hidden = 5
class W2(object):
__slots__=['__hidden']
def __init__(self):
self.__hidden = 5
class _W1(object):
def __init__(self):
self.__hidden = 5
class _W2(object):
__slots__=['__hidden']
def __init__(self):
self.__hidden = 5
class a_W1(object):
def __init__(self):
self.__hidden = 5
class a_W2(object):
__slots__=['__hidden']
def __init__(self):
self.__hidden = 5
class W1_(object):
def __init__(self):
self.__hidden = 5
class W2_(object):
__slots__=['__hidden']
def __init__(self):
self.__hidden = 5
for w in (W1, W2, _W1, _W2, a_W1, a_W2, W1_, W2_):
try:
print(w)
print(dir(w))
a = w()
except AttributeError:
print( 'bug in %s' % w )
Nuitka-0.5.0.1/tests/basics/ParameterErrors.py 0000644 0001750 0001750 00000012603 12265264105 021442 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def functionNoParameters():
pass
print "Call a function with no parameters with a plain argument:"
try:
functionNoParameters( 1 )
except TypeError, e:
print repr(e)
print "Call a function with no parameters with a keyword argument:"
try:
functionNoParameters( z = 1 )
except TypeError, e:
print repr(e)
def functionOneParameter(a):
print a
print "Call a function with one parameter with two plain arguments:"
try:
functionOneParameter( 1, 1 )
except TypeError, e:
print repr(e)
print "Call a function with one parameter too many, and duplicate arguments:"
try:
functionOneParameter(6, a=4, *(1, 2, 3))
except TypeError, e:
print repr(e)
print "Call a function with two parameters with three plain arguments:"
def functionTwoParameters(a, b):
print a, b
try:
functionTwoParameters( 1, 2, 3 )
except TypeError, e:
print repr(e)
print "Call a function with two parameters with one plain argument:"
try:
functionTwoParameters( 1 )
except TypeError, e:
print repr(e)
print "Call a function with two parameters with three plain arguments:"
try:
functionTwoParameters( 1, 2, 3 )
except TypeError, e:
print repr(e)
print "Call a function with two parameters with one keyword argument:"
try:
functionTwoParameters( a = 1 )
except TypeError, e:
print repr(e)
print "Call a function with two parameters with three keyword arguments:"
try:
functionTwoParameters( a = 1, b = 2, c = 3 )
except TypeError, e:
print repr(e)
class MethodContainer:
def methodNoParameters(self):
pass
def methodOneParameter(self, a):
print a
def methodTwoParameters(self, a, b):
print a, b
obj = MethodContainer()
print "Call a method with no parameters with a plain argument:"
try:
obj.methodNoParameters( 1 )
except TypeError, e:
print repr(e)
print "Call a method with no parameters with a keyword argument:"
try:
obj.methodNoParameters( z = 1 )
except TypeError, e:
print repr(e)
print "Call a method with one parameter with two plain arguments:"
try:
obj.methodOneParameter( 1, 1 )
except TypeError, e:
print repr(e)
print "Call a method with two parameters with three plain arguments:"
try:
obj.methodTwoParameters( 1, 2, 3 )
except TypeError, e:
print repr(e)
print "Call a method with two parameters with one plain argument:"
try:
obj.methodTwoParameters( 1 )
except TypeError, e:
print repr(e)
print "Call a method with two parameters with one keyword argument:"
try:
obj.methodTwoParameters( a = 1 )
except TypeError, e:
print repr(e)
print "Call a method with two parameters with three keyword arguments:"
try:
obj.methodTwoParameters( a = 1, b = 2, c = 3 )
except TypeError, e:
print repr(e)
def functionPosBothStarArgs(a, b, c, *l, **d):
print a, b, c, l, d
l = [2]
d = { "other" : 7 }
print "Call a function with both star arguments and too little arguments:"
try:
functionPosBothStarArgs( 1, *l, **d )
except TypeError, e:
print repr(e)
print "Call a function with defaults with too little arguments:"
def functionWithDefaults(a, b, c, d = 3):
print a, b, c, d
try:
functionWithDefaults( 1 )
except TypeError, e:
print repr(e)
print "Call a function with defaults with too many arguments:"
try:
functionWithDefaults( 1 )
except TypeError, e:
print repr(e)
print "Complex call with invalid star list and star arguments:"
try:
a = 1
b = 2.0
functionWithDefaults(1,c=3,*a,**b)
except TypeError, e:
print repr(e)
try:
a = 1
b = 2.0
functionWithDefaults(1,*a,**b)
except TypeError, e:
print repr(e)
try:
a = 1
b = 2.0
functionWithDefaults(c=1, *a,**b)
except TypeError, e:
print repr(e)
try:
a = 1
b = 2.0
functionWithDefaults(*a,**b)
except TypeError, e:
print repr(e)
try:
a = 1
functionWithDefaults(*a)
except TypeError, e:
print repr(e)
try:
a = 1
MethodContainer(*a)
except TypeError, e:
print repr(e)
try:
a = 1
MethodContainer()(*a)
except TypeError, e:
print repr(e)
try:
a = 1
MethodContainer.methodTwoParameters(*a)
except TypeError, e:
print repr(e)
try:
a = 1
None(*a)
except TypeError, e:
print repr(e)
try:
a = 1
None(**a)
except TypeError, e:
print repr(e)
print "Call object with name as both keyword and in star dict argument:"
try:
a = {"a" : 3}
None(a=2, **a)
except TypeError, e:
print repr(e)
print "Call function with only defaulted value given as keyword argument:"
def functionwithTwoArgsOneDefaulted(a, b=5):
pass
try:
functionwithTwoArgsOneDefaulted(b=12)
except TypeError, e:
print repr(e)
Nuitka-0.5.0.1/tests/basics/BuiltinOverload.py 0000644 0001750 0001750 00000001550 12265264105 021426 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __builtin__ import len as _len
def len(x):
print x
return _len(x)
print len(range(9))
Nuitka-0.5.0.1/tests/basics/Lamdas.py 0000644 0001750 0001750 00000002563 12265264105 017532 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def lamdaContainer(x):
f = lambda c : c
g = lambda c : c if x else c*c
# h = lambda c: 'a' <= c <= 'z'
y = f(x)
if 'a' <= x <= y <= 'z':
print "Four"
if 'a' <= x <= 'z':
print "Yes"
if 'a' <= x > 'z':
print "Yes1"
if 'a' <= ('1' if x else '2') > 'z':
print "Yes2"
if 'a' <= ('1' if x else '2') > 'z' > i:
print "Yes3"
z = lambda huhu = y : huhu
print "Lambda defaulted gives", z()
lamdaContainer( "b" )
def lambaGenerator():
x = lambda : (yield 3)
gen = x()
print "Lambda generator gives", gen.next()
lambaGenerator()
Nuitka-0.5.0.1/tests/basics/ModuleAttributes.py 0000644 0001750 0001750 00000002760 12265264105 021624 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Some module documentation.
With newline and stuff."""
import os, sys
print "doc:", __doc__
print "filename:", __file__
print "builtins:", __builtins__
print "debug", __debug__
print "debug in builtins", __builtins__.__debug__
print "__initializing__",
try:
print __initializing__
except NameError:
print "not found"
def checkFromFunction():
frame = sys._getframe(1)
locals = frame.f_locals
def displayDict(d):
d = dict(d)
if "__loader__" in d:
d[ "__loader__" ] = ""
return repr( d )
print "Globals", displayDict( frame.f_globals )
print "Locals", displayDict( frame.f_locals )
print "Is identical", frame.f_locals is frame.f_globals
checkFromFunction()
Nuitka-0.5.0.1/tests/basics/Importing.py 0000644 0001750 0001750 00000003256 12265264105 020301 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def localImporter1():
import os
return os
def localImporter1a():
import os as my_os_name
return my_os_name
def localImporter2():
from os import path
return path
def localImporter2a():
from os import path as renamed
return renamed
print "Direct module import", localImporter1()
print "Direct module import using rename", localImporter1a()
print "From module import", localImporter2()
print "From module import using rename", localImporter2a()
from os import *
print "Star import gave us", path
import os.path as myname
print "As import gave", myname
def localImportFailure():
try:
from os import path, lala, listdir
except Exception as e:
print type(e), repr(e)
try:
print listdir
except UnboundLocalError:
print " and listdir was not imported",
print "but path was", path
print "From import that fails in the middle", localImportFailure()
Nuitka-0.5.0.1/tests/basics/Unicode.py 0000644 0001750 0001750 00000001566 12265264105 017721 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print u"gfcrk"
print repr( u"g\xfcrk" )
print r"""\x00"""
print "\ttest\n"
print """
something
with
new
lines"""
Nuitka-0.5.0.1/tests/basics/DoubleDeletions.py 0000644 0001750 0001750 00000002017 12265264105 021404 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
a = 3
del a
try:
del a
except NameError, e:
print "Raised expected exception:", repr(e)
def someFunction(b, c):
b = 1
del b
try:
del b
except UnboundLocalError, e:
print "Raised expected exception:", repr(e)
someFunction( 3, 4 )
Nuitka-0.5.0.1/tests/basics/TryExceptFrames.py 0000644 0001750 0001750 00000002213 12265264105 021406 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys
class X:
def __del__(self):
print "X.__del__ occured"
def raising(doit):
x = X()
if doit:
1 / 0
# Call it without an exception
raising( False )
def catcher():
try:
raising( True )
except ZeroDivisionError:
print "Catching"
print sys.exc_info()[2].tb_next.tb_frame.f_locals
pass
catcher()
print "Good bye."
Nuitka-0.5.0.1/tests/basics/Constants27.py 0000644 0001750 0001750 00000001432 12265264105 020450 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
x = ( { 1, } )
print x
Nuitka-0.5.0.1/tests/basics/MinimalClass.py 0000644 0001750 0001750 00000001447 12265264105 020705 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
a = 1
class B:
b = a
print B.b
Nuitka-0.5.0.1/tests/basics/TryExceptFinally.py 0000644 0001750 0001750 00000004153 12265264105 021574 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"Some doc"
def tryScope1(x):
try:
try:
x += 1
finally:
print "Finally is executed"
try:
z = 1
finally:
print "Deep Nested finally is executed"
except:
print "Exception occured"
else:
print "No exception occured"
tryScope1(1)
print "*" * 20
tryScope1([1])
def tryScope2(x, someExceptionClass):
try:
x += 1
except someExceptionClass, e:
print "Exception class from argument occured:", someExceptionClass, repr(e)
else:
print "No exception occured"
def tryScope3(x):
if x:
try:
x += 1
except TypeError:
print "TypeError occured"
else:
print "Not taken"
print "*" * 20
tryScope2(1, TypeError)
tryScope2([ 1 ], TypeError)
print "*" * 20
tryScope3(1)
tryScope3([1])
tryScope3([])
print "*" * 20
def tryScope4(x):
try:
x += 1
except:
print "exception occured"
else:
print "no exception occured"
finally:
print "finally obeyed"
tryScope4(1 )
tryScope4([1])
def tryScope5():
import sys
print "Exception info is initially", sys.exc_info()
try:
try:
X += 1
finally:
print "Exception info in 'finally' clause is", sys.exc_info()
except:
pass
tryScope5()
Nuitka-0.5.0.1/tests/basics/Assignments.py 0000644 0001750 0001750 00000012256 12265264105 020624 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys
def someFunction():
a = 2
print "Simple assignment to variable:", a
b = c = 3
print "Assignment to 2 variables", b, c
z = [ 1, 2, 3 ]
z[2] = z[1] = 5
print "Assignment to list subscripts:", z
d, e = 1, 2
print "Assignment to variable tuple:", d, e
[ f, g ] = 7, 9
print "Assignment to variable list:", f, g
j = [ h, i ] = ( 7, 9 )
print "Complex Assignment from variabe list:", j, type(j), h, i
a, (b,c) = 1, (2,3 )
print "Assigment to nested tuples:", a, b, c
v = [ 1, 2, 3, 4 ]
v[2:3] = (8,9)
print "Assignment to list slice", v
def varargsFunction(*args):
f1, f2, f3, f4 = args
print "Assignment from list", f1, f2, f3, f4
def otherFunction():
class Iterable:
def __iter__(self):
return iter(range(3))
a, b, c = Iterable()
print "Assignments from iterable", a ,b ,c
print "Assignments from too small iterable",
try:
f, g = 1,
except Exception, e:
print "gave", type(e), repr(e)
try:
print f
except UnboundLocalError:
print "Variable f is untouched"
try:
print g
except UnboundLocalError:
print "Variable g is untouched"
print "Assignments from too large iterable",
try:
d, j = 1, 2, 3
except Exception, e:
print "gave", type(e), repr(e)
try:
print d
except UnboundLocalError:
print "Variable d is untouched"
try:
print j
except UnboundLocalError:
print "Variable j is untouched"
class BasicIterClass:
def __init__(self, n):
self.n = n
self.i = 0
def __next__(self):
res = self.i
if res >= self.n:
raise StopIteration
self.i = res + 1
return res
if sys.version_info[0] < 3:
def next(self):
return self.__next__()
class IteratingSequenceClass:
def __init__(self, n):
self.n = n
def __iter__(self):
return BasicIterClass(self.n)
try:
a, b, c = IteratingSequenceClass(2)
except ValueError:
print "Exception from iterating over too short class", sys.exc_info()
def anotherFunction():
d = {}
print "Assignment to dictionary with comma subscript:",
# d[ "f" ] = 3
d[ "a", "b" ] = 6
d[ "c", "b" ] = 9
print sorted( d.iteritems() )
def swapVariables():
print "Strange swap form:"
a = 1
b = 2
a, b, a = b, a, b
print a, b
def interuptedUnpack():
a = 1
b = 2
print "Assignment from a too short tuple to multiple targets",
try:
s = a,
c, d = s
except ValueError, e:
print "gives ValueError", repr(e),
try:
print c
except UnboundLocalError, e:
print "and then nothing is assigned", repr(e)
del a, b
z = []
try:
a, z.unknown, b = 1, 2, 3
except AttributeError:
print "Interrupted unpack, leaves value assigned", a
def multiTargetInterrupt():
a = 1
b = 2
print "Multiple, overlapping targets",
d = c, d = a, b
print d, c,
del c
del d
c, d = d = a, b
print d, c
print "Error during multiple assignments",
del c
del d
e = 9
z = []
try:
c, d = e, z.a = a, b
except AttributeError:
print "having attribute error", c, d, e
del c
del d
e = 9
print "Error during multiple assignments",
try:
c, d = z.a, e = a, b
except AttributeError:
print "having attribute error", c, d, e
def optimizeableTargets():
a = [ 1, 2 ]
a[ int(1) ] = 3
print "Optimizable slice operation, results in", a
def complexDel():
a = b = c = d = 1
del a, b, ( c, d )
try:
print c
except UnboundLocalError, e:
print "yes, del worked", repr(e)
def globalErrors():
global unassigned_1, unassigned_2
try:
unassigned_1 = unassigned_1
except NameError as e:
print "Accessing unassigned global gives", repr(e)
try:
del unassigned_2
except NameError as e:
print "Del on unassigned global gives", repr(e)
someFunction()
varargsFunction(1,2,3,4)
otherFunction()
anotherFunction()
swapVariables()
interuptedUnpack()
multiTargetInterrupt()
optimizeableTargets()
complexDel()
globalErrors()
Nuitka-0.5.0.1/tests/basics/TryYieldFinally.py 0000644 0001750 0001750 00000004351 12265264105 021412 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def tryContinueFinallyTest():
for x in range(10):
try:
if x % 2 == 1:
continue
finally:
yield x
def tryBreakFinallyTest():
for x in range(10):
try:
if x == 5:
break
finally:
yield x
def tryFinallyAfterYield():
try:
yield 3
finally:
print "Executing finally"
def tryReturnFinallyYield():
try:
return
finally:
yield 1
def tryReturnExceptYield():
try:
return
except StopIteration:
print "Caught StopIteration"
yield 2
except:
yield 1
else:
print "No exception"
def tryStopIterationExceptYield():
try:
raise StopIteration
except StopIteration:
print "Caught StopIteration"
yield 2
except:
yield 1
else:
print "No exception"
print "Check if finally is executed in a continue using for loop:"
print tuple( tryContinueFinallyTest() )
print "Check if finally is executed in a break using for loop:"
print tuple( tryBreakFinallyTest() )
print "Check what try yield finally something does:"
print tuple( tryFinallyAfterYield() )
print "Check if yield is executed in finally after return:"
print tuple( tryReturnFinallyYield() )
print "Check if yield is executed in except after return:"
print tuple( tryReturnExceptYield() )
print "Check if yield is executed in except after StopIteration:"
print tuple( tryReturnExceptYield() )
Nuitka-0.5.0.1/tests/basics/OverflowFunctions.py 0000644 0001750 0001750 00000002654 12265264105 022026 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def starImporterFunction():
from sys import *
print "Version", version.split()[0].split( "." )[:-1]
starImporterFunction()
def deepExec():
for_closure = 3
def deeper():
for_closure_as_well = 4
def execFunction():
code = "f=2"
# Can fool it to nest
exec code in None, None
print "Locals now", locals()
print "Closure one level up was taken", for_closure_as_well
print "Closure two levels up was taken", for_closure
print "Globals still work", starImporterFunction
print "Added local from code", f
execFunction()
deeper()
deepExec()
Nuitka-0.5.0.1/tests/basics/Functions32.py 0000644 0001750 0001750 00000006157 12265264105 020451 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def kwonlysimple(*, a):
return a
print( "Most simple case", kwonlysimple( a = 3 ) )
def kwonlysimpledefaulted(*, a = 5):
return a
print( "Default simple case", kwonlysimpledefaulted() )
def default1():
print( "Called", default1 )
return 1
def default2():
print( "Called", default2 )
return 2
def default3():
print( "Called", default3 )
return 3
def default4():
print( "Called", default4 )
return 4
def annotation1():
print ( "Called", annotation1 )
return "a1"
def annotation2():
print ( "Called", annotation2 )
return "a2"
def annotation3():
print ( "Called", annotation3 )
return "a3"
def annotation4():
print ( "Called", annotation4 )
return "a4"
def annotation5():
print ( "Called", annotation5 )
return "a5"
def annotation6():
print ( "Called", annotation6 )
return "a6"
def annotation7():
print ( "Called", annotation7 )
return "a7"
def annotation8():
print ( "Called", annotation8 )
return "a8"
def annotation9():
print ( "Called", annotation9 )
return "a9"
def kwonlyfunc(x: annotation1(), y: annotation2() = default1(), z: annotation3() = default2(), *, a: annotation4(), b: annotation5() = default3(), c: annotation6() = default4(), d: annotation7(), **kw: annotation8()) -> annotation9():
print( x, y, z, a, b, c, d )
print( kwonlyfunc.__kwdefaults__ )
print( "Keyword only function" )
kwonlyfunc( 7, a = 8, d = 12 )
print( "Annotations come out as", sorted( kwonlyfunc.__annotations__ ) )
kwonlyfunc.__annotations__ = {}
print( "After updating to None it is", kwonlyfunc.__annotations__ )
kwonlyfunc.__annotations__ = { "k" : 9 }
print( "After updating to None it is", kwonlyfunc.__annotations__ )
def kwonlystarfunc(*, a, b, **d):
return a, b, d
print( "kwonlystarfunc", kwonlystarfunc( a = 8, b = 12, k = 9, j = 7 ) )
def deeplyNestedNonLocalWrite():
x = 0
y = 0
def f():
def g():
nonlocal x
x = 3
return x
return g()
return f(), x
print( "Deeply nested non local writing function", deeplyNestedNonLocalWrite() )
def deletingClosureVariables():
try:
x = 1
def g():
nonlocal x
del x
g()
g()
except Exception as e:
return e
print( "Using deleted non-local vaiables", deletingClosureVariables() )
Nuitka-0.5.0.1/tests/basics/Future32.py 0000644 0001750 0001750 00000001606 12265264105 017745 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import barry_as_FLUFL
print( eval( "1 <> 2" ) )
print( eval( '"a"<>"b"' ) )
print( eval( 'range(7) <> range(7)' ) )
Nuitka-0.5.0.1/tests/basics/ExceptionRaising.py 0000644 0001750 0001750 00000025523 12265264105 021605 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys
print "Raising an exception type in a function:"
def raiseExceptionClass():
raise ValueError
try:
raiseExceptionClass()
except Exception, e:
print "Caught exception", e, repr(e), type(e)
print "After catching, sys.exc_info is this", sys.exc_info()
print "*" * 20
print "Raising an exception instance in a function:"
def raiseExceptionInstance():
raise ValueError( "hallo" )
try:
raiseExceptionInstance()
except Exception, f:
print "Caught exception", f, repr(f), type(f)
print "After catching, sys.exc_info is this", sys.exc_info()
print "*" * 20
print "Raising an exception, then catch it to re-raise it:"
def raiseExceptionAndReraise():
try:
x = 0
y = x / x
except:
raise
try:
raiseExceptionAndReraise()
except:
print "Catched reraised"
print "After catching, sys.exc_info is this", sys.exc_info()
print "*" * 20
print "Access an undefined global variable in a function:"
def raiseNonGlobalError():
return undefined_value
try:
raiseNonGlobalError()
except:
print "NameError caught"
print "After catching, sys.exc_info is this", sys.exc_info()
print "*" * 20
print "Raise a new style class as an exception, should be rejected:"
def raiseIllegalError():
class X(object):
pass
raise X()
try:
raiseIllegalError()
except TypeError, E:
print "New style class exception correctly rejected:", E
except:
print sys.exc_info()
assert False, "Error, new style class exception was not rejected"
print "After catching, sys.exc_info is this", sys.exc_info()
print "*" * 20
print "Raise an old-style class, version dependent outcome:"
class ClassicClassException:
pass
def raiseCustomError():
raise ClassicClassException()
try:
try:
raiseCustomError()
except ClassicClassException:
print "Caught classic class exception"
except:
print sys.exc_info()
assert False, "Error, old style class exception was not caught"
except TypeError, e:
print "Python3 hates to even try and catch classic classes", e
else:
print "Classic exception catching was considered fine."
print "After catching, sys.exc_info is this", sys.exc_info()
print "*" * 20
print "Checking tracebacks:"
def checkTraceback():
import sys, traceback
try:
raise "me"
except:
assert sys.exc_info()[0] is not None
assert sys.exc_info()[1] is not None
assert sys.exc_info()[2] is not None
print "Check traceback:"
traceback.print_tb( sys.exc_info()[2], file = sys.stdout )
print "End of traceback"
print "Type is", sys.exc_info()[0]
print "Value is", sys.exc_info()[1]
checkTraceback()
print "*" * 20
print "Check lazy exception creation:"
def checkExceptionConversion():
try:
raise Exception( "some string")
except Exception, err:
print "Catched raised object", err, type( err )
try:
raise Exception, "some string"
except Exception, err:
print "Catched raised type, value pair", err, type( err )
checkExceptionConversion()
print "*" * 20
print "Check exc_info scope:"
def checkExcInfoScope():
try:
raise ValueError
except:
assert sys.exc_info()[0] is not None
assert sys.exc_info()[1] is not None
assert sys.exc_info()[2] is not None
if sys.version_info[0] < 3:
print "Exc_info remains visible after exception handler for Python2"
assert sys.exc_info()[0] is not None
assert sys.exc_info()[1] is not None
assert sys.exc_info()[2] is not None
else:
print "Exc_info is clear after exception handler for Python3"
assert sys.exc_info()[0] is None
assert sys.exc_info()[1] is None
assert sys.exc_info()[2] is None
def subFunction():
print "Entering with exception info", sys.exc_info()
assert sys.exc_info()[0] is not None
assert sys.exc_info()[1] is not None
assert sys.exc_info()[2] is not None
try:
print "Trying"
except:
pass
print "After trying something and didn't have an exception, info is", sys.exc_info()
print "Call a function inside the exception handler and check there too."
try:
raise KeyError
except:
assert sys.exc_info()[0] is not None
assert sys.exc_info()[1] is not None
assert sys.exc_info()[2] is not None
subFunction()
print "Call it twice and see."
try:
raise "me"
except:
assert sys.exc_info()[0] is not None
assert sys.exc_info()[1] is not None
assert sys.exc_info()[2] is not None
subFunction()
subFunction()
if sys.version_info[0] < 3:
sys.exc_clear()
checkExcInfoScope()
print "*" * 20
# Check that the sys.exc_info is cleared again, after being set inside the
# function checkExcInfoScope, it should now be clear again.
assert sys.exc_info()[0] is None, sys.exc_info()[0]
assert sys.exc_info()[1] is None
assert sys.exc_info()[2] is None
print "Check catching subclasses"
def checkDerivedCatch():
class A(BaseException):
pass
class B(A):
def __init__(self):
pass
a = A()
b = B()
try:
raise A, b
except B, v:
print "Caught B", v
except A, v:
print "Didn't catch as B, but as A, Python3 does that", v
else:
print "Not caught A class, not allowed to happen."
try:
raise B, a
except TypeError, e:
print "TypeError with pair form for class not taking args:", e
checkDerivedCatch()
print "*" * 20
def checkNonCatch1():
print "Testing if the else branch is executed in the optimizable case:"
try:
0
except TypeError:
print "Should not catch"
else:
print "Executed else branch correctly"
checkNonCatch1()
print "*" * 20
def checkNonCatch2():
try:
print "Testing if the else branch is executed in the non-optimizable case:"
except TypeError:
print "Should not catch"
else:
print "Executed else branch correctly"
checkNonCatch2()
print "*" * 20
print "Checking raise that with exception arguments that raise error themselves."
def checkRaisingRaise():
def geterror():
return 1/0
try:
geterror()
except Exception, e:
print "Had exception", e
try:
raise TypeError, geterror()
except Exception, e:
print "Had exception", e
try:
raise TypeError, 7, geterror()
except Exception, e:
print "Had exception", e
checkRaisingRaise()
print "*" * 20
print "Checking a re-raise that isn't one:"
def checkMisRaise():
raise
try:
checkMisRaise()
except Exception, e:
print "Without existing exception, re-raise gives:", e
print "*" * 20
print "Raising an exception in an exception handler gives:"
def nestedExceptions(a, b):
try:
a / b
except ZeroDivisionError:
a / b
try:
nestedExceptions( 1, 0 )
except Exception, e:
print "Nested exception gives", e
print "*" * 20
print "Checking unpacking from an exception as a sequence:"
def unpackingCatcher():
try:
raise ValueError(1,2)
except ValueError as (a,b):
print "Unpacking caught exception and unpacked", a, b
unpackingCatcher()
print "*" * 20
print "Testing exception that escapes __del__ and therefore cannot be raised"
def unraisableExceptionInDel():
class C:
def __del__(self):
c = 1 / 0
def f():
C()
f()
unraisableExceptionInDel()
print "*" * 20
print "Testing exception changes between generator switches:"
def yieldExceptionInteraction():
def yield_raise():
try:
raise KeyError("caught")
except KeyError:
yield sys.exc_info()[0]
yield sys.exc_info()[0]
yield sys.exc_info()[0]
g = yield_raise()
print "Initial yield from catch in generator", next( g )
print "Checking from here", sys.exc_info()[0]
print "Second yield from the catch reentered", next( g )
print "Checking from here again ", sys.exc_info()[0]
print "After leaving the catch generator yielded", next( g )
yieldExceptionInteraction()
print "*" * 20
print "Testing exception change between generator switches while handling an own exception"
def yieldExceptionInteraction2():
def yield_raise():
print "Yield finds at generator entry", sys.exc_info()[0]
try:
raise ValueError("caught")
except ValueError:
yield sys.exc_info()[0]
yield sys.exc_info()[0]
yield sys.exc_info()[0]
try:
z
except Exception:
print "Checking from here", sys.exc_info()[0]
g = yield_raise()
v = next( g )
print "Initial yield from catch in generator", v
print "Checking from here", sys.exc_info()[0]
print "Second yield from the catch reentered", next( g )
print "Checking from here again ", sys.exc_info()[0]
print "After leaving the catch generator yielded", next( g )
yieldExceptionInteraction2()
print "*" * 20
print "Check what happens if a function attempts to clear the exception in a handler"
def clearingException():
def clearit():
try:
if sys.version_info[0] < 3:
sys.exc_clear()
except KeyError:
pass
try:
raise KeyError
except:
print "Before clearing, it's", sys.exc_info()
clearit()
print "After clearing, it's", sys.exc_info()
clearingException()
print "*" * 20
print "Check that multiple exceptions can be caught in a handler through a variable:"
def multiCatchViaTupleVariable():
some_exceptions = (KeyError, ValueError)
try:
raise KeyError
except some_exceptions:
print "Yes, indeed."
multiCatchViaTupleVariable()
def raiseValueWithValue():
try:
raise ValueError(1,2,3), (ValueError(1,2,3))
except Exception as e:
print "Gives", e
print "Check exception given when value is raised with value", raiseValueWithValue()
# Make sure the repr is fine, at one time for Python3, they were actually really string objects, unnoticed:
a = IOError
print repr(a)
Nuitka-0.5.0.1/tests/basics/WithStatements.py 0000644 0001750 0001750 00000006353 12265264105 021315 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
x = 0
# This is used to trace the exact interaction with the context manager to uncover and
# decide orddering and correctness of calls.
class MyContextManager(object):
def __getattribute__(self, attribute_name):
print "Asking context manager attribute", attribute_name
return object.__getattribute__( self, attribute_name )
def __enter__(self):
global x
x += 1
print "Entered context manager with counter value", x
return x
def __exit__(self, exc_type, exc_value, traceback):
print "Exit sees", exc_type, exc_value, traceback
return False
print "Use context manager and raise exception in the body:"
with MyContextManager() as x:
print "x has become", x
try:
with MyContextManager() as x:
print "x has become", x
raise Exception( "Lalala" )
print x
except Exception, e:
print "Caught raised exception", repr(e)
l = range(3)
print "Use context manager and assign to subscription target:"
with MyContextManager() as l[0]:
print "Complex assignment target works", l[0]
try:
import sys
with MyContextManager():
sys.exit(9)
except BaseException as e:
print "Caught base exception", repr(e)
print "Use context manager and fail to assign to attribute:"
try:
import sys
with MyContextManager() as l.wontwork:
sys.exit(9)
except BaseException as e:
print "Caught base exception", repr(e)
print "Use context manager to do nothing inside:"
with MyContextManager() as x:
pass
# Use context manager and fail to assign.
def returnFromContextBlock():
# Use context manager to do nothing.
with MyContextManager() as x:
return 7
print "Use context manager to return value:"
r = returnFromContextBlock()
print "Return value", r
class NonContextManager1:
def __enter__(self):
return self
class NonContextManager2:
def __exit__(self):
return self
try:
with NonContextManager1() as x:
print x
except Exception, e:
print repr(e)
try:
with NonContextManager2() as x:
print x
except Exception, e:
print repr(e)
class NotAtAllContextManager:
pass
try:
with NotAtAllContextManager() as x:
print x
except Exception, e:
print repr(e)
class MeanContextManager:
def __enter__(self):
raise ValueError( "Nah, I won't play" )
def __exit__(self):
print "Called exit, yes"
try:
with MeanContextManager() as x:
print x
except Exception, e:
print repr(e)
Nuitka-0.5.0.1/tests/basics/Referencing33.py 0000644 0001750 0001750 00000005675 12265264105 020735 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys, gc
def simpleFunction1():
def g():
for a in range(20):
yield a
def h():
yield 4
yield 5
yield 6
def f():
yield from g()
yield from h()
x = list( f() )
def simpleFunction2():
def g():
for a in range(20):
yield a
def h():
yield 4
yield 5
yield 6
raise TypeError
def f():
yield from g()
yield from h()
try:
x = list( f() )
except TypeError:
pass
m1 = {}
m2 = {}
def snapObjRefCntMap(before):
if before:
global m1
m = m1
else:
global m2
m = m2
for x in gc.get_objects():
if x is m1:
continue
if x is m2:
continue
m[ str( x ) ] = sys.getrefcount( x )
def checkReferenceCount(checked_function, max_rounds = 10):
assert sys.exc_info() == ( None, None, None ), sys.exc_info()
print( checked_function.__name__ + ":", end = "" )
ref_count1 = 17
ref_count2 = 17
explain = False
for count in range( max_rounds ):
x1 = 0
x2 = 0
gc.collect()
ref_count1 = sys.gettotalrefcount()
if explain and count == max_rounds - 1:
snapObjRefCntMap( True )
checked_function()
assert sys.exc_info() == ( None, None, None ), sys.exc_info()
gc.collect()
if explain and count == max_rounds - 1:
snapObjRefCntMap( False )
ref_count2 = sys.gettotalrefcount()
if ref_count1 == ref_count2:
print( "PASSED" )
break
# print count, ref_count1, ref_count2
else:
print( "FAILED", ref_count1, ref_count2, "leaked", ref_count2 - ref_count1 )
if explain:
assert m1
assert m2
for key in m1.keys():
if key not in m2:
print( "*" * 80 )
print( key )
elif m1[key] != m2[key]:
print( "*" * 80 )
print( key )
else:
pass
# print m1[key]
assert sys.exc_info() == ( None, None, None ), sys.exc_info()
gc.collect()
checkReferenceCount( simpleFunction1 )
checkReferenceCount( simpleFunction2 )
Nuitka-0.5.0.1/tests/basics/ExtremeClosure.py 0000644 0001750 0001750 00000001675 12265264105 021302 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
a = 1
b = 1
def someFunction():
a = a
class someClass():
b = b
someClass()
try:
someFunction()
except UnboundLocalError:
print "Expected unbound local error occured."
Nuitka-0.5.0.1/tests/basics/Constants.py 0000644 0001750 0001750 00000006405 12265264105 020304 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Playing around with constants only. """
for value in (0, 0L, 3L, -4L, 17, "hey", (0, ),(0L, ), 0.0, -0.0 ):
print value, repr(value)
print 1 == 0
print repr(0L), repr(0L) == "0L"
print {} is {}
a = ( {}, [] )
a[0][1] = 2
a[1].append( 3 )
print a
print ( {}, [] )
def argChanger(a):
a[0][1] = 2
a[1].append( 3 )
return a
print argChanger( ( {}, [] ) )
print ( {}, [] )
print set(['foo'])
def mutableConstantChanger():
a = ( [ 1, 2 ], [ 3 ] )
print a
a[ 1 ].append( 5 )
print a
d = { "l": [], "m" : [] }
d["l"].append( 7 )
print d
declspec = None
spec = dict(qual=[], storage=set(), type=[], function=set(), q = 1)
spec[ "type" ].insert( 0, 2 )
spec[ "storage" ].add(3)
print sorted( spec )
mutableConstantChanger()
mutableConstantChanger()
def defaultKeepsIdentity(arg = "str_value"):
print arg is "str_value"
defaultKeepsIdentity()
# Dictionary creation from call args
def dd(**d):
return d
def f():
def one():
print "one"
def two():
print "two"
a = dd(qual=one(), storage=two(), type=[], function=[])
print "f mutable", a
a = dd(qual=1, storage=2, type=3, function=4)
print "f immutable", a
# TODO: This exposes a bug in how the star dict argument should populate the
# dictionary first instead of last, and the called arguments might have to
# come from pairs so hashing does not reorder.
# x = { "p" : 7 }
# a = dd(qual=[], storage=[], type=[], function=[],**x)
# print "f ext mutable", a
# x = { "p" : 8 }
# a = dd(qual=1, storage=2, type=3, function=4,**x)
# print "f ext immutable", a
f()
# Dictionary creation one after another
x={}
x["function"] = []
x["type"] = []
x["storage"] = []
x["qual"] = []
print "m", x
x={}
x["function"] = 1
x["type"] = 2
x["storage"] = 3
x["qual"] = 4
print "m", x
# Constants in the code must be created differently.
d = { "qual" : [], "storage" : [], "type2" : [], "function" : [] }
print "c", d
d = { "qual" : 1, "storage" : 2, "type2" : 3, "function" : 4 }
print "c", d
# Constants that might be difficult
min_signed_int = int( -(2**(8*8-1)-1)-1 )
print "small int", min_signed_int, type(min_signed_int)
min_signed_int = int( -(2**(8*4-1)-1)-1 )
print "small int", min_signed_int, type(min_signed_int)
# Constants that might be difficult
min_signed_long = long( -(2**(8*8-1)-1)-1 )
print "small long", min_signed_long, type(min_signed_long)
min_signed_long = long( -(2**(8*4-1)-1)-1 )
print "small long", min_signed_long, type(min_signed_long)
Nuitka-0.5.0.1/tests/basics/Classes32.py 0000644 0001750 0001750 00000003517 12265264105 020073 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def a():
x = 1
class A:
print( x )
print( "Called", a )
return A
def b():
class B:
pass
print( "Called", b )
return B
from collections import OrderedDict
def m():
class M(type):
# @classmethod
def __new__(metacls, class_name, bases, attrs, **over):
print( "Metaclass M.__new__ metacls", metacls, "name", class_name, "bases", bases, "dict", attrs, "extra class defs", over )
return type.__new__( metacls, class_name, bases, attrs )
def __init__(self, name, bases, attrs, **over):
print( "Metaclass M.__init__", name, bases, attrs, over )
super().__init__( name, bases, attrs )
# TODO: Removing this
# @classmethod
def __prepare__(name, bases, **over):
print( "Metaclass M.__prepare__", name, bases, over )
return OrderedDict()
print( "Called", m )
return M
def d():
print( "Called", d )
return 1
def e():
print( "Called", e )
return 2
class C1(a(), b(), other = d(), metaclass = m(), yet_other = e()):
pass
print( type( C1.__dict__ ) )
Nuitka-0.5.0.1/tests/basics/Functions.py 0000644 0001750 0001750 00000031652 12265264105 020302 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
var_on_module_level = 1
def closureTest1(some_arg):
x = 3
def enclosed(f = "default_value"):
return x
return enclosed
print "Call closured function returning function:", closureTest1( some_arg = "ignored" )()
def closureTest2(some_arg):
def enclosed(f = "default_value"):
return x
x = 4
return enclosed
print "Call closured function returning function:", closureTest2( some_arg = "ignored" )()
def defaultValueTest1(no_default, some_default_constant = 1):
return some_default_constant
print "Call function with 2 parameters, one defaulted, and check that the default value is used:", defaultValueTest1("ignored")
def defaultValueTest1a(no_default, some_default_constant_1 = 1, some_default_constant_2 = 2):
return some_default_constant_2 - some_default_constant_1
print "Call function with 3 parameters, 2 defaulted, check they are used correctly:", defaultValueTest1a("ignored")
def defaultValueTest2(no_default, some_default_variable = var_on_module_level*2):
return some_default_variable
print "Call function with 2 parameters, 1 defaulted with an expression, check its result", defaultValueTest2("ignored")
var_on_module_level = 2
print "Call function with 2 parameters, 1 defaulted with an expression, values have changed since, check its result", defaultValueTest2("ignored")
def contractionTest():
j = 2
return [ j + i for i in range(8) ]
print "Call function that returns a list contraction:", contractionTest()
def defaultValueTest3a( no_default, funced_defaulted = defaultValueTest2(var_on_module_level)):
return [ i + funced_defaulted for i in range(8) ]
print "Call function that has a default value coming from a function call:", defaultValueTest3a("ignored")
def defaultValueTest3b(no_default, funced_defaulted = defaultValueTest2(var_on_module_level)):
local_var = [ funced_defaulted + i for i in range(8) ]
return local_var
print "Call function that returns a list contraction result via a local variable:", defaultValueTest3b("ignored")
def defaultValueTest3c(no_default, funced_defaulted = defaultValueTest2(var_on_module_level)):
local_var = [ [ j+funced_defaulted+1 for j in range(i) ] for i in range(8) ]
return local_var
print "Call function that returns a nested list contraction with input from default parameter", defaultValueTest3c("ignored")
def defaultValueTest4( no_default, funced_defaulted = lambda x: x**2):
return funced_defaulted(4)
print "Call function that returns value caculated by a lamba function as default parameter", defaultValueTest4("ignored")
def defaultValueTest4a( no_default, funced_defaulted = lambda x: x**2):
c = 1
d = funced_defaulted(1)
r = ( i+j+c+d for i, j in zip( range(8), range( 9 ) ) )
l = []
for x in r:
l.append( x )
return l
print "Call function that has a lambda calculated default parameter and a generator expression", defaultValueTest4a("ignored")
def defaultValueTest4b( no_default, funced_defaulted = lambda x: x**3):
c = 1
d = funced_defaulted(1)
# Nested generators
l = []
for x in ( (d+j for j in range(4)) for i in range(8) ):
for y in x:
l.append( y )
return l
print "Call function that has a lambda calculated default parameter and a nested generator expression", defaultValueTest4b("ignored")
def defaultValueTest5( no_default, tuple_defaulted = (1,2,3)):
return tuple_defaulted
print "Call function with default value that is a tuple", defaultValueTest5("ignored")
def defaultValueTest6( no_default, list_defaulted = [1,2,3]):
return list_defaulted
print "Call function with default value that is a list", defaultValueTest6("ignored")
def lookup(unused, something):
something.very.namelookup.chaining()
something.very.namelookup.chaining()
def local_function(a,z=9):
b = `a*a+1`
c = (a,b,a**32,a+a)
d = long("0")
e = int("77")
d= long(b)
e= long(1+1)
return a, b, c, d, e, z
print "Call function with many variables calculated and returned", local_function(1,z=5)
x = len( "hey" )
def in_test(a):
# if 7 in a:
# print "hey"
8 in a
9 not in a
def printing():
print "Hallo"
print "du"
print "da"
def my_deco(function):
def new_function(c, d):
return function( d, c )
return new_function
@my_deco
def decoriert(a,b):
def subby(a):
return 2 + a
return 1+subby(b)
print "Function with decoration", decoriert( 3, 9 )
#def var_test(a):
# b = len(a)
# c = len(a)
def user():
global a
return a
a = "oh common"
some_constant_tuple = (2,5,7)
some_semiconstant_tuple = (2,5,a)
f = a * 2
print defaultValueTest1("ignored")
# The change of the default variable doesn't influence the default
# parameter of defaultValueTest2, that means it's also calculated
# at the time the function is defined.
module_level = 7
print defaultValueTest2("also ignored")
def starArgTest(a, b, c):
return a, b, c
print "Function called with star arg from tuple"
star_list_arg = ( 11, 44, 77 )
print starArgTest( *star_list_arg )
print "Function called with star arg from list"
star_list_arg = [ 7, 8, 9 ]
print starArgTest( *star_list_arg )
star_dict_arg = {
"a" : 9, "b" : 3, "c": 8
}
print "Function called with star arg from dict"
print starArgTest( **star_dict_arg )
lambda_func = lambda a, b : a < b
lambda_args = ( 8, 9 )
print "Lambda function called with star args from tuple"
print lambda_func( *lambda_args )
print "Lambda function called with star args from list"
lambda_args = [ 8, 7 ]
print lambda_func( *lambda_args )
print "Function with nested args"
def nested_args_function((a,b), c):
return a, b, c
print nested_args_function( ( 1, 2 ), 3 )
try:
nested_args_function( ( 1, 2, 3 ), 3 )
except ValueError, e:
print "Calling nested with too long tuple gave:", e
try:
nested_args_function( ( 1, ), 3 )
except ValueError, e:
print "Calling nested with too short tuple gave:", e
def deeply_nested_function( ( ( a, ), b, c, ( d, (e,f ) ) )):
return a, b, c, d, e, f
print "Deeply nested function", deeply_nested_function( ( ( 1, ), 2, 3, ( 4, ( 5, 6 ) ) ) )
def default_giver():
class R:
def __iter__(self):
print "Giving iter"
return iter( range(2) )
return R()
print "Function with nested args that have defaults"
def nested_args_function_with_defaults((a,b) = default_giver(), c = 5):
return a, b, c
print "Calling it"
print nested_args_function_with_defaults()
print "Generator function without context"
def generator_without_context_function():
gen = ( x for x in range(9) )
return tuple( gen )
print generator_without_context_function()
print "Generator function with 2 iterateds"
def generator_with_2_fors():
return tuple( ( x, y ) for x in range(2) for y in range(3) )
print generator_with_2_fors()
def someYielder():
yield 1
yield 2
def someYieldFunctionUser():
print "someYielder", someYielder()
result = []
for a in someYielder():
result.append( a )
return result
print "Function that uses some yielding function coroutine"
print someYieldFunctionUser()
def someLoopYielder():
for i in (0, 1, 2):
yield i
def someLoopYieldFunctionUser():
result = []
for a in someLoopYielder():
result.append( a )
return result
print "Function that uses some yielding function coroutine that loops"
print someLoopYieldFunctionUser()
def someGeneratorClosureUser():
def someGenerator():
result = []
def userOfGeneratorLocalVar():
return x+1
x = 2
yield userOfGeneratorLocalVar()
yield 6
gen = someGenerator()
return [ gen.next(), gen.next() ]
print "Function generator that uses a local function accessing its local variables to yield:"
print someGeneratorClosureUser()
def someClosureUsingGeneratorUser():
offered = 7
def someGenerator():
yield offered
return someGenerator().next()
print "Function generator that yield from its closure"
print someClosureUsingGeneratorUser()
print "Function call with both star args and named args"
def someFunction(a, b, c, d):
print a, b, c, d
someFunction( a = 1, b = 2, **{ "c" : 3, "d" : 4 } )
print "Order of evaluation of function and args:"
def getFunction():
print "getFunction",
def x(y, u, a, k):
return y, u, k, a
return x
def getPlainArg1():
print "getPlainArg1",
return 9
def getPlainArg2():
print "getPlainArg2",
return 13
def getKeywordArg1():
print "getKeywordArg1",
return "a"
def getKeywordArg2():
print "getKeywordArg2",
return "b"
getFunction()( getPlainArg1(), getPlainArg2(), k = getKeywordArg1(), a = getKeywordArg2() )
print
def getListStarArg():
print "getListStarArg",
return [1]
def getDictStarArg():
print "getDictStarArg",
return { "k" : 9 }
print "Same with star args"
getFunction()( getPlainArg1(), a = getKeywordArg1(), *getListStarArg(), **getDictStarArg() )
print
print "Dictionary creation order:"
d = {
getKeywordArg1() : getPlainArg1(),
getKeywordArg2() : getPlainArg2()
}
print
print "Throwing an exception to a generator function:"
def someGeneratorFunction():
try:
yield 1
yield 2
except:
yield 3
yield 4
gen1 = someGeneratorFunction()
print "Fresh Generator Function throwing gives",
try:
print gen1.throw( ValueError ),
except ValueError:
print "exception indeed"
gen2 = someGeneratorFunction()
print "Used Generator Funtion throwing gives",
gen2.next()
print gen2.throw( ValueError ), "indeed"
gen3 = someGeneratorFunction()
print "Fresh Generator Function close gives",
print gen3.close()
gen4 = someGeneratorFunction()
print "Used Generator Function that miscatches close gives",
gen4.next()
try:
print gen4.close(),
except RuntimeError:
print "runtime exception indeed"
gen5 = someGeneratorFunction()
print "Used Generator Function close gives",
gen5.next()
gen5.next()
gen5.next()
print gen5.close(),
def receivingGenerator():
while True:
a = yield 4
yield a
print "Generator function that receives",
gen6 = receivingGenerator()
print gen6.next(),
print gen6.send( 5 ),
print gen6.send( 6 ),
print gen6.send( 7 ),
print gen6.send( 8 )
print "Generator function whose generator is copied",
def generatorFunction():
yield 1
yield 2
gen7 = generatorFunction()
gen7.next()
gen8 = iter(gen7)
print gen8.next()
def doubleStarArgs(*a, **d):
return a, d
try:
from UserDict import UserDict
except ImportError:
print "Using Python3, making own non-dict dict"
class UserDict(dict):
pass
print "Function that has keyword argument matching the list star arg name",
print doubleStarArgs( 1, **UserDict( a = 2 ) )
def generatorFunctionUnusedArg(a):
yield 1
generatorFunctionUnusedArg( 3 )
def closureHavingGenerator(arg):
def gen(x = 1):
yield arg
return gen()
print "Function generator that has a closure and default argument",
print list( closureHavingGenerator(3) )
def comp_args1((a, b)):
return a,b
def comp_args2((a, b)=(3, 4)):
return a, b
def comp_args3(a, (b, c)):
return a, b, c
def comp_args4(a=2, (b, c)=(3, 4)):
return a, b, c
print "Complex args functions", comp_args1( (2, 1) ), comp_args2(), comp_args2( (7,9)), comp_args3( 7, (8,9)), comp_args4()
def functionWithDualStarArgsAndKeywordsOnly(a1, a2, a3, a4, b):
return a1, a2, a3, a4, b
l = [ 1, 2, 3 ]
d = { "b": 8 }
print "Dual star args, but not positional call", functionWithDualStarArgsAndKeywordsOnly( a4 = 1, *l, **d )
def posDoubleStarArgsFunction(a, b, c, *l, **d):
return a, b, c, l, d
l = [2]
d = { "other" : 7, "c" : 3 }
print "Dual star args consuming function", posDoubleStarArgsFunction( 1, *l, **d )
import inspect, sys
for value in sorted( dir() ):
main_value = getattr( sys.modules[ "__main__" ], value )
if inspect.isfunction( main_value ):
print main_value, main_value.func_code, main_value.func_code.co_varnames[:main_value.func_code.co_argcount] # inspect.getargs( main_value.func_code )
# TODO: Make this work as well, currently disabled, because of nested arguments not
# being compatible yet.
# print main_value, main_value.func_code.co_varnames, inspect.getargspec( main_value )
pass
Nuitka-0.5.0.1/tests/basics/TryContinueFinally.py 0000644 0001750 0001750 00000003527 12265264105 022134 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def tryWhileContinueFinallyTest():
print "Check if finally is executed in a continue using for loop:"
x = 0
while x < 10:
x += 1
try:
if x % 2 == 1:
continue
finally:
print x,
print
def tryForContinueFinallyTest():
print "Check if finally is executed in a continue using for loop:"
for x in range(10):
try:
if x % 2 == 1:
continue
finally:
print x,
print
def tryWhileBreakFinallyTest():
print "Check if finally is executed in a break using while loop:"
x = 0
while x < 10:
x += 1
try:
if x == 5:
break
finally:
print x,
print
def tryForBreakFinallyTest():
print "Check if finally is executed in a break using for loop:"
for x in range(10):
try:
if x == 5:
break
finally:
print x,
print
tryWhileContinueFinallyTest()
tryWhileBreakFinallyTest()
tryForContinueFinallyTest()
tryForBreakFinallyTest()
Nuitka-0.5.0.1/tests/basics/BuiltinSuper.py 0000644 0001750 0001750 00000006057 12265264105 020760 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import print_function
# Python2 will fallback to this variable, which Python3 will ignore.
__class__ = "Using module level __class__ variable, wrong for Python3"
class ClassWithUnderClassClosure:
def g(self):
def h():
print( "Variable __class__ in ClassWithUnderClassClosure is", __class__)
h()
try:
print( "ClassWithUnderClassClosure: Super in ClassWithUnderClassClosure is", super() )
except Exception as e:
print( "ClassWithUnderClassClosure: Occured during super call", repr(e) )
ClassWithUnderClassClosure().g()
class ClassWithoutUnderClassClosure:
def g(self):
__class__ = "Providing __class__ ourselves, then it must be used"
print( __class__)
try:
print( "ClassWithoutUnderClassClosure: Super", super() )
except Exception as e:
print( "ClassWithoutUnderClassClosure: Occured during super call", repr(e) )
ClassWithoutUnderClassClosure().g()
# For Python2 only.
__class__ = "Global __class__"
def deco(C):
print( "Decorating", repr( C ) )
class D(C):
pass
return D
@deco
class X:
__class__ = "some string"
def f1(self):
print( "f1", locals() )
try:
print( "f1", __class__ )
except Exception as e:
print( "Accessing __class__ in f1 gave", repr(e) )
def f2(self):
print( "f2", locals() )
def f3(self):
print( "f3", locals() )
super
def f4(self):
print( "f4", self )
self = X()
print( "f4", self )
try:
print( "f4", super() )
print( "f4", super().__self__ )
except TypeError:
import sys
assert sys.version < (3,)
f5 = lambda x: __class__
def f6(self_by_another_name):
try:
print( "f6", super() )
except TypeError:
import sys
assert sys.version < (3,)
def f7(self):
try:
yield super()
except TypeError:
import sys
assert sys.version < (3,)
print( "Early pre-class calls begin" )
print( "Set in class __class__", __class__ )
f1( 1 )
f2( 2 )
f3( 3 )
print( "Early pre-class calls end" )
del __class__
x = X()
x.f1()
x.f2()
x.f3()
x.f4()
print( "f5", x.f5() )
x.f6()
print( "f7", list( x.f7() ) )
Nuitka-0.5.0.1/tests/basics/Empty.py 0000644 0001750 0001750 00000001401 12265264105 017415 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/tests/basics/Referencing32.py 0000644 0001750 0001750 00000005607 12265264105 020727 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys, gc
def simpleFunction1():
def abc(*, exc=IOError):
pass
for _ in range(100):
abc()
def simpleFunction2():
def abc(*, exc=IOError):
raise ValueError from None
try:
abc()
except (ValueError, TypeError):
pass
def simpleFunction3():
a = 1
def nonlocal_writer():
nonlocal a
for a in range(10):
pass
nonlocal_writer()
assert a == 9, a
m1 = {}
m2 = {}
def snapObjRefCntMap(before):
if before:
global m1
m = m1
else:
global m2
m = m2
for x in gc.get_objects():
if x is m1:
continue
if x is m2:
continue
m[ str( x ) ] = sys.getrefcount( x )
def checkReferenceCount(checked_function, max_rounds = 10):
assert sys.exc_info() == ( None, None, None ), sys.exc_info()
print( checked_function.__name__ + ":", end = "" )
ref_count1 = 17
ref_count2 = 17
explain = False
for count in range( max_rounds ):
x1 = 0
x2 = 0
gc.collect()
ref_count1 = sys.gettotalrefcount()
if explain and count == max_rounds - 1:
snapObjRefCntMap( True )
checked_function()
assert sys.exc_info() == ( None, None, None ), sys.exc_info()
gc.collect()
if explain and count == max_rounds - 1:
snapObjRefCntMap( False )
ref_count2 = sys.gettotalrefcount()
if ref_count1 == ref_count2:
print( "PASSED" )
break
# print count, ref_count1, ref_count2
else:
print( "FAILED", ref_count1, ref_count2, "leaked", ref_count2 - ref_count1 )
if explain:
assert m1
assert m2
for key in m1.keys():
if key not in m2:
print( "*" * 80 )
print( key )
elif m1[key] != m2[key]:
print( "*" * 80 )
print( key )
else:
pass
# print m1[key]
assert sys.exc_info() == ( None, None, None ), sys.exc_info()
gc.collect()
checkReferenceCount( simpleFunction1 )
checkReferenceCount( simpleFunction2 )
checkReferenceCount( simpleFunction3 )
Nuitka-0.5.0.1/tests/basics/ComparisonChains.py 0000644 0001750 0001750 00000007005 12265264105 021565 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def simple_comparisons(x, y):
if 'a' <= x <= y <= 'z':
print "One"
if 'a' <= x <= 'z':
print "Two"
if 'a' <= x > 'z':
print "Three"
print "Simple comparisons:"
simple_comparisons( "c", "d" )
def side_effect():
print ""
return 7
def side_effect_comparisons():
print "Should have side effect:"
print 1 < side_effect() < 9
print "Should not have side effect due to short circuit"
print 3 < 2 < side_effect() < 9
print "Check for expected side effects only:"
side_effect_comparisons()
def function_torture_is():
a = ( 1, 2, 3 )
for x in a:
for y in a:
for z in a:
print x, y, z, ":", x is y is z, x is not y is not z
function_torture_is()
print "Check if lambda can have expression chains:"
def function_lambda_with_chain():
a = ( 1, 2, 3 )
x = lambda x : x[0] < x[1] < x[2]
print x(a)
function_lambda_with_chain()
def generator_function_with_chain():
x = ( 1, 2, 3 )
yield x[0] < x[1] < x[2]
print list( generator_function_with_chain() )
def contraction_with_chain():
return [ x[0] < x[1] < x[2] for x in [ ( 1, 2, 3 ) ] ]
print contraction_with_chain()
def genexpr_with_chain():
return ( x[0] < x[1] < x[2] for x in [ ( 1, 2, 3 ) ] )
print list( genexpr_with_chain() )
class class_with_chain:
x = ( 1, 2, 3 )
print x[0] < x[1] < x[2]
x = ( 1, 2, 3 )
print x[0] < x[1] < x[2]
class CustomOps(int):
def __lt__(self, other):
print "<", self, other
return True
def __gt__(self, other):
print ">", self, other
return False
print "Custom ops, to enforce chain eval order and short circuit:"
print CustomOps( 7 ) < CustomOps( 8 ) > CustomOps( 6 )
print "Custom ops, do short circuit:"
print CustomOps( 8 ) > CustomOps( 7 ) < CustomOps( 6 )
def inOperatorChain():
print "In operator chains:"
print 3 in [3,4] in [[3,4]]
print 3 in [3,4] not in [[3,4]]
if 3 in [3,4] in [[3,4]]:
print "Yes"
else:
print "No"
if 3 in [3,4] not in [[3,4]]:
print "Yes"
else:
print "No"
inOperatorChain()
# Make sure the values are called and order is correct:
class A(object):
def __init__(self, name, value):
self.name = name
self.value = value
def __repr__(self):
return "" % ( self.name, self.value )
def __lt__(self, other):
print "less than called for:", self, other, self.value, other.value, self.value < other.value
if self.value < other.value:
print "good"
return 7
else:
print "bad"
return 0
a = A("a",1)
b = A("b",2)
c = A("c",0)
print a < b < c
print "*" * 80
a = A("a",2)
b = A("b",1)
c = A("c",0)
print a < b < c
Nuitka-0.5.0.1/tests/basics/GeneratorExpressions.py 0000644 0001750 0001750 00000012344 12265264105 022520 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print "Generator expression that demonstrates the timing:"
def iteratorCreationTiming():
def getIterable(x):
print "Getting iterable", x
return Iterable( x )
class Iterable:
def __init__(self, x):
self.x = x
self.values = range( x )
self.count = 0
def __iter__(self):
print "Giving iter now", self.x
return self
def next(self):
print "Next of", self.x, "is", self.count
if len( self.values ) > self.count:
self.count += 1
return self.values[ self.count - 1 ]
else:
print "Raising StopIteration for", self.x
raise StopIteration
def __del__(self):
print "Deleting", self.x
gen = ( (y,z) for y in getIterable( 3 ) for z in getIterable( 2 ) )
print "Using generator", gen
gen.next()
res = tuple( gen )
print res
print "*" * 20
try:
gen.next()
except StopIteration:
print "Use past end gave StopIteration as expected"
try:
import inspect
print "Generator state then is", inspect.getgeneratorstate( gen )
except AttributeError:
pass
print "Its frame is now", gen.gi_frame
print "Early aborting generator"
gen2 = ( (y,z) for y in getIterable( 3 ) for z in getIterable( 2 ) )
del gen2
iteratorCreationTiming()
print "Generator expressions that demonstrate the use of conditions:"
print tuple( x for x in range(8) if x % 2 == 1 )
print tuple( x for x in range(8) if x % 2 == 1 for z in range(8) if z == x )
print tuple( x for (x,y) in zip(range(2),range(4)))
print "Directory of generator expressions:"
for_dir = ( x for x in [1] )
gen_dir = dir( for_dir )
print sorted( g for g in gen_dir )
def genexprSend():
x = ( x for x in range(9) )
print "Sending too early:"
try:
x.send(3)
except TypeError, e:
print "Gave expected TypeError with text:", e
z = x.next()
y = x.send(3)
print "Send return value", y
print "And then next gave", x.next()
print "Throwing an exception to it."
try:
x.throw( 2, 2, None )
assert False
except TypeError, e:
print "Gave expected TypeError:", e
print "Throwing an exception to it."
try:
x.throw( ValueError, 2, None )
except ValueError, e:
print "Gave expected ValueError:", e
try:
x.next()
print "Next worked even after thrown error"
except StopIteration, e:
print "Gave expected stop iteration after throwing exception in it:", e
print "Throwing another exception from it."
try:
x.throw( ValueError, 5, None )
except ValueError, e:
print "Gave expected ValueError with text:", e
print "Generator expressions have send too:"
genexprSend()
def genexprClose():
x = ( x for x in range(9) )
print "Immediate close:"
x.close()
print "Closed once"
x.close()
print "Closed again without any trouble"
genexprClose()
def genexprThrown():
def checked(z):
if z == 3:
raise ValueError
return z
x = ( checked( x ) for x in range(9) )
try:
for count, value in enumerate( x ):
print count, value
except ValueError:
print count+1, ValueError
try:
x.next()
print "Allowed to do next() after raised exception from the generator expression"
except StopIteration:
print "Exception in generator, disallowed next() afterwards."
genexprThrown()
def nestedExpressions():
a = [x for x in range(10)]
b = (x for x in (y for y in a))
print "nested generator expression", list(b)
nestedExpressions()
def lambdaGenerators():
a = 1
x = lambda : (yield a)
print "Simple lambda generator", x, x(), list( x() )
y = lambda : ((yield 1),(yield 2))
print "Complex lambda generator", y, y(), list( y() )
lambdaGenerators()
def functionGenerators():
# Like lambdaGenerators, to show how functions behave differently if at all.
a = 1
def x():
yield a
print "Simple function generator", x, x(), list( x() )
def y():
yield((yield 1),(yield 2))
print "Complex function generator", y, y(), list( y() )
functionGenerators()
def strangeLambdaGeneratorExpression():
x = ((yield) for i in (1,2) if (yield))
print "Strange lamba generator expression"
print list(x)
strangeLambdaGeneratorExpression()
Nuitka-0.5.0.1/tests/basics/LateClosureAssignment.py 0000644 0001750 0001750 00000004666 12265264105 022612 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def closureTest1():
# Assign, but the value is not supposed to be used by the function, instead the later
# update is effective.
d = 1
def subby():
return d
d = 22222*2222
return subby()
def closureTest2():
# Using a closure variable that is not initialized at the time it is closured should
# work as well.
def subby():
return d
d = 2222*2222
return subby()
def closureTest3():
def subby():
return d
try:
return subby()
except NameError:
return 88
d = 1
def scopeTest4():
try:
return d
d = 1
except UnboundLocalError, e:
return repr(e)
print "Test closure where value is overwritten:", closureTest1()
print "Test closure where value is assigned only late:", closureTest2()
print "Test function where closured value is never assigned:", closureTest3()
print "Scope test where UnboundLocalError is expected:", scopeTest4()
def function():
pass
class ClosureLocalizerClass:
print "Function before assigned in a class", function
function = 1
print "Function after it was assigned in class", function
ClosureLocalizerClass()
def ClosureLocalizerFunction():
try:
function = function
print "Function didn't give unbound local error"
except UnboundLocalError, e:
print "Function gave unbound local error when accessing function before assignment.", repr(e)
ClosureLocalizerFunction()
class X:
def __init__(self, x):
self.x = x
def changingClosure():
a = 1
def closureTaker():
return X(a)
x = closureTaker()
a=2
print x.x
x = closureTaker()
print x.x
changingClosure()
Nuitka-0.5.0.1/tests/basics/ExecEval.py 0000644 0001750 0001750 00000013633 12265264105 020025 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import tempfile, sys, os
print "eval 3+3=", eval("3+3")
print "eval 3+3=", eval(" 3+3")
def functionEval1():
return eval(" 3+3")
print "eval in a function with nothing provided", functionEval1()
def functionEval2():
a = [2]
g = {}
r = eval( "1+3", g )
return r, g.keys()
print "eval in a function with globals provided", functionEval2()
def functionEval3():
result = []
for x in eval( "(1,2)"):
result.append( x )
return result
print "eval in a for loop as iterator giver", functionEval3()
print "exec on a global level",
exec( "d=2+2" )
print "2+2=",d
def functionExec1():
a = 1
code = "a=2"
exec( code )
return a
def functionExec2():
a = 1
code = "a=2"
exec code in globals(), locals()
return a
print "exec in function without and with locals() provided:", functionExec1(), functionExec2()
tmp_filename = tempfile.gettempdir() + "/execfile.py"
f = open( tmp_filename, "w" )
f.write( "e=7\nf=8\n" )
f.close()
execfile( tmp_filename )
print "execfile with defaults f,g=", e, f
global_vars = { 'e' : '0', 'f' : 0 }
local_vars = dict( global_vars )
execfile( tmp_filename, global_vars )
print "execfile with globals dict:", global_vars.keys()
execfile( tmp_filename, global_vars, local_vars )
print "execfile with globals and locals dict:", local_vars
def functionExecfile():
e = 0
f = 0
global_vars = { 'e' : '0', 'f' : 0 }
local_vars = dict( global_vars )
print "execfile with globals and locals dict in a function:",
print execfile( tmp_filename, global_vars, local_vars ),
print global_vars.keys(), local_vars, e, f
functionExecfile()
class classExecfile:
e = 0
f = 0
print "execfile in a class:",
# TODO: Won't work yet, Issue#5
# print execfile( tmp_filename ),
execfile( tmp_filename )
print e, f
def functionExecNones():
f = 0
exec( "f=1", None, None )
print "Exec with None as tuple args did update locals:", f
exec "f=2" in None, None
print "Exec with None as normal args did update locals:", f
functionExecNones()
def functionEvalNones2():
f = 11
code = "f"
g = None
l = None
f1 = eval ( code, l, g )
print "Eval with None arguments from variables did access locals:", f1
functionEvalNones2()
def functionExecNones2():
f = 0
code = "f=1"
g = None
l = None
exec ( code, l, g )
print "Exec with None as tuple args from variable did update locals:", f
code = "f=2"
exec code in l, g
print "Exec with None as normal args did update locals:", f
functionExecNones2()
print "Exec with a future division definition and one without:"
exec """
from __future__ import division
from __future__ import print_function
print( "3/2 is with future division", 3/2 )
"""
exec """
from __future__ import print_function
print( "3/2 is without future division", 3/2 )
"""
x = 1
y = 1
def functionGlobalsExecShadow():
global x
print "Global x outside is", x
y = 0
print "Local y is initially", y
print "Locals initially", locals()
exec """
from __future__ import print_function
x = 2
print( "Exec local x is", x )
"""
print "Function global x is", x
exec """
from __future__ import print_function
print( "Re-exec local x", x )
"""
print "Locals after exec assigning to local x", locals()
exec """
from __future__ import print_function
global x
x = 3
print( "Exec global x is", x )
"""
print "Exec level global x is", x
exec """
from __future__ import print_function
def change_y():
global y
y = 4
print( "Exec function global y is", y )
y = 7
change_y()
# TODO: The below will not work
print( "Exec local y is", y )
"""
# print "Local y is afterwards", y
def print_global_y():
global y
# TODO: The below will not work
print "Global y outside", y
print_global_y()
print "Outside y", y
functionGlobalsExecShadow()
def functionWithClosureProvidedByExec():
code = "ValueError = TypeError"
exec code in None, None
def func( ):
print "Closure from exec not used", ValueError
func()
functionWithClosureProvidedByExec()
x = 2
def functionWithExecAffectingClosure():
x = 4
code = "d=3"
space = locals()
exec code in space
def closureMaker():
return x
return d, closureMaker()
print "Closure in a function with exec to not none", functionWithExecAffectingClosure()
def generatorFunctionWithExec():
yield 1
code = "y = 2"
exec code
yield y
print "Exec in a generator function", tuple( generatorFunctionWithExec() )
def evalInContractions():
r1 = list( eval( str( s ) ) for s in range( 3 ) )
r2 = [ eval( str( s ) ) for s in range( 4 ) ]
return r1, r2
print "Eval in a list contraction or generator expression", evalInContractions()
def execDefinesFunctionToLocalsExplicity():
exec """\
def makeAddPair(a, b):
def addPair(c, d):
return (a + c, b + d)
return addPair
""" in locals()
if sys.version_info < (3,):
assert makeAddPair
return "yes"
print "Exec adds functions declares in explicit locals() given.", execDefinesFunctionToLocalsExplicity()
os.unlink( tmp_filename )
Nuitka-0.5.0.1/tests/programs/ 0000755 0001750 0001750 00000000000 12265271051 016335 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/imports/ 0000755 0001750 0001750 00000000000 12265271051 020032 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/imports/some_package/ 0000755 0001750 0001750 00000000000 12265271051 022450 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/imports/some_package/Child2.py 0000644 0001750 0001750 00000002056 12265264105 024134 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "*** Child2: Begin", __name__ )
try:
import Child1
except ImportError:
print( "This must be Python3, doing local import then." )
from . import Child1
print( "*** Child2: Child2 is in", __package__ )
print( "*** Child2: Imported nearby child", Child1 )
print( "*** Child2: End" )
Nuitka-0.5.0.1/tests/programs/imports/some_package/Child3.py 0000644 0001750 0001750 00000001473 12265264105 024137 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "*** Child3: Begin" )
print( "*** Child3: End" )
Nuitka-0.5.0.1/tests/programs/imports/some_package/Child1.py 0000644 0001750 0001750 00000001621 12265264105 024130 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "*** Child1: Begin" )
from . import Child3 as localname
print( "*** Child1: Imported Child3", localname )
print( "*** Child1: End" )
Nuitka-0.5.0.1/tests/programs/imports/some_package/__init__.py 0000644 0001750 0001750 00000001610 12265264105 024561 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "*** some_package: Coming from '%s'" % __file__.replace( ".pyc", ".py" ) )
print( "*** some_package: Path is '%s'" % __path__ )
Nuitka-0.5.0.1/tests/programs/imports/Main.py 0000644 0001750 0001750 00000001716 12265264105 021277 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "*** Main: Importing" )
import some_package.Child2
print( "*** Main: Imported" )
print( "*** Main: Some package", some_package )
print( "*** Main: Imported package child", some_package.Child2 )
Nuitka-0.5.0.1/tests/programs/plugin_import/ 0000755 0001750 0001750 00000000000 12265271051 021225 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/plugin_import/PluginImportMain.py 0000644 0001750 0001750 00000001534 12265264105 025042 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
name = "some_module"
module = getattr(__import__('some_package', fromlist=[name]), name)
Nuitka-0.5.0.1/tests/programs/plugin_import/some_package/ 0000755 0001750 0001750 00000000000 12265271051 023643 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/plugin_import/some_package/some_module.py 0000644 0001750 0001750 00000001417 12265264105 026532 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "hi" )
Nuitka-0.5.0.1/tests/programs/plugin_import/some_package/__init__.py 0000644 0001750 0001750 00000001403 12265264105 025754 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#
Nuitka-0.5.0.1/tests/programs/main_raises2/ 0000755 0001750 0001750 00000000000 12265271051 020711 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/main_raises2/ErrorInFunctionMain.py 0000644 0001750 0001750 00000002076 12265264105 025165 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Just plain exception from the function level, supposed to report the correct file and line
def generator_function():
import ErrorRaising
x = ( lambda : ErrorRaising.raiseException() for z in range(3) )
next( x )()
def normal_function():
y = generator_function()
y()
normal_function()
Nuitka-0.5.0.1/tests/programs/main_raises2/ErrorRaising.py 0000644 0001750 0001750 00000001452 12265264105 023675 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def raiseException():
return 1 / 0
Nuitka-0.5.0.1/tests/programs/dash_import/ 0000755 0001750 0001750 00000000000 12265271051 020646 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/dash_import/DashImportMain.py 0000644 0001750 0001750 00000001571 12265270513 024105 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
__import__("dash-module")
b = "dash-module"
__import__(b)
__import__("plus+module")
c = "plus+module"
__import__(c)
Nuitka-0.5.0.1/tests/programs/dash_import/dash-module.py 0000644 0001750 0001750 00000001464 12265264105 023431 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "Module with dash imported as", __name__ )
Nuitka-0.5.0.1/tests/programs/dash_import/plus+module.py 0000644 0001750 0001750 00000001464 12265270513 023473 0 ustar hayen hayen 0000000 0000000 # Copyright 2014, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "Module with plus imported as", __name__ )
Nuitka-0.5.0.1/tests/programs/with space/ 0000755 0001750 0001750 00000000000 12265271051 020364 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/with space/Space Main.py 0000755 0001750 0001750 00000001474 12265264105 022651 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "Hello from main program with space in its paths" )
Nuitka-0.5.0.1/tests/programs/package_contains_main/ 0000755 0001750 0001750 00000000000 12265271051 022632 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/package_contains_main/local.py 0000644 0001750 0001750 00000001442 12265264105 024301 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print ("Imported local package")
Nuitka-0.5.0.1/tests/programs/package_contains_main/PackageContainsMain.py 0000644 0001750 0001750 00000001426 12265264105 027050 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from . import local
Nuitka-0.5.0.1/tests/programs/package_contains_main/__init__.py 0000644 0001750 0001750 00000001401 12265264105 024741 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/tests/programs/dash_main/ 0000755 0001750 0001750 00000000000 12265271051 020260 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/dash_main/Dash-Main.py 0000644 0001750 0001750 00000001453 12265264105 022400 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "This is running from",__file__ )
Nuitka-0.5.0.1/tests/programs/run_all.py 0000755 0001750 0001750 00000007145 12265264105 020357 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import os, sys
# Find common code relative in file system. Not using packages for test stuff.
sys.path.insert(
0,
os.path.normpath(
os.path.join(
os.path.dirname(os.path.abspath(__file__)),
".."
)
)
)
from test_common import (
my_print,
setup,
convertUsing2to3,
compareWithCPython
)
python_version = setup()
search_mode = len( sys.argv ) > 1 and sys.argv[1] == "search"
start_at = sys.argv[2] if len( sys.argv ) > 2 else None
if start_at:
active = False
else:
active = True
extra_options = os.environ.get("NUITKA_EXTRA_OPTIONS","")
for filename in sorted(os.listdir( "." )):
if not os.path.isdir(filename) or filename.endswith(".build"):
continue
path = os.path.relpath( filename )
if not active and start_at in ( filename, path ):
active = True
expected_errors = [
"module_exits", "main_raises", "main_raises2",
"package_contains_main"
]
# Allowed after Python3, packages need no more "__init__.py"
if python_version < b"3.3":
expected_errors.append( "package_missing_init" )
if filename not in expected_errors:
extra_flags = [ "expect_success" ]
else:
extra_flags = [ "expect_failure" ]
if filename in ( "package_missing_init", "dash_import", "reimport_main" ):
extra_flags.append( "ignore_stderr" )
extra_flags.append( "remove_output" )
# Cannot include the files with syntax errors, these would then become
# ImportError, but that's not the test. In all other cases, use two
# step execution, which will not add the program original source to
# PYTHONPATH.
if filename != "syntax_errors":
extra_flags.append("two_step_execution")
else:
extra_flags.append("binary_python_path")
if filename == "plugin_import":
os.environ[ "NUITKA_EXTRA_OPTIONS" ] = extra_options + \
" --recurse-all --recurse-directory=%s/some_package" % (
os.path.abspath( filename )
)
else:
os.environ[ "NUITKA_EXTRA_OPTIONS" ] = extra_options + " --recurse-all"
if active:
my_print( "Consider output of recursively compiled program:", path )
for filename_main in os.listdir( filename ):
if filename_main.endswith( "Main.py" ):
break
if filename_main.endswith( "Main" ):
break
else:
sys.exit(
"""\
Error, no file ends with 'Main.py' or 'Main' in %s, incomplete test case.""" % (
filename
)
)
compareWithCPython(
path = os.path.join( filename, filename_main ),
extra_flags = extra_flags,
search_mode = search_mode,
needs_2to3 = False
)
else:
my_print( "Skipping", filename )
Nuitka-0.5.0.1/tests/programs/Issue16/ 0000755 0001750 0001750 00000000000 12265271051 017574 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/Issue16/err/ 0000755 0001750 0001750 00000000000 12265271051 020364 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/Issue16/err/CrasherModule16.py 0000644 0001750 0001750 00000001576 12265264105 023655 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# $Id$
class BaseOptions(object):
_save_attr = ()
if __debug__:
if __name__=='__main__':
o = BaseOptions()
Nuitka-0.5.0.1/tests/programs/Issue16/err/__init__.py 0000644 0001750 0001750 00000001401 12265264105 022473 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/tests/programs/Issue16/Issue16Main.py 0000644 0001750 0001750 00000001441 12265264105 022214 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from err import CrasherModule16
Nuitka-0.5.0.1/tests/programs/module_exits/ 0000755 0001750 0001750 00000000000 12265271051 021036 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/module_exits/ErrorExitingModule.py 0000644 0001750 0001750 00000001555 12265264105 025207 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys
print( type( __builtins__ ) )
sys.exit("Module doing sys.exit")
print( "This won't happen!" )
Nuitka-0.5.0.1/tests/programs/module_exits/Main.py 0000644 0001750 0001750 00000001532 12265264105 022277 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( type(__builtins__) )
import ErrorExitingModule
print( "Should not get here!" )
Nuitka-0.5.0.1/tests/programs/package_program/ 0000755 0001750 0001750 00000000000 12265271051 021457 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/package_program/PackageAsMain/ 0000755 0001750 0001750 00000000000 12265271051 024103 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/package_program/PackageAsMain/__main__.py 0000644 0001750 0001750 00000001511 12265264105 026175 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys
print( "Hello world!", __name__, sys.modules[ __name__ ] )
Nuitka-0.5.0.1/tests/programs/relative_import/ 0000755 0001750 0001750 00000000000 12265271051 021542 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/relative_import/dircache.py 0000644 0001750 0001750 00000001401 12265264105 023654 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/tests/programs/relative_import/RelativeImportMain.py 0000644 0001750 0001750 00000001514 12265264105 025672 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import absolute_import
import dircache
print( dircache )
Nuitka-0.5.0.1/tests/programs/package_missing_init/ 0000755 0001750 0001750 00000000000 12265271051 022504 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/package_missing_init/some_package/ 0000755 0001750 0001750 00000000000 12265271051 025122 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/package_missing_init/some_package/some_module.py 0000644 0001750 0001750 00000001655 12265264105 030015 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "This must be Python3.3, which no longer needs __init__.py to accept a package." )
import sys
print( "The parent path is", sys.modules[ "some_package"].__path__ )
Nuitka-0.5.0.1/tests/programs/package_missing_init/Main.py 0000644 0001750 0001750 00000001441 12265264105 023744 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import some_package.some_module
Nuitka-0.5.0.1/tests/programs/stdlib_overload/ 0000755 0001750 0001750 00000000000 12265271051 021511 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/stdlib_overload/some_package/ 0000755 0001750 0001750 00000000000 12265271051 024127 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/stdlib_overload/some_package/star_importing.py 0000644 0001750 0001750 00000001674 12265264105 027554 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "some_package.star_importing, doing the star import" )
print( "Before", sorted( dir() ) )
from .pyexpat import *
lala = 1
print( "After", sorted( dir() ) )
print( "Finished" )
Nuitka-0.5.0.1/tests/programs/stdlib_overload/some_package/pyexpat.py 0000644 0001750 0001750 00000001454 12265264105 026201 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
defined_in_pyexpat_subpackage = "see me"
Nuitka-0.5.0.1/tests/programs/stdlib_overload/some_package/__init__.py 0000644 0001750 0001750 00000001401 12265264105 026236 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/tests/programs/stdlib_overload/some_package/normal_importing.py 0000644 0001750 0001750 00000001533 12265264105 030065 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import pyexpat
print( "Imported pyexpat, should use our one." )
print( dir( pyexpat ) )
Nuitka-0.5.0.1/tests/programs/stdlib_overload/StdlibOverloadMain.py 0000644 0001750 0001750 00000002224 12265264105 025607 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print("Main importing nearby package")
import pyexpat
try:
print(pyexpat.defined_in_pyexpat )
except AttributeError:
print("Must be Python3, where absolute imports are default.")
print( "Main importing from package doing star import" )
from some_package import star_importing
print( "Main importing from package doing normal import" )
from some_package import normal_importing
print( "Done." )
Nuitka-0.5.0.1/tests/programs/stdlib_overload/pyexpat.py 0000644 0001750 0001750 00000001441 12265264105 023557 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
defined_in_pyexpat = "see me"
Nuitka-0.5.0.1/tests/programs/syntax_errors/ 0000755 0001750 0001750 00000000000 12265271051 021257 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/syntax_errors/SyntaxErroring.py 0000644 0001750 0001750 00000001440 12265264105 024630 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
class x(metaclass=y):
pass
Nuitka-0.5.0.1/tests/programs/syntax_errors/Main.py 0000644 0001750 0001750 00000002065 12265264105 022522 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "Running as", __file__ )
try:
from SyntaxErroring import x
except Exception as e:
print( "Importing with syntax error gave", type(e), e )
try:
from IndentationErroring import x
except Exception as e:
print( "Importing with indentation error gave", type(e), e )
print( "Finished." )
Nuitka-0.5.0.1/tests/programs/syntax_errors/IndentationErroring.py 0000644 0001750 0001750 00000001427 12265264105 025623 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def f():
x
y
Nuitka-0.5.0.1/tests/programs/main_raises/ 0000755 0001750 0001750 00000000000 12265271051 020627 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/main_raises/ErrorMain.py 0000644 0001750 0001750 00000001617 12265264105 023106 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Just plain exception from the module level, supposed to report the correct file and line
import ErrorRaising
ErrorRaising.raiseException()
Nuitka-0.5.0.1/tests/programs/main_raises/ErrorRaising.py 0000644 0001750 0001750 00000001452 12265264105 023613 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
def raiseException():
return 1 / 0
Nuitka-0.5.0.1/tests/programs/package_overload/ 0000755 0001750 0001750 00000000000 12265271051 021623 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/package_overload/foo/ 0000755 0001750 0001750 00000000000 12265271051 022406 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/package_overload/foo/bar2.py 0000644 0001750 0001750 00000001401 12265264105 023604 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/tests/programs/package_overload/foo/bar.py 0000644 0001750 0001750 00000001401 12265264105 023522 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/tests/programs/package_overload/foo/__init__.py 0000644 0001750 0001750 00000001437 12265264105 024526 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
bar = 42
not_overloaded = 45
Nuitka-0.5.0.1/tests/programs/package_overload/Main.py 0000644 0001750 0001750 00000001567 12265264105 023074 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from foo import bar, not_overloaded
print( bar, not_overloaded )
from foo import bar2, not_overloaded
print( bar2 )
Nuitka-0.5.0.1/tests/programs/unicode_bom/ 0000755 0001750 0001750 00000000000 12265271051 020620 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/unicode_bom/unicode_bom.py 0000644 0001750 0001750 00000001521 12265264105 023456 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# import unicodedata
print( "This is from file with BOM unicode marker" )
Nuitka-0.5.0.1/tests/programs/unicode_bom/UnicodeBomMain.py 0000644 0001750 0001750 00000001472 12265264105 024031 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print ('Importing unicode BOM file:')
import unicode_bom
Nuitka-0.5.0.1/tests/programs/package_init_import/ 0000755 0001750 0001750 00000000000 12265271051 022345 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/package_init_import/some_package/ 0000755 0001750 0001750 00000000000 12265271051 024763 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/package_init_import/some_package/PackageLocal.py 0000644 0001750 0001750 00000001443 12265264105 027647 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "Imported PackageLocal" )
Nuitka-0.5.0.1/tests/programs/package_init_import/some_package/__init__.py 0000644 0001750 0001750 00000001620 12265264105 027075 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
try:
import PackageLocal
except ImportError:
print( "This must be Python3, doing local import then." )
from . import PackageLocal
Nuitka-0.5.0.1/tests/programs/package_init_import/Main.py 0000644 0001750 0001750 00000001472 12265264105 023611 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import some_package
print( some_package.PackageLocal )
Nuitka-0.5.0.1/tests/programs/reimport_main/ 0000755 0001750 0001750 00000000000 12265271051 021202 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/reimport_main/ImportItselfMain.py 0000644 0001750 0001750 00000001600 12265264105 025001 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys
print( "Here I am before import", __name__ )
import ImportItselfMain
print( "Here I am after import", __name__ )
Nuitka-0.5.0.1/tests/programs/deep/ 0000755 0001750 0001750 00000000000 12265271051 017252 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/deep/DeepProgramMain.py 0000644 0001750 0001750 00000001541 12265264105 022641 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import some_package.DeepChild
import some_package.deep_package.DeepDeepChild
print( "Done." )
Nuitka-0.5.0.1/tests/programs/deep/some_package/ 0000755 0001750 0001750 00000000000 12265271051 021670 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/deep/some_package/DeepChild.py 0000644 0001750 0001750 00000001550 12265264105 024066 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "Importing child." )
class A:
pass
print( "Class defined here, has these vars", vars(A ))
Nuitka-0.5.0.1/tests/programs/deep/some_package/DeepBrother.py 0000644 0001750 0001750 00000001660 12265264105 024452 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "This is deep brother module talking.", __name__ )
def someBrotherFunction():
pass
print( "The __module__ of function here is", someBrotherFunction.__module__ )
Nuitka-0.5.0.1/tests/programs/deep/some_package/__init__.py 0000644 0001750 0001750 00000001401 12265264105 023777 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/tests/programs/deep/some_package/deep_package/ 0000755 0001750 0001750 00000000000 12265271051 024260 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/deep/some_package/deep_package/DeepDeepChild.py 0000644 0001750 0001750 00000001510 12265264105 027250 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "This is DeepDeepChild talking." )
from .. import DeepBrother
Nuitka-0.5.0.1/tests/programs/deep/some_package/deep_package/__init__.py 0000644 0001750 0001750 00000001401 12265264105 026367 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/tests/programs/package_code/ 0000755 0001750 0001750 00000000000 12265271051 020722 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/package_code/some_package/ 0000755 0001750 0001750 00000000000 12265271051 023340 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/tests/programs/package_code/some_package/SomeModule.py 0000644 0001750 0001750 00000001454 12265264105 025771 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "Thanks for importing SomeModule" )
Nuitka-0.5.0.1/tests/programs/package_code/some_package/__init__.py 0000644 0001750 0001750 00000001453 12265264105 025456 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
print( "Thanks for importing", __name__ )
Nuitka-0.5.0.1/tests/programs/package_code/PackageInitCodeMain.py 0000644 0001750 0001750 00000001440 12265264105 025054 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Python tests originally created or extracted from other peoples work. The
# parts were too small to be protected.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import some_package.SomeModule
Nuitka-0.5.0.1/nuitka/ 0000755 0001750 0001750 00000000000 12265271051 014634 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/nodes/ 0000755 0001750 0001750 00000000000 12265271051 015744 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/nodes/ImportNodes.py 0000644 0001750 0001750 00000030776 12265264105 020600 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Nodes related to importing modules or names.
Normally imports are mostly relatively static, but Nuitka also attempts to
cover the uses of "__import__" builtin and other import techniques, that
allow dynamic values.
If other optimizations make it possible to predict these, the compiler can go
deeper that what it normally could. The import expression node can recurse. An
"__import__" builtin may be converted to it, once the module name becomes a
compile time constant.
"""
from .NodeBases import (
ExpressionChildrenHavingBase,
StatementChildrenHavingBase,
ExpressionMixin,
NodeBase
)
from .ConstantRefNodes import ExpressionConstantRef
from nuitka import Importing, Utils
from logging import warning
class ExpressionImportModule(NodeBase, ExpressionMixin):
kind = "EXPRESSION_IMPORT_MODULE"
# Set of modules, that we failed to import, and gave warning to the user
# about it.
_warned_about = set()
def __init__(self, module_name, import_list, level, source_ref):
NodeBase.__init__(
self,
source_ref = source_ref
)
self.module_name = module_name
self.import_list = import_list
self.level = level
self.module = None
self.attempted_recurse = False
self.found_modules = ()
def getDetails(self):
return {
"module_name" : self.module_name,
"level" : self.level
}
def getModuleName(self):
return self.module_name
def getImportList(self):
return self.import_list
def getLevel(self):
if self.level == 0:
if self.source_ref.getFutureSpec().isAbsoluteImport():
return 0
else:
return -1
else:
return self.level
def getModule(self):
return self.module
def setModule(self, module):
# Modules have no parent.
assert module.parent is None
self.module = module
def _consider(self, constraint_collection, module_filename, module_package):
assert module_package is None or \
(type(module_package) is str and module_package != "")
module_filename = Utils.normpath(module_filename)
if Utils.isDir(module_filename):
module_name = Utils.basename(module_filename)
module_kind = "py"
elif module_filename.endswith(".py"):
module_name = Utils.basename(module_filename)[:-3]
module_kind = "py"
elif module_filename.endswith(".so"):
module_kind = "shlib"
module_name = Utils.basename(module_filename)[:-3]
elif module_filename.endswith(".pyd"):
module_kind = "shlib"
module_name = Utils.basename(module_filename)[:-4]
else:
module_kind = None
module_name = None
if module_kind is not None:
from nuitka.tree import Recursion
decision, reason = Recursion.decideRecursion(
module_filename = module_filename,
module_name = module_name,
module_package = module_package,
module_kind = module_kind
)
if decision:
module_relpath = Utils.relpath(module_filename)
imported_module, added_flag = Recursion.recurseTo(
module_package = module_package,
module_filename = module_filename,
module_relpath = module_relpath,
module_kind = module_kind,
reason = reason
)
if added_flag:
constraint_collection.signalChange(
"new_code",
imported_module.getSourceReference(),
"Recursed to module."
)
return imported_module
elif decision is None and module_kind == "py":
if module_package is None:
module_fullpath = module_name
else:
module_fullpath = module_package + "." + module_name
if module_filename not in self._warned_about:
self._warned_about.add( module_filename )
warning(
"""\
Not recursing to '%(full_path)s' (%(filename)s), please specify \
--recurse-none (do not warn), \
--recurse-all (recurse to all), \
--recurse-not-to=%(full_path)s (ignore it), \
--recurse-to=%(full_path)s (recurse to it) to change.""" % {
"full_path" : module_fullpath,
"filename" : module_filename
}
)
def _attemptRecursion(self, constraint_collection):
assert self.getModule() is None
parent_module = self.getParentModule()
if parent_module.isPythonPackage():
parent_package = parent_module.getFullName()
else:
parent_package = self.getParentModule().getPackage()
module_package, _module_name, module_filename = Importing.findModule(
source_ref = self.source_ref,
module_name = self.getModuleName(),
parent_package = parent_package,
level = self.getLevel(),
warn = True
)
# That would be an illegal package name, catch it.
assert module_package != ""
if module_filename is not None:
imported_module = self._consider(
constraint_collection = constraint_collection,
module_filename = module_filename,
module_package = module_package
)
if imported_module is not None:
self.setModule(imported_module)
self.found_modules = []
import_list = self.getImportList()
if import_list and imported_module.isPythonPackage():
for import_item in import_list:
module_package, _module_name, module_filename = \
Importing.findModule(
source_ref = self.source_ref,
module_name = import_item,
parent_package = imported_module.getFullName(),
level = -1,
warn = False
)
if module_filename is not None:
sub_imported_module = self._consider(
constraint_collection = constraint_collection,
module_filename = module_filename,
module_package = module_package
)
if sub_imported_module is not None:
self.found_modules.append(sub_imported_module)
def computeExpression(self, constraint_collection):
# Attempt to recurse if not already done.
if not self.attempted_recurse:
self._attemptRecursion(
constraint_collection = constraint_collection
)
self.attempted_recurse = True
if self.getModule() is not None:
from nuitka.ModuleRegistry import addUsedModule
addUsedModule(self.getModule())
for found_module in self.found_modules:
addUsedModule(found_module)
# TODO: May return a module reference of some sort in the future with
# embedded modules.
return self, None, None
class ExpressionBuiltinImport(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_IMPORT"
named_children = (
"import_name", "globals", "locals", "fromlist", "level"
)
def __init__(self, name, import_globals, import_locals, fromlist, level,
source_ref):
if fromlist is None:
fromlist = ExpressionConstantRef(
constant = [],
source_ref = source_ref
)
if level is None:
level = 0 if source_ref.getFutureSpec().isAbsoluteImport() else -1
level = ExpressionConstantRef(
constant = level,
source_ref = source_ref
)
ExpressionChildrenHavingBase.__init__(
self,
values = {
"import_name" : name,
"globals" : import_globals,
"locals" : import_locals,
"fromlist" : fromlist,
"level" : level
},
source_ref = source_ref
)
getImportName = ExpressionChildrenHavingBase.childGetter("import_name")
getFromList = ExpressionChildrenHavingBase.childGetter("fromlist")
getGlobals = ExpressionChildrenHavingBase.childGetter("globals")
getLocals = ExpressionChildrenHavingBase.childGetter("locals")
getLevel = ExpressionChildrenHavingBase.childGetter("level")
def computeExpression(self, constraint_collection):
module_name = self.getImportName()
fromlist = self.getFromList()
level = self.getLevel()
# TODO: In fact, if the module is not a package, we don't have to insist
# on the fromlist that much, but normally it's not used for anything but
# packages, so it will be rare.
if module_name.isExpressionConstantRef() and \
fromlist.isExpressionConstantRef() and \
level.isExpressionConstantRef():
new_node = ExpressionImportModule(
module_name = module_name.getConstant(),
import_list = fromlist.getConstant(),
level = level.getConstant(),
source_ref = self.getSourceReference()
)
return (
new_node,
"new_import",
"Replaced __import__ call with module import expression."
)
# TODO: May return a module or module variable reference of some sort in
# the future with embedded modules.
return self, None, None
class StatementImportStar(StatementChildrenHavingBase):
kind = "STATEMENT_IMPORT_STAR"
named_children = ( "module", )
def __init__(self, module_import, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"module" : module_import
},
source_ref = source_ref
)
getModule = StatementChildrenHavingBase.childGetter( "module" )
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getModule() )
# Need to invalidate everything, and everything could be assigned to
# something else now.
constraint_collection.removeAllKnowledge()
return self, None, None
class ExpressionImportName(ExpressionChildrenHavingBase):
kind = "EXPRESSION_IMPORT_NAME"
named_children = ( "module", )
def __init__(self, module, import_name, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"module" : module
},
source_ref = source_ref
)
self.import_name = import_name
def getImportName(self):
return self.import_name
def getDetails(self):
return { "import_name" : self.getImportName() }
def getDetail(self):
return "import %s from %s" % ( self.getImportName(), self.getModule() )
getModule = ExpressionChildrenHavingBase.childGetter( "module" )
def computeExpression(self, constraint_collection):
# TODO: May return a module or module variable reference of some sort in
# the future with embedded modules.
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/OperatorNodes.py 0000644 0001750 0001750 00000021740 12265264105 021110 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Nodes for unary and binary operations.
No short-circuit involved, boolean 'not' is an unary operation like '-' is,
no real difference.
"""
from .NodeBases import ExpressionChildrenHavingBase
from nuitka import PythonOperators
import math
class ExpressionOperationBase(ExpressionChildrenHavingBase):
def __init__(self, operator, simulator, values, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = values,
source_ref = source_ref
)
self.operator = operator
self.simulator = simulator
def getDetail(self):
return self.operator
def getDetails(self):
return { "operator" : self.operator }
def getOperator(self):
return self.operator
def getSimulator(self):
return self.simulator
def isKnownToBeIterable(self, count):
# TODO: Could be true, if the arguments said so
return None
class ExpressionOperationBinary(ExpressionOperationBase):
kind = "EXPRESSION_OPERATION_BINARY"
named_children = ( "left", "right" )
def __init__(self, operator, left, right, source_ref):
assert left.isExpression() and right.isExpression, ( left, right )
ExpressionOperationBase.__init__(
self,
operator = operator,
simulator = PythonOperators.binary_operator_functions[ operator ],
values = {
"left" : left,
"right" : right
},
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
operator = self.getOperator()
operands = self.getOperands()
left, right = operands
if left.willRaiseException( BaseException ):
return (
left,
"new_raise",
"Left argument of binary operation raises exception"
)
if right.willRaiseException( BaseException ):
from .NodeMakingHelpers import wrapExpressionWithNodeSideEffects
result = wrapExpressionWithNodeSideEffects(
new_node = right,
old_node = left
)
return (
result,
"new_raise",
"Right argument of binary operation raises exception"
)
if left.isCompileTimeConstant() and right.isCompileTimeConstant():
left_value = left.getCompileTimeConstant()
right_value = right.getCompileTimeConstant()
if operator == "Mult" and right.isNumberConstant():
iter_length = left.getIterationLength()
if iter_length is not None:
if iter_length * right_value > 256:
return self, None, None
if left.isNumberConstant():
if left.isIndexConstant() and right.isIndexConstant():
# Estimate with logarithm, if the result of number
# calculations is computable with acceptable effort,
# otherwise, we will have to do it at runtime.
if left_value != 0 and right_value != 0:
if math.log10( abs( left_value ) ) + math.log10( abs( right_value ) ) > 20:
return self, None, None
elif operator == "Mult" and left.isNumberConstant():
iter_length = right.getIterationLength()
if iter_length is not None:
if iter_length * left_value > 256:
return self, None, None
elif operator == "Add" and \
left.isKnownToBeIterable( None ) and \
right.isKnownToBeIterable( None ):
iter_length = left.getIterationLength() + \
right.getIterationLength()
if iter_length > 256:
return self, None, None
from .NodeMakingHelpers import getComputationResult
return getComputationResult(
node = self,
computation = lambda : self.getSimulator()(
left_value,
right_value
),
description = "Operator '%s' with constant arguments." % operator
)
else:
return self, None, None
def getOperands(self):
return ( self.getLeft(), self.getRight() )
getLeft = ExpressionChildrenHavingBase.childGetter( "left" )
getRight = ExpressionChildrenHavingBase.childGetter( "right" )
class ExpressionOperationUnary(ExpressionOperationBase):
kind = "EXPRESSION_OPERATION_UNARY"
named_children = ( "operand", )
def __init__(self, operator, operand, source_ref):
assert operand.isExpression(), operand
ExpressionOperationBase.__init__(
self,
operator = operator,
simulator = PythonOperators.unary_operator_functions[ operator ],
values = {
"operand" : operand
},
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
operator = self.getOperator()
operand = self.getOperand()
if operand.isCompileTimeConstant():
operand_value = operand.getCompileTimeConstant()
from .NodeMakingHelpers import getComputationResult
return getComputationResult(
node = self,
computation = lambda : self.getSimulator()(
operand_value,
),
description = "Operator '%s' with constant argument." % operator
)
else:
return self, None, None
getOperand = ExpressionChildrenHavingBase.childGetter( "operand" )
def getOperands(self):
return ( self.getOperand(), )
class ExpressionOperationNOT(ExpressionOperationUnary):
kind = "EXPRESSION_OPERATION_NOT"
def __init__(self, operand, source_ref):
ExpressionOperationUnary.__init__(
self,
operator = "Not",
operand = operand,
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
operand = self.getOperand()
if operand.willRaiseException( BaseException ):
return (
operand,
"new_raise",
"Argument of 'not' operation raises exception"
)
return operand.computeExpressionOperationNot(
not_node = self,
constraint_collection = constraint_collection
)
def getTruthValue(self):
result = self.getOperand().getTruthValue()
# Need to invert the truth value of operand of course here.
return None if result is None else not result
def mayHaveSideEffects(self):
operand = self.getOperand()
if operand.mayHaveSideEffects():
return True
return operand.mayHaveSideEffectsBool()
def mayHaveSideEffectsBool(self):
return self.getOperand().mayHaveSideEffectsBool()
def extractSideEffects(self):
operand = self.getOperand()
# TODO: Find the common ground of these, and make it an expression
# method.
if operand.isExpressionMakeSequence():
return operand.extractSideEffects()
if operand.isExpressionMakeDict():
return operand.extractSideEffects()
return (self,)
def mayProvideReference(self):
# Dedicated code returns "True" or "False" only, which requires no
# reference, except for rich comparisons, which do.
return False
class ExpressionOperationBinaryInplace(ExpressionOperationBinary):
kind = "EXPRESSION_OPERATION_BINARY_INPLACE"
def __init__(self, operator, left, right, source_ref):
operator = "I" + operator
ExpressionOperationBinary.__init__(
self,
operator = operator,
left = left,
right = right,
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
# TODO: Inplace operation requires extra care to avoid corruption of
# values.
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/ConditionalNodes.py 0000644 0001750 0001750 00000031463 12265264105 021563 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Conditional nodes.
These is the conditional expression '(a if b else c)' and the conditional
statement, 'if a: ... else: ...' and there is no 'elif', because that is
expressed via nesting of conditional statements.
"""
from .NodeBases import ExpressionChildrenHavingBase, StatementChildrenHavingBase
# Delayed import into multiple branches is not an issue, pylint: disable=W0404
class ExpressionConditional(ExpressionChildrenHavingBase):
kind = "EXPRESSION_CONDITIONAL"
named_children = ( "condition", "expression_yes", "expression_no" )
def __init__(self, condition, yes_expression, no_expression, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"condition" : condition,
"expression_yes" : yes_expression,
"expression_no" : no_expression
},
source_ref = source_ref
)
def getBranches(self):
return (
self.getExpressionYes(),
self.getExpressionNo()
)
getExpressionYes = ExpressionChildrenHavingBase.childGetter(
"expression_yes"
)
getExpressionNo = ExpressionChildrenHavingBase.childGetter(
"expression_no"
)
getCondition = ExpressionChildrenHavingBase.childGetter(
"condition"
)
def computeExpressionRaw(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
# Query the truth value after the expression is evaluated, once it is
# evaluated in onExpression, it is known.
constraint_collection.onExpression(
expression = self.getCondition()
)
condition = self.getCondition()
# No need to look any further, if the condition raises, the branches do
# not matter at all.
if condition.willRaiseException(BaseException):
return condition, "new_raise", """\
Conditional statements already raises implicitely in condition, removing \
branches."""
# If the condition raises, we let that escape.
if condition.willRaiseException(BaseException):
return condition, "new_raise", """\
Conditional expression raises in condition."""
from nuitka.optimizations.ConstraintCollections import \
ConstraintCollectionBranch
# Decide this based on truth value of condition.
truth_value = condition.getTruthValue()
# TODO: We now know that condition evaluates to true for the yes branch
# and to not true for no branch, the branch should know that.
yes_branch = self.getExpressionYes()
# Continue to execute for yes branch unless we know it's not going to be
# relevant.
if truth_value is not False:
branch_yes_collection = ConstraintCollectionBranch(
parent = constraint_collection,
branch = yes_branch
)
# May have just gone away, so fetch it again.
yes_branch = self.getExpressionYes()
# If it's aborting, it doesn't contribute to merging.
if yes_branch.willRaiseException(BaseException):
branch_yes_collection = None
else:
branch_yes_collection = None
no_branch = self.getExpressionNo()
# Continue to execute for yes branch.
if truth_value is not True:
branch_no_collection = ConstraintCollectionBranch(
parent = constraint_collection,
branch = no_branch
)
# May have just gone away, so fetch it again.
no_branch = self.getExpressionNo()
# If it's aborting, it doesn't contribute to merging.
if no_branch.willRaiseException(BaseException):
branch_no_collection = None
else:
branch_no_collection = None
# Merge into parent execution.
constraint_collection.mergeBranches(
branch_yes_collection,
branch_no_collection
)
if truth_value is True:
from .NodeMakingHelpers import wrapExpressionWithNodeSideEffects
return (
wrapExpressionWithNodeSideEffects(
new_node = self.getExpressionYes(),
old_node = condition
),
"new_expression",
"Conditional expression predicted to yes case"
)
elif truth_value is False:
from .NodeMakingHelpers import wrapExpressionWithNodeSideEffects
return (
wrapExpressionWithNodeSideEffects(
new_node = self.getExpressionNo(),
old_node = condition
),
"new_expression",
"Conditional expression predicted to no case"
)
else:
return self, None, None
def mayHaveSideEffectsBool(self):
condition = self.getCondition()
if condition.mayHaveSideEffectsBool():
return True
if self.getExpressionYes().mayHaveSideEffectsBool():
return True
if self.getExpressionNo().mayHaveSideEffectsBool():
return True
return False
def mayProvideReference(self):
return self.getExpressionYes().mayProvideReference() or \
self.getExpressionNo().mayProvideReference()
class StatementConditional(StatementChildrenHavingBase):
kind = "STATEMENT_CONDITIONAL"
named_children = ( "condition", "yes_branch", "no_branch" )
def __init__(self, condition, yes_branch, no_branch, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"condition" : condition,
"yes_branch" : yes_branch,
"no_branch" : no_branch
},
source_ref = source_ref
)
getCondition = StatementChildrenHavingBase.childGetter( "condition" )
getBranchYes = StatementChildrenHavingBase.childGetter( "yes_branch" )
setBranchYes = StatementChildrenHavingBase.childSetter( "yes_branch" )
getBranchNo = StatementChildrenHavingBase.childGetter( "no_branch" )
setBranchNo = StatementChildrenHavingBase.childSetter( "no_branch" )
def isStatementAborting(self):
yes_branch = self.getBranchYes()
if yes_branch is not None:
if yes_branch.isStatementAborting():
no_branch = self.getBranchNo()
if no_branch is not None:
return no_branch.isStatementAborting()
else:
return False
else:
return False
else:
return False
def computeStatement(self, constraint_collection):
# This is rather complex stuff, pylint: disable=R0912
# Query the truth value after the expression is evaluated, once it is
# evaluated in onExpression, it is known.
constraint_collection.onExpression(
expression = self.getCondition()
)
condition = self.getCondition()
# No need to look any further, if the condition raises, the branches do
# not matter at all.
if condition.willRaiseException(BaseException):
from .NodeMakingHelpers import makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = condition,
node = self
)
return result, "new_raise", """\
Conditional statements already raises implicitely in condition, removing \
branches."""
from nuitka.optimizations.ConstraintCollections import \
ConstraintCollectionBranch
# Consider to not execute branches that we know to be true, but execute
# the ones that may be true, potentially both.
truth_value = condition.getTruthValue()
# TODO: We now know that condition evaluates to true for the yes branch
# and to not true for no branch, the branch should know that.
yes_branch = self.getBranchYes()
# Handle branches that became empty behind our back
if yes_branch is not None:
if not yes_branch.getStatements():
yes_branch = None
# Continue to execute for yes branch unless we know it's not going to be
# relevant.
if yes_branch is not None and truth_value is not False:
branch_yes_collection = ConstraintCollectionBranch(
parent = constraint_collection,
branch = yes_branch
)
# May have just gone away, so fetch it again.
yes_branch = self.getBranchYes()
# If it's aborting, it doesn't contribute to merging.
if yes_branch is None or yes_branch.isStatementAborting():
branch_yes_collection = None
else:
branch_yes_collection = None
no_branch = self.getBranchNo()
# Handle branches that became empty behind our back
if no_branch is not None:
if not no_branch.getStatements():
no_branch = None
# Continue to execute for yes branch.
if no_branch is not None and truth_value is not True:
branch_no_collection = ConstraintCollectionBranch(
parent = constraint_collection,
branch = no_branch
)
# May have just gone away, so fetch it again.
no_branch = self.getBranchNo()
# If it's aborting, it doesn't contribute to merging.
if no_branch is None or no_branch.isStatementAborting():
branch_no_collection = None
else:
branch_no_collection = None
# Merge into parent execution.
constraint_collection.mergeBranches(
branch_yes_collection,
branch_no_collection
)
# Both branches may have become empty.
if yes_branch is None and no_branch is None:
from .NodeMakingHelpers import \
makeStatementExpressionOnlyReplacementNode
# With both branches eliminated, the condition remains as a side
# effect.
result = makeStatementExpressionOnlyReplacementNode(
expression = condition,
node = self
)
return result, "new_statements", """\
Both branches have no effect, reduced to evaluate condition."""
if yes_branch is None:
# Would be eliminated already, if there wasn't any "no" branch
# either.
assert no_branch is not None
from .OperatorNodes import ExpressionOperationNOT
new_statement = StatementConditional(
condition = ExpressionOperationNOT(
operand = condition,
source_ref = condition.getSourceReference()
),
yes_branch = no_branch,
no_branch = None,
source_ref = self.getSourceReference()
)
return new_statement, "new_statements", """\
Empty 'yes' branch for condition was replaced with inverted condition check."""
# Note: Checking the condition late, so that the surviving branch got
# processed already. Returning without doing that, will corrupt the SSA
# results. TODO: Could pretend the other branch didn't exist to save
# complexity the merging of processing.
if truth_value is not None:
from .NodeMakingHelpers import wrapStatementWithSideEffects
if truth_value is True:
choice = "true"
new_statement = self.getBranchYes()
else:
choice = "false"
new_statement = self.getBranchNo()
new_statement = wrapStatementWithSideEffects(
new_node = new_statement,
old_node = condition,
allow_none = True # surviving branch may empty
)
return new_statement, "new_statements", """\
Condition for branch was predicted to be always %s.""" % choice
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/KeeperNodes.py 0000644 0001750 0001750 00000010465 12265264105 020532 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Keeper nodes.
We need keeper nodes for comparison chains to hold the previous value during the
evaluation of an expression. They are otherwise not used and should be avoided,
all other constructs use real temporary variables.
"""
from .NodeBases import (
ExpressionChildrenHavingBase,
ExpressionMixin,
NodeBase
)
class ExpressionAssignmentTempKeeper(ExpressionChildrenHavingBase):
kind = "EXPRESSION_ASSIGNMENT_TEMP_KEEPER"
named_children = ( "source", )
def __init__(self, variable, source, source_ref):
assert source is not None
ExpressionChildrenHavingBase.__init__(
self,
values = {
"source" : source,
},
source_ref = source_ref
)
self.variable = variable
self.variable_version = variable.allocateTargetNumber()
assert self.variable_version != 0
def getDetail(self):
return "%s from %s" % (
self.getVariableName(),
self.getAssignSource()
)
def getDetails(self):
return {
"name" : self.getVariableName()
}
def getVariable(self):
return self.variable
def getVariableName(self):
return self.variable.getName()
def getVariableVersion(self):
return self.variable_version
getAssignSource = ExpressionChildrenHavingBase.childGetter(
"source"
)
def computeExpression(self, constraint_collection):
source = self.getAssignSource()
if source.willRaiseException(BaseException):
return source, "new_raise", "Temp keeper assignment source raises."
constraint_collection.onVariableSet(
assign_node = self
)
return self, None, None
def mayRaiseException(self, exception_type):
return self.getAssignSource().mayRaiseException(
exception_type = exception_type
)
def willRaiseException(self, exception_type):
return self.getAssignSource().willRaiseException(
exception_type = exception_type
)
def getTruthValue(self):
return self.getAssignSource().getTruthValue()
class ExpressionTempKeeperRef(NodeBase, ExpressionMixin):
kind = "EXPRESSION_TEMP_KEEPER_REF"
def __init__(self, variable, source_ref):
NodeBase.__init__( self, source_ref = source_ref )
self.variable = variable
def getDetails(self):
return {
"name" : self.getVariableName()
}
def getDetail(self):
return self.getVariableName()
def getVariable(self):
return self.variable
def getVariableName(self):
return self.variable.getName()
def computeExpression(self, constraint_collection):
variable_trace = constraint_collection.getVariableCurrentTrace(
variable = self.variable
)
if variable_trace.isAssignTrace():
assign_source = variable_trace.getAssignNode().getAssignSource()
if not assign_source.mayHaveSideEffects():
return (
assign_source.makeCloneAt( self.getSourceReference() ),
"new_expression", """\
Replaced keeper variable usage for no side effects value."""
)
return self, None, None
def mayRaiseException(self, exception_type):
# Can't happen
return False
def mayHaveSideEffects(self):
return False
def isOrderRelevant(self):
return True
def mayProvideReference(self):
return self.variable.getReferenced().getNeedsFree()
Nuitka-0.5.0.1/nuitka/nodes/BuiltinFormatNodes.py 0000644 0001750 0001750 00000003064 12265264105 022073 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Format nodes bin/oct/hex.
These will most often be used for outputs, and the hope is, the type prediction or the
result prediction will help to be smarter, but generally these should not be that much
about performance critical.
"""
from .NodeBases import ExpressionBuiltinSingleArgBase
from nuitka.optimizations import BuiltinOptimization
class ExpressionBuiltinBin(ExpressionBuiltinSingleArgBase):
kind = "EXPRESSION_BUILTIN_BIN"
builtin_spec = BuiltinOptimization.builtin_bin_spec
class ExpressionBuiltinOct(ExpressionBuiltinSingleArgBase):
kind = "EXPRESSION_BUILTIN_OCT"
builtin_spec = BuiltinOptimization.builtin_oct_spec
class ExpressionBuiltinHex(ExpressionBuiltinSingleArgBase):
kind = "EXPRESSION_BUILTIN_HEX"
builtin_spec = BuiltinOptimization.builtin_hex_spec
Nuitka-0.5.0.1/nuitka/nodes/YieldNodes.py 0000644 0001750 0001750 00000006023 12265264105 020360 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Yield node.
The yield node returns to the caller of the generator and therefore may execute
absolutely abitrary code, from the point of view of this code. It then returns
something, which may often be 'None', but doesn't have to be.
Often it will be used as a statement, which should also be reflected in a
dedicated node.
"""
from .NodeBases import ExpressionChildrenHavingBase
class ExpressionYield(ExpressionChildrenHavingBase):
kind = "EXPRESSION_YIELD"
named_children = ( "expression", )
def __init__(self, expression, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"expression" : expression
},
source_ref = source_ref
)
self.exception_preserving = False
def markAsExceptionPreserving(self):
self.exception_preserving = True
def isExceptionPreserving(self):
return self.exception_preserving
getExpression = ExpressionChildrenHavingBase.childGetter( "expression" )
def computeExpression(self, constraint_collection):
value = self.getExpression()
if value.willRaiseException( BaseException ):
return value, "new_raise", "The 'yield' argument raises exception"
# Nothing possible really here.
return self, None, None
class ExpressionYieldFrom(ExpressionChildrenHavingBase):
kind = "EXPRESSION_YIELD_FROM"
named_children = ( "expression", )
def __init__(self, expression, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"expression" : expression
},
source_ref = source_ref
)
self.exception_preserving = False
def markAsExceptionPreserving(self):
self.exception_preserving = True
def isExceptionPreserving(self):
return self.exception_preserving
getExpression = ExpressionChildrenHavingBase.childGetter( "expression" )
def computeExpression(self, constraint_collection):
value = self.getExpression()
if value.willRaiseException( BaseException ):
return value, "new_raise", """\
The 'yield from' argument raises exception"""
# Nothing possible really here.
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/BuiltinRefNodes.py 0000644 0001750 0001750 00000017174 12265264105 021366 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Tree nodes for builtin references.
There is 2 major types of builtin references. One is the values from
builtins, the other is builtin exceptions. They work differently and
mean different things, but they have similar origin, that is, access
to variables only ever read.
"""
from .NodeBases import NodeBase, CompileTimeConstantExpressionMixin
from .ConstantRefNodes import ExpressionConstantRef
from nuitka.optimizations import BuiltinOptimization
from nuitka.Builtins import (
builtin_exception_names,
builtin_exception_values,
builtin_anon_names,
builtin_names
)
from nuitka.Utils import python_version
class ExpressionBuiltinRefBase(CompileTimeConstantExpressionMixin, NodeBase):
def __init__(self, builtin_name, source_ref):
NodeBase.__init__( self, source_ref = source_ref )
CompileTimeConstantExpressionMixin.__init__( self )
self.builtin_name = builtin_name
def makeCloneAt(self, source_ref):
return self.__class__(self.builtin_name, source_ref)
def getDetails(self):
return { "builtin_name" : self.builtin_name }
def getBuiltinName(self):
return self.builtin_name
def mayHaveSideEffects(self):
# Referencing the builtin name has no side effect
return False
class ExpressionBuiltinRef(ExpressionBuiltinRefBase):
kind = "EXPRESSION_BUILTIN_REF"
def __init__(self, builtin_name, source_ref):
assert builtin_name in builtin_names, builtin_name
ExpressionBuiltinRefBase.__init__(
self,
builtin_name = builtin_name,
source_ref = source_ref
)
def isCompileTimeConstant(self):
# Virtual method, pylint: disable=R0201
return True
def getCompileTimeConstant(self):
return __builtins__[ self.builtin_name ]
def computeExpression(self, constraint_collection):
quick_names = {
"None" : None,
"True" : True,
"False" : False,
"__debug__" : __debug__,
"Ellipsis" : Ellipsis,
}
if self.builtin_name in quick_names:
new_node = ExpressionConstantRef(
constant = quick_names[self.builtin_name],
source_ref = self.getSourceReference()
)
return new_node, "new_constant", """\
Builtin constant %s resolved""" % self.builtin_name
return self, None, None
def computeExpressionCall(self, call_node, constraint_collection):
from nuitka.optimizations.OptimizeBuiltinCalls import computeBuiltinCall
new_node, tags, message = computeBuiltinCall(
call_node = call_node,
called = self
)
if new_node.isExpressionBuiltinLocals() or \
new_node.isExpressionBuiltinEval():
constraint_collection.assumeUnclearLocals()
return new_node, tags, message
def getStringValue(self):
return repr( self.getCompileTimeConstant() )
def isKnownToBeIterable(self, count):
# TODO: Why yes, some may be, could be told here.
return None
def mayProvideReference(self):
# Dedicated code returns which returns from builtin module dictionary, but isn't
# available for Python3 yet.
return python_version >= 300
class ExpressionBuiltinOriginalRef(ExpressionBuiltinRef):
kind = "EXPRESSION_BUILTIN_ORIGINAL_REF"
def isCompileTimeConstant(self):
# TODO: Actually the base class should not be constant and this one should be.
# Virtual method, pylint: disable=R0201
return False
def computeExpression(self, constraint_collection):
# Needs whole program analysis, we don't really know much about it.
return self, None, None
class ExpressionBuiltinAnonymousRef(ExpressionBuiltinRefBase):
kind = "EXPRESSION_BUILTIN_ANONYMOUS_REF"
def __init__(self, builtin_name, source_ref):
assert builtin_name in builtin_anon_names
ExpressionBuiltinRefBase.__init__(
self,
builtin_name = builtin_name,
source_ref = source_ref
)
def isCompileTimeConstant(self):
# Virtual method, pylint: disable=R0201
return True
def mayProvideReference(self):
# No reference provided from this, there are just a global identifiers, or
# accesses to them.
return False
def getCompileTimeConstant(self):
return builtin_anon_names[ self.builtin_name ]
def computeExpression(self, constraint_collection):
return self, None, None
def getStringValue(self):
return repr( self.getCompileTimeConstant() )
class ExpressionBuiltinExceptionRef(ExpressionBuiltinRefBase):
kind = "EXPRESSION_BUILTIN_EXCEPTION_REF"
def __init__(self, exception_name, source_ref):
assert exception_name in builtin_exception_names
ExpressionBuiltinRefBase.__init__(
self,
builtin_name = exception_name,
source_ref = source_ref
)
def getDetails(self):
return { "exception_name" : self.builtin_name }
getExceptionName = ExpressionBuiltinRefBase.getBuiltinName
def isCompileTimeConstant(self):
# Virtual method, pylint: disable=R0201
return True
def mayProvideReference(self):
# No reference provided from this, it's just a global identifier.
return False
def getCompileTimeConstant(self):
return builtin_exception_values[ self.builtin_name ]
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
return self, None, None
def computeExpressionCall(self, call_node, constraint_collection):
exception_name = self.getExceptionName()
# TODO: Keyword only arguments of it, are not properly handled yet by
# the built-in call code.
if exception_name == "ImportError" and python_version >= 330:
kw = call_node.getCallKw()
if not kw.isExpressionConstantRef() or kw.getConstant() != {}:
return call_node, None, None
def createBuiltinMakeException(args, source_ref):
from nuitka.nodes.ExceptionNodes import ExpressionBuiltinMakeException
return ExpressionBuiltinMakeException(
exception_name = exception_name,
args = args,
source_ref = source_ref
)
new_node = BuiltinOptimization.extractBuiltinArgs(
node = call_node,
builtin_class = createBuiltinMakeException,
builtin_spec = BuiltinOptimization.makeBuiltinParameterSpec(
exception_name = exception_name
)
)
# TODO: Don't allow this to happen.
if new_node is None:
return call_node, None, None
return new_node, "new_expression", "detected builtin exception making"
Nuitka-0.5.0.1/nuitka/nodes/ExceptionNodes.py 0000644 0001750 0001750 00000024222 12265264105 021251 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Nodes related to raising and making exceptions.
"""
from .NodeBases import (
ExpressionChildrenHavingBase,
StatementChildrenHavingBase,
ExpressionMixin,
NodeBase
)
class StatementRaiseException(StatementChildrenHavingBase):
kind = "STATEMENT_RAISE_EXCEPTION"
named_children = (
"exception_type",
"exception_value",
"exception_trace",
"exception_cause"
)
def __init__(self, exception_type, exception_value, exception_trace,
exception_cause, source_ref):
if exception_type is None:
assert exception_value is None
if exception_value is None:
assert exception_trace is None
StatementChildrenHavingBase.__init__(
self,
values = {
"exception_type" : exception_type,
"exception_value" : exception_value,
"exception_trace" : exception_trace,
"exception_cause" : exception_cause
},
source_ref = source_ref
)
self.reraise_local = False
self.reraise_finally = False
getExceptionType = StatementChildrenHavingBase.childGetter(
"exception_type"
)
getExceptionValue = StatementChildrenHavingBase.childGetter(
"exception_value"
)
getExceptionTrace = StatementChildrenHavingBase.childGetter(
"exception_trace"
)
getExceptionCause = StatementChildrenHavingBase.childGetter(
"exception_cause"
)
def isReraiseException(self):
return self.getExceptionType() is None
def isReraiseExceptionLocal(self):
assert self.isReraiseException()
return self.reraise_local
def isReraiseExceptionFinally(self):
assert self.isReraiseException()
return self.reraise_finally
def markAsReraiseLocal(self):
self.reraise_local = True
def markAsReraiseFinally(self):
self.reraise_finally = True
def isStatementAborting(self):
return True
def needsLineNumber(self):
return not self.isReraiseException()
def isImplicit(self):
return False
def computeStatement(self, constraint_collection):
constraint_collection.onExpression(
expression = self.getExceptionType(),
allow_none = True
)
exception_type = self.getExceptionType()
if exception_type is not None and \
exception_type.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = exception_type,
node = self
)
return result, "new_raise", """\
Explicit raise already raises implicitely building exception type."""
constraint_collection.onExpression(
expression = self.getExceptionValue(),
allow_none = True
)
exception_value = self.getExceptionValue()
if exception_value is not None and exception_value.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
exception_type,
exception_value
)
)
return result, "new_node", """\
Explicit raise already raises implicitely building exception value."""
constraint_collection.onExpression(
expression = self.getExceptionTrace(),
allow_none = True
)
exception_trace = self.getExceptionTrace()
if exception_trace is not None and \
exception_trace.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
exception_type,
exception_value,
exception_trace
)
)
return result, "new_raise", """\
Explicit raise already raises implicitely building exception traceback."""
constraint_collection.onExpression(
expression = self.getExceptionCause(),
allow_none = True
)
exception_cause = self.getExceptionCause()
if exception_cause is not None and \
exception_cause.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
exception_type,
exception_cause,
)
)
return result, "new_raise", """
Explicit raise already raises implicitely building exception cause."""
return self, None, None
class StatementRaiseExceptionImplicit(StatementRaiseException):
kind = "STATEMENT_RAISE_EXCEPTION_IMPLICIT"
def isStatementRaiseException(self):
return True
def isImplicit(self):
return True
class ExpressionRaiseException(ExpressionChildrenHavingBase):
""" This node type is only produced via optimization.
CPython only knows exception raising as a statement, but often the raising
of exceptions can be predicted to occur as part of an expression, which it
replaces then.
"""
kind = "EXPRESSION_RAISE_EXCEPTION"
named_children = ( "exception_type", "exception_value" )
def __init__(self, exception_type, exception_value, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"exception_type" : exception_type,
"exception_value" : exception_value
},
source_ref = source_ref
)
def willRaiseException(self, exception_type):
# Virtual method, pylint: disable=R0201,W0613
# One thing is clear, it will raise. TODO: Match exception_type more
# closely if it is predictable.
if exception_type is BaseException:
return True
else:
return False
getExceptionType = ExpressionChildrenHavingBase.childGetter(
"exception_type"
)
getExceptionValue = ExpressionChildrenHavingBase.childGetter(
"exception_value"
)
def mayProvideReference(self):
return False
def computeExpression(self, constraint_collection):
return self, None, None
def computeExpressionDrop(self, statement, constraint_collection):
from .ExceptionNodes import StatementRaiseExceptionImplicit
result = StatementRaiseExceptionImplicit(
exception_type = self.getExceptionType(),
exception_value = self.getExceptionValue(),
exception_trace = None,
exception_cause = None,
source_ref = self.getSourceReference()
)
return result, "new_raise", """\
Propgated implict raise expression to raise statement."""
class ExpressionBuiltinMakeException(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_MAKE_EXCEPTION"
named_children = ( "args", )
def __init__(self, exception_name, args, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"args" : tuple( args ),
},
source_ref = source_ref
)
self.exception_name = exception_name
def getDetails(self):
return { "exception_name" : self.exception_name }
def getExceptionName(self):
return self.exception_name
getArgs = ExpressionChildrenHavingBase.childGetter( "args" )
def computeExpression(self, constraint_collection):
return self, None, None
class ExpressionCaughtExceptionTypeRef(NodeBase, ExpressionMixin):
kind = "EXPRESSION_CAUGHT_EXCEPTION_TYPE_REF"
def __init__(self, source_ref):
NodeBase.__init__(
self,
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
# TODO: Might be predictable based on the exception handler this is in.
return self, None, None
def mayHaveSideEffects(self):
# Referencing the expression type has no side effect
return False
class ExpressionCaughtExceptionValueRef(NodeBase, ExpressionMixin):
kind = "EXPRESSION_CAUGHT_EXCEPTION_VALUE_REF"
def __init__(self, source_ref):
NodeBase.__init__(
self,
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
# TODO: Might be predictable based on the exception handler this is in.
return self, None, None
def mayHaveSideEffects(self):
# Referencing the expression type has no side effect
return False
def makeCloneAt(self, source_ref):
return ExpressionCaughtExceptionValueRef(
source_ref = source_ref
)
class ExpressionCaughtExceptionTracebackRef(NodeBase, ExpressionMixin):
kind = "EXPRESSION_CAUGHT_EXCEPTION_TRACEBACK_REF"
def __init__(self, source_ref):
NodeBase.__init__(
self,
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
return self, None, None
def mayHaveSideEffects(self):
# Referencing the expression type has no side effect
return False
Nuitka-0.5.0.1/nuitka/nodes/FutureSpecs.py 0000644 0001750 0001750 00000005463 12265264105 020600 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Specification record for future flags.
A source reference also implies a specific set of future flags in use by the
parser at that location. Can be different inside a module due to e.g. the
inlining of exec statements with their own future imports, or inlining of code
from other modules.
"""
from nuitka import Utils
_future_division_default = Utils.python_version >= 300
_future_absolute_import_default = Utils.python_version >= 300
class FutureSpec:
def __init__(self):
self.future_division = _future_division_default
self.unicode_literals = False
self.absolute_import = _future_absolute_import_default
self.future_print = False
self.barry_bdfl = False
def clone(self):
result = FutureSpec()
result.future_division = self.future_division
result.unicode_literals = self.unicode_literals
result.absolute_import = self.absolute_import
result.future_print = self.future_print
result.barry_bdfl = self.barry_bdfl
return result
def isFutureDivision(self):
return self.future_division
def enableFutureDivision(self):
self.future_division = True
def enableFuturePrint(self):
self.future_print = True
def enableUnicodeLiterals(self):
self.unicode_literals = True
def enableAbsoluteImport(self):
self.absolute_import = True
def enableBarry(self):
self.barry_bdfl = True
def isAbsoluteImport(self):
return self.absolute_import
def asFlags(self):
result = []
if self.future_division:
result.append("CO_FUTURE_DIVISION")
if self.unicode_literals:
result.append("CO_FUTURE_UNICODE_LITERALS")
if self.absolute_import:
result.append("CO_FUTURE_ABSOLUTE_IMPORT")
if self.future_print:
result.append("CO_FUTURE_PRINT_FUNCTION")
if self.barry_bdfl and Utils.python_version >= 300:
result.append("CO_FUTURE_BARRY_AS_BDFL")
return result
Nuitka-0.5.0.1/nuitka/nodes/TryNodes.py 0000644 0001750 0001750 00000025420 12265264105 020072 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Nodes for try/except and try/finally
The try/except needs handlers, and these blocks are complex control flow.
"""
from .NodeBases import StatementChildrenHavingBase
class StatementTryFinally(StatementChildrenHavingBase):
kind = "STATEMENT_TRY_FINALLY"
named_children = ( "tried", "final" )
def __init__(self, tried, final, source_ref):
assert tried is None or tried.isStatementsSequence()
assert final is None or final.isStatementsSequence()
StatementChildrenHavingBase.__init__(
self,
values = {
"tried" : tried,
"final" : final
},
source_ref = source_ref
)
self.break_exception = False
self.continue_exception = False
self.return_value_exception_catch = False
self.return_value_exception_reraise = False
getBlockTry = StatementChildrenHavingBase.childGetter( "tried" )
setBlockTry = StatementChildrenHavingBase.childSetter( "tried" )
getBlockFinal = StatementChildrenHavingBase.childGetter( "final" )
setBlockFinal = StatementChildrenHavingBase.childSetter( "final" )
def isStatementAborting(self):
# In try/finally there are two chances to raise or return a value, so we
# need to "or" the both branches. One of them will do.
tried_block = self.getBlockTry()
if tried_block is not None and tried_block.isStatementAborting():
return True
final_block = self.getBlockFinal()
if final_block is not None and final_block.isStatementAborting():
return True
return False
def markAsExceptionContinue(self):
self.continue_exception = True
def markAsExceptionBreak(self):
self.break_exception = True
def markAsExceptionReturnValueCatch(self):
self.return_value_exception_catch = True
def markAsExceptionReturnValueReraise(self):
self.return_value_exception_reraise = True
def needsExceptionContinue(self):
return self.continue_exception
def needsExceptionBreak(self):
return self.break_exception
def needsExceptionReturnValueCatcher(self):
return self.return_value_exception_catch
def needsExceptionReturnValueReraiser(self):
return self.return_value_exception_reraise
def computeStatement(self, constraint_collection):
# The tried block must be considered as a branch, if it is not empty
# already.
tried_statement_sequence = self.getBlockTry()
# May be "None" from the outset, so guard against that, later in this
# function we are going to remove it.
if tried_statement_sequence is not None:
result = tried_statement_sequence.computeStatementsSequence(
constraint_collection = constraint_collection
)
# Might be changed.
if result is not tried_statement_sequence:
tried_statement_sequence.replaceWith( result )
tried_statement_sequence = result
final_statement_sequence = self.getBlockFinal()
# TODO: The final must not assume that all of tried was executed,
# instead it may have aborted after any part of it, which is a rather
# complex definition.
if final_statement_sequence is not None:
if tried_statement_sequence is not None:
from nuitka.tree.Extractions import getVariablesWritten
variable_writes = getVariablesWritten(
tried_statement_sequence
)
# Mark all variables as unknown that are written in the tried
# block, so it destroys the assumptions for loop turn around.
for variable, _variable_version in variable_writes:
constraint_collection.markActiveVariableAsUnknown(
variable = variable
)
# Then assuming no exception, the no raise block if present.
result = final_statement_sequence.computeStatementsSequence(
constraint_collection = constraint_collection
)
if result is not final_statement_sequence:
self.setBlockFinal(result)
final_statement_sequence = result
# Note: Need to query again, because the object may have changed in the
# "computeStatementsSequence" calls.
if tried_statement_sequence is None:
# If the tried block is empty, go to the final block directly, if
# any.
return final_statement_sequence, "new_statements", """\
Removed try/finally with empty tried block."""
elif final_statement_sequence is None:
# If the final block is empty, just need to execute the tried block
# then.
return tried_statement_sequence, "new_statements", """\
Removed try/finally with empty final block."""
else:
# TODO: Can't really merge it yet.
constraint_collection.removeAllKnowledge()
# Otherwise keep it as it.
return self, None, None
class StatementExceptHandler(StatementChildrenHavingBase):
kind = "STATEMENT_EXCEPT_HANDLER"
named_children = (
"exception_types",
"body"
)
def __init__(self, exception_types, body, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"exception_types" : tuple( exception_types ),
"body" : body,
},
source_ref = source_ref
)
getExceptionTypes = StatementChildrenHavingBase.childGetter(
"exception_types"
)
getExceptionBranch = StatementChildrenHavingBase.childGetter(
"body"
)
setExceptionBranch = StatementChildrenHavingBase.childSetter(
"body"
)
class StatementTryExcept(StatementChildrenHavingBase):
kind = "STATEMENT_TRY_EXCEPT"
named_children = ( "tried", "handlers" )
def __init__(self, tried, handlers, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"tried" : tried,
"handlers" : tuple( handlers )
},
source_ref = source_ref
)
getBlockTry = StatementChildrenHavingBase.childGetter(
"tried"
)
setBlockTry = StatementChildrenHavingBase.childSetter(
"tried"
)
getExceptionHandlers = StatementChildrenHavingBase.childGetter(
"handlers"
)
def isStatementAborting(self):
tried_block = self.getBlockTry()
# Happens during tree building only.
if tried_block is None:
return False
if not tried_block.isStatementAborting():
return False
for handler in self.getExceptionHandlers():
if not handler.isStatementAborting():
return False
return True
def isStatementTryExceptOptimized(self):
tried_block = self.getBlockTry()
tried_statements = tried_block.getStatements()
if len( tried_statements ) == 1:
tried_statement = tried_statements[0]
if tried_statement.isStatementAssignmentVariable():
source = tried_statement.getAssignSource()
if source.isExpressionBuiltinNext1():
if not source.getValue().mayRaiseException( BaseException ):
# Note: Now we know the source lookup is the only thing
# that may raise.
handlers = self.getExceptionHandlers()
if len( handlers ) == 1:
catched_types = handlers[0].getExceptionTypes()
if len( catched_types ) == 1:
catched_type = catched_types[0]
if catched_type.isExpressionBuiltinExceptionRef():
if catched_type.getExceptionName() == "StopIteration":
if handlers[0].getExceptionBranch().isStatementAborting():
return True
return False
def computeStatement(self, constraint_collection):
# The tried block can be processed normally.
tried_statement_sequence = self.getBlockTry()
# May be "None" from the outset, so guard against that, later we are
# going to remove it.
if tried_statement_sequence is not None:
result = tried_statement_sequence.computeStatementsSequence(
constraint_collection = constraint_collection
)
if result is not tried_statement_sequence:
self.setBlockTry(result)
tried_statement_sequence = result
if tried_statement_sequence is None:
return None, "new_statements", """\
Removed try/except with empty tried block."""
from nuitka.optimizations.ConstraintCollections import \
ConstraintCollectionHandler
# The exception branches triggers in unknown state, any amount of tried
# code may have happened. A similar approach to loops should be taken to
# invalidate the state before.
for handler in self.getExceptionHandlers():
collection_exception_branch = ConstraintCollectionHandler(
parent = constraint_collection,
handler = handler
)
# Without exception handlers remaining, nothing else to do. They may
# e.g. be removed as only re-raising.
if not self.getExceptionHandlers():
return tried_statement_sequence, "new_statements", """\
Removed try/except without any remaing handlers."""
# Give up, merging this is too hard for now, any amount of the tried
# sequence may have executed together with one of the handlers, or all
# of tried and no handlers. TODO: improve this to an actual merge, even
# if a pessimistic one.
constraint_collection.removeAllKnowledge()
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/BuiltinIteratorNodes.py 0000644 0001750 0001750 00000020731 12265264105 022434 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Builtin iterator nodes.
These play a role in for loops, and in unpacking. They can something be
predicted to succeed or fail, in which case, code can become less complex.
The length of things is an important optimization issue for these to be
good.
"""
from .NodeBases import (
ExpressionBuiltinSingleArgBase,
ExpressionChildrenHavingBase,
StatementChildrenHavingBase
)
from nuitka.optimizations import BuiltinOptimization
class ExpressionBuiltinLen(ExpressionBuiltinSingleArgBase):
kind = "EXPRESSION_BUILTIN_LEN"
builtin_spec = BuiltinOptimization.builtin_len_spec
def getIntegerValue(self):
return self.getValue().getIterationLength()
def computeExpression(self, constraint_collection):
from .NodeMakingHelpers import makeConstantReplacementNode, wrapExpressionWithNodeSideEffects
new_node, change_tags, change_desc = ExpressionBuiltinSingleArgBase.\
computeExpression(
self,
constraint_collection = constraint_collection
)
if new_node is self:
arg_length = self.getIntegerValue()
if arg_length is not None:
change_tags = "new_constant"
change_desc = "Predicted len argument"
new_node = wrapExpressionWithNodeSideEffects(
new_node = makeConstantReplacementNode( arg_length, self ),
old_node = self.getValue()
)
if new_node.isExpressionSideEffects(): # false alarm pylint: disable=E1103
change_desc += " maintaining side effects"
return new_node, change_tags, change_desc
class ExpressionBuiltinIter1(ExpressionBuiltinSingleArgBase):
kind = "EXPRESSION_BUILTIN_ITER1"
def computeExpression(self, constraint_collection):
value = self.getValue()
# Iterator of an iterator can be removed.
if value.isIteratorMaking():
return value, "new_builtin", "Eliminated useless iterator creation."
return value.computeExpressionIter1(
iter_node = self,
constraint_collection = constraint_collection
)
def isIteratorMaking(self):
return True
def isKnownToBeIterable(self, count):
if count is None:
return True
# TODO: Should ask value if it is.
return None
def getIterationLength(self):
return self.getValue().getIterationLength()
def extractSideEffects(self):
# Iterator making is the side effect itself.
if self.getValue().isCompileTimeConstant():
return ()
else:
return ( self, )
def mayHaveSideEffects(self):
if self.getValue().isCompileTimeConstant():
return self.getValue().isKnownToBeIterable( None )
return True
def mayProvideReference(self):
# Method overload, where it's fixed by type, pylint: disable=R0201
return True
def isKnownToBeIterableAtMin(self, count):
assert type( count ) is int
iter_length = self.getValue().getIterationLength()
return iter_length is not None and iter_length < count
def isKnownToBeIterableAtMax(self, count):
assert type( count ) is int
iter_length = self.getValue().getIterationLength()
return iter_length is not None and count <= iter_length
def onRelease(self, constraint_collection):
# print "onRelease", self
pass
class ExpressionBuiltinNext1(ExpressionBuiltinSingleArgBase):
kind = "EXPRESSION_BUILTIN_NEXT1"
def getDetails(self):
return {
"iter" : self.getValue()
}
def makeCloneAt(self, source_ref):
return self.__class__(
value = self.getValue(),
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
# TODO: Predict iteration result if possible via SSA variable trace of
# the iterator state.
return self, None, None
class ExpressionSpecialUnpack(ExpressionBuiltinNext1):
kind = "EXPRESSION_SPECIAL_UNPACK"
def __init__(self, value, count, source_ref):
ExpressionBuiltinNext1.__init__(
self,
value = value,
source_ref = source_ref
)
self.count = count
def makeCloneAt(self, source_ref):
return self.__class__(
value = self.getValue(),
count = self.getCount(),
source_ref = source_ref
)
def getDetails(self):
result = ExpressionBuiltinNext1.getDetails( self )
result[ "element_index" ] = self.getCount()
return result
def getCount(self):
return self.count
class StatementSpecialUnpackCheck(StatementChildrenHavingBase):
kind = "STATEMENT_SPECIAL_UNPACK_CHECK"
named_children = ( "iterator", )
def __init__(self, iterator, count, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"iterator" : iterator
},
source_ref = source_ref
)
self.count = count
def getDetails(self):
return {
"count" : self.getCount(),
}
def getCount(self):
return self.count
getIterator = StatementChildrenHavingBase.childGetter( "iterator" )
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getIterator() )
iterator = self.getIterator()
if iterator.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = iterator,
node = self
)
return result, "new_raise", "Explicit raise already raises implicitely building exception type"
# Remove the check if it can be decided at compile time.
if iterator.isKnownToBeIterableAtMax( 0 ):
return None, "new_statements", "Determined iteration end check to be always true."
return self, None, None
class ExpressionBuiltinIter2(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_ITER2"
named_children = ( "callable", "sentinel", )
# Need to accept 'callable' keyword argument, that is just the API of iter,
# pylint: disable=W0622
def __init__(self, callable, sentinel, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"callable" : callable,
"sentinel" : sentinel,
},
source_ref = source_ref
)
getCallable = ExpressionChildrenHavingBase.childGetter( "callable" )
getSentinel = ExpressionChildrenHavingBase.childGetter( "sentinel" )
def computeExpression(self, constraint_collection):
# TODO: The "callable" should be investigated here, pylint: disable=W0613
return self, None, None
def isIteratorMaking(self):
return True
class ExpressionBuiltinNext2(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_NEXT2"
named_children = ( "iterator", "default", )
def __init__(self, iterator, default, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"iterator" : iterator,
"default" : default,
},
source_ref = source_ref
)
getIterator = ExpressionChildrenHavingBase.childGetter( "iterator" )
getDefault = ExpressionChildrenHavingBase.childGetter( "default" )
def computeExpression(self, constraint_collection):
# TODO: The "iterator" should be investigated here, pylint: disable=W0613
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/SubscriptNodes.py 0000644 0001750 0001750 00000004056 12265264105 021274 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Subscript node.
Subscripts are important when working with lists and dictionaries. Tracking them can allow
to achieve more compact code, or predict results at compile time.
There will be a method "computeExpressionSubscript" to aid predicting them.
"""
from .NodeBases import ExpressionChildrenHavingBase
class ExpressionSubscriptLookup(ExpressionChildrenHavingBase):
kind = "EXPRESSION_SUBSCRIPT_LOOKUP"
named_children = ( "expression", "subscript" )
def __init__(self, expression, subscript, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"expression" : expression,
"subscript" : subscript
},
source_ref = source_ref
)
getLookupSource = ExpressionChildrenHavingBase.childGetter( "expression" )
getSubscript = ExpressionChildrenHavingBase.childGetter( "subscript" )
def computeExpression(self, constraint_collection):
lookup_source = self.getLookupSource()
return lookup_source.computeExpressionSubscript(
lookup_node = self,
subscript = self.getSubscript(),
constraint_collection = constraint_collection
)
def isKnownToBeIterable(self, count):
return None
Nuitka-0.5.0.1/nuitka/nodes/ReturnNodes.py 0000644 0001750 0001750 00000005232 12265264105 020572 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Return node
This one exits functions. The only other exit is the default exit of functions with 'None'
value, if no return is done.
"""
from .NodeBases import StatementChildrenHavingBase
class StatementReturn(StatementChildrenHavingBase):
kind = "STATEMENT_RETURN"
named_children = ( "expression", )
def __init__(self, expression, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"expression" : expression
},
source_ref = source_ref
)
self.exception_driven = None
getExpression = StatementChildrenHavingBase.childGetter( "expression" )
def isStatementAborting(self):
return True
def mayRaiseException(self, exception_type):
return self.getExpression().mayRaiseException( exception_type )
def setExceptionDriven(self, value):
self.exception_driven = value
def isExceptionDriven(self):
assert self.exception_driven is not None
return self.exception_driven
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getExpression() )
expression = self.getExpression()
if expression.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = expression,
node = self
)
return result, "new_raise", "Return statement raises in returned expression, removed return"
return self, None, None
class StatementGeneratorReturn(StatementReturn):
kind = "STATEMENT_GENERATOR_RETURN"
def __init__(self, expression, source_ref):
StatementReturn.__init__(
self,
expression = expression,
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/nodes/NodeMakingHelpers.py 0000644 0001750 0001750 00000020510 12265264105 021655 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" These are just helpers to create nodes, often to replace existing nodes
These are for use in optimizations and computations, and therefore cover
mostly exceptions and constants.
Often cyclic dependencies kicks in, which is why this module is mostly only
imported locally.
"""
from .ConstantRefNodes import ExpressionConstantRef
from nuitka.Constants import isConstant
from nuitka.Builtins import builtin_names
from nuitka.Options import shallWarnImplicitRaises, isDebug
from .BuiltinRefNodes import (
ExpressionBuiltinExceptionRef,
ExpressionBuiltinRef
)
from .ExceptionNodes import ExpressionRaiseException
from .StatementNodes import (
StatementExpressionOnly,
StatementsSequence
)
from .ComparisonNodes import (
ExpressionComparison,
ExpressionComparisonIs,
ExpressionComparisonIsNOT
)
from .SideEffectNodes import ExpressionSideEffects
from logging import warning
def makeConstantReplacementNode(constant, node):
return ExpressionConstantRef(
constant = constant,
source_ref = node.getSourceReference()
)
def makeRaiseExceptionReplacementExpression(expression, exception_type,
exception_value):
source_ref = expression.getSourceReference()
assert type( exception_type ) is str
if shallWarnImplicitRaises():
warning(
"%s: Static exception raise",
expression.getSourceReference().getAsString(),
)
result = ExpressionRaiseException(
exception_type = ExpressionBuiltinExceptionRef(
exception_name = exception_type,
source_ref = source_ref
),
exception_value = makeConstantReplacementNode(
constant = exception_value,
node = expression
),
source_ref = source_ref
)
return result
def makeRaiseExceptionReplacementExpressionFromInstance(expression, exception):
assert isinstance( exception, Exception )
args = exception.args
assert type( args ) is tuple and len( args ) == 1, args
return makeRaiseExceptionReplacementExpression(
expression = expression,
exception_type = exception.__class__.__name__,
exception_value = args[0]
)
def isCompileTimeConstantValue(value):
# This needs to match code in makeCompileTimeConstantReplacementNode
if isConstant( value ):
return True
elif type( value ) is type:
return True
else:
return False
def makeCompileTimeConstantReplacementNode(value, node):
# This needs to match code in isCompileTimeConstantValue
if isConstant( value ):
return makeConstantReplacementNode(
constant = value,
node = node
)
elif type( value ) is type:
if value.__name__ in builtin_names:
return ExpressionBuiltinRef(
builtin_name = value.__name__,
source_ref = node.getSourceReference()
)
else:
return node
else:
return node
def getComputationResult(node, computation, description):
""" With a computation function, execute it and return constant result or
exception node.
"""
# Try and turn raised exceptions into static raises. pylint: disable=W0703
try:
result = computation()
except Exception as e:
new_node = makeRaiseExceptionReplacementExpressionFromInstance(
expression = node,
exception = e
)
change_tags = "new_raise"
change_desc = description + " Was predicted to raise an exception."
else:
new_node = makeCompileTimeConstantReplacementNode(
value = result,
node = node
)
if isDebug():
assert new_node is not node, ( node, result )
if new_node is not node:
change_tags = "new_constant"
change_desc = description + " Was predicted to constant result."
else:
change_tags = None
change_desc = None
return new_node, change_tags, change_desc
def makeStatementExpressionOnlyReplacementNode(expression, node):
return StatementExpressionOnly(
expression = expression,
source_ref = node.getSourceReference()
)
def mergeStatements(statements):
""" Helper function that merges nested statement sequences. """
merged_statements = []
for statement in statements:
if statement.isStatement() or statement.isStatementsFrame():
merged_statements.append(statement)
elif statement.isStatementsSequence():
merged_statements.extend(mergeStatements(statement.getStatements()))
else:
assert False, statement
return merged_statements
def makeStatementsSequenceReplacementNode(statements, node):
return StatementsSequence(
statements = mergeStatements(statements),
source_ref = node.getSourceReference()
)
def convertNoneConstantToNone(node):
if node is None:
return None
elif node.isExpressionConstantRef() and node.getConstant() is None:
return None
else:
return node
def wrapExpressionWithSideEffects(side_effects, old_node, new_node):
assert new_node.isExpression()
if side_effects:
new_node = ExpressionSideEffects(
expression = new_node,
side_effects = side_effects,
source_ref = old_node.getSourceReference()
)
return new_node
def wrapExpressionWithNodeSideEffects(new_node, old_node):
return wrapExpressionWithSideEffects(
side_effects = old_node.extractSideEffects(),
old_node = old_node,
new_node = new_node
)
def wrapStatementWithSideEffects(new_node, old_node, allow_none = False):
assert new_node is not None or allow_none
side_effects = old_node.extractSideEffects()
if side_effects:
side_effects = tuple(
StatementExpressionOnly(
expression = side_effect,
source_ref = side_effect.getSourceReference()
)
for side_effect in side_effects
)
if new_node is not None:
new_node = makeStatementsSequenceReplacementNode(
statements = side_effects + ( new_node, ),
node = old_node
)
else:
new_node = makeStatementsSequenceReplacementNode(
statements = side_effects,
node = old_node
)
return new_node
def makeStatementOnlyNodesFromExpressions(expressions):
statements = tuple(
StatementExpressionOnly(
expression = expression,
source_ref = expression.getSourceReference()
)
for expression in expressions
)
if not statements:
return None
elif len( statements ) == 1:
return statements[ 0 ]
else:
return StatementsSequence(
statements = statements,
source_ref = statements[0].getSourceReference()
)
def makeComparisonNode(left, right, comparator, source_ref):
if comparator == "Is":
return ExpressionComparisonIs(
left = left,
right = right,
source_ref = source_ref
)
elif comparator == "IsNot":
return ExpressionComparisonIsNOT(
left = left,
right = right,
source_ref = source_ref
)
else:
return ExpressionComparison(
left = left,
right = right,
comparator = comparator,
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/nodes/BuiltinDecodingNodes.py 0000644 0001750 0001750 00000003611 12265264105 022355 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Builtin ord/chr nodes
These are good for optimizations, as they give a very well known result. In the case of
'chr', it's one of 256 strings, and in case of 'ord' it's one of 256 numbers, so these can
answer quite a few questions at compile time.
"""
from .NodeBases import (
ExpressionBuiltinSingleArgBase,
ExpressionBuiltinNoArgBase
)
from nuitka.optimizations import BuiltinOptimization
class ExpressionBuiltinOrd0(ExpressionBuiltinNoArgBase):
kind = "EXPRESSION_BUILTIN_ORD0"
def __init__(self, source_ref):
ExpressionBuiltinNoArgBase.__init__(
self,
builtin_function = ord,
source_ref = source_ref
)
class ExpressionBuiltinOrd(ExpressionBuiltinSingleArgBase):
kind = "EXPRESSION_BUILTIN_ORD"
builtin_spec = BuiltinOptimization.builtin_ord_spec
def isKnownToBeIterable(self, count):
return False
class ExpressionBuiltinChr(ExpressionBuiltinSingleArgBase):
kind = "EXPRESSION_BUILTIN_CHR"
builtin_spec = BuiltinOptimization.builtin_chr_spec
def isKnownToBeIterable(self, count):
return count is None or count == 1
Nuitka-0.5.0.1/nuitka/nodes/AssignNodes.py 0000644 0001750 0001750 00000043670 12265264105 020547 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Assignment related nodes.
All kinds of assignment targets as well as the assignment statement and
expression are located here. These are the core of value control flow.
Note: Currently there is also assignment to keeper nodes in KeeperNodes,
that should be unified at some point.
"""
from .NodeBases import StatementChildrenHavingBase
# Delayed import into multiple branches is not an issue, pylint: disable=W0404
class StatementAssignmentVariable(StatementChildrenHavingBase):
kind = "STATEMENT_ASSIGNMENT_VARIABLE"
named_children = ( "source", "variable_ref" )
def __init__(self, variable_ref, source, source_ref):
assert variable_ref is not None, source_ref
assert source is not None, source_ref
assert variable_ref.isTargetVariableRef()
StatementChildrenHavingBase.__init__(
self,
values = {
"source" : source,
"variable_ref" : variable_ref
},
source_ref = source_ref
)
getTargetVariableRef = StatementChildrenHavingBase.childGetter(
"variable_ref"
)
getAssignSource = StatementChildrenHavingBase.childGetter( "source" )
def mayRaiseException(self, exception_type):
return self.getAssignSource().mayRaiseException( exception_type )
def computeStatement(self, constraint_collection):
# Assignment source may re-compute here:
constraint_collection.onExpression( self.getAssignSource() )
constraint_collection.onVariableSet(
assign_node = self,
)
# TODO: Remove this, it's old.
return constraint_collection._onStatementAssignmentVariable( self )
class StatementAssignmentAttribute(StatementChildrenHavingBase):
kind = "STATEMENT_ASSIGNMENT_ATTRIBUTE"
named_children = ( "source", "expression" )
def __init__(self, expression, attribute_name, source, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"expression" : expression,
"source" : source,
},
source_ref = source_ref
)
self.attribute_name = attribute_name
def getDetails(self):
return { "attribute" : self.attribute_name }
def getDetail(self):
return "to attribute %s" % self.attribute_name
def getAttributeName(self):
return self.attribute_name
def setAttributeName(self, attribute_name):
self.attribute_name = attribute_name
getLookupSource = StatementChildrenHavingBase.childGetter( "expression" )
getAssignSource = StatementChildrenHavingBase.childGetter( "source" )
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getAssignSource() )
source = self.getAssignSource()
# No assignment will occur, if the assignment source raises, so strip it
# away.
if source.willRaiseException( BaseException ):
from .NodeMakingHelpers import \
makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = source,
node = self
)
return result, "new_raise", """\
Attribute assignment raises exception in assigned value, removed assignment"""
constraint_collection.onExpression( self.getLookupSource() )
lookup_source = self.getLookupSource()
if lookup_source.willRaiseException( BaseException ):
from .NodeMakingHelpers import \
makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
source,
lookup_source
)
)
return result, "new_raise", """\
Attribute assignment raises exception in attribute source, removed assignment"""
return self, None, None
class StatementAssignmentSubscript(StatementChildrenHavingBase):
kind = "STATEMENT_ASSIGNMENT_SUBSCRIPT"
named_children = ( "source", "expression", "subscript" )
def __init__(self, expression, subscript, source, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"source" : source,
"expression" : expression,
"subscript" : subscript
},
source_ref = source_ref
)
getSubscribed = StatementChildrenHavingBase.childGetter( "expression" )
getSubscript = StatementChildrenHavingBase.childGetter( "subscript" )
getAssignSource = StatementChildrenHavingBase.childGetter( "source" )
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getAssignSource() )
source = self.getAssignSource()
# No assignment will occur, if the assignment source raises, so strip it
# away.
if source.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = source,
node = self
)
return result, "new_raise", """\
Subscript assignment raises exception in assigned value, removed assignment"""
constraint_collection.onExpression( self.getSubscribed() )
subscribed = self.getSubscribed()
if subscribed.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
source,
subscribed
)
)
return result, "new_raise", """\
Subscript assignment raises exception in subscribed value, removed assignment"""
constraint_collection.onExpression( self.getSubscript() )
subscript = self.getSubscript()
if subscript.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
source,
subscribed,
subscript
)
)
return result, "new_raise", """
Subscript assignment raises exception in subscript value, removed \
assignment."""
return self, None, None
class StatementAssignmentSlice(StatementChildrenHavingBase):
kind = "STATEMENT_ASSIGNMENT_SLICE"
named_children = ( "source", "expression", "lower", "upper" )
def __init__(self, expression, lower, upper, source, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"source" : source,
"expression" : expression,
"lower" : lower,
"upper" : upper
},
source_ref = source_ref
)
getLookupSource = StatementChildrenHavingBase.childGetter( "expression" )
getLower = StatementChildrenHavingBase.childGetter( "lower" )
getUpper = StatementChildrenHavingBase.childGetter( "upper" )
getAssignSource = StatementChildrenHavingBase.childGetter( "source" )
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getAssignSource() )
source = self.getAssignSource()
# No assignment will occur, if the assignment source raises, so strip it
# away.
if source.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = source,
node = self
)
return result, "new_raise", """\
Slice assignment raises exception in assigned value, removed assignment"""
constraint_collection.onExpression( self.getLookupSource() )
lookup_source = self.getLookupSource()
if lookup_source.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
source,
lookup_source
)
)
return result, "new_raise", """\
Slice assignment raises exception in sliced value, removed assignment"""
constraint_collection.onExpression( self.getLower(), allow_none = True )
lower = self.getLower()
if lower is not None and lower.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
source,
lookup_source,
lower
)
)
return result, "new_raise", """\
Slice assignment raises exception in lower slice boundary value, removed \
assignment."""
constraint_collection.onExpression( self.getUpper(), allow_none = True )
upper = self.getUpper()
if upper is not None and upper.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
source,
lookup_source,
lower,
upper
)
)
return result, "new_raise", """\
Slice assignment raises exception in upper slice boundary value, removed \
assignment."""
return self, None, None
class StatementDelVariable(StatementChildrenHavingBase):
kind = "STATEMENT_DEL_VARIABLE"
named_children = ( "variable_ref", )
def __init__(self, variable_ref, tolerant, source_ref):
assert variable_ref is not None
assert variable_ref.isTargetVariableRef()
assert tolerant is True or tolerant is False
StatementChildrenHavingBase.__init__(
self,
values = {
"variable_ref" : variable_ref
},
source_ref = source_ref
)
self.tolerant = tolerant
def getDetail(self):
variable_ref = self.getTargetVariableRef()
variable = variable_ref.getVariable()
if variable is not None:
return "to variable %s" % variable
else:
return "to variable %s" % self.getTargetVariableRef()
def getDetails(self):
return { "tolerant" : self.tolerant }
# TODO: Value propagation needs to make a difference based on this.
def isTolerant(self):
return self.tolerant
getTargetVariableRef = StatementChildrenHavingBase.childGetter(
"variable_ref"
)
def computeStatement(self, constraint_collection):
variable = self.getTargetVariableRef().getVariable()
trace = constraint_collection.getVariableCurrentTrace( variable )
# Optimize away tolerant "del" that is not needed.
if trace.isUninitTrace():
if self.isTolerant():
return (
None,
"new_statements",
"Removed tolerate del without effect."
)
constraint_collection.onVariableDel(
target_node = self.getTargetVariableRef()
)
return self, None, None
def mayHaveSideEffects(self):
return True
def mayRaiseException(self, exception_type):
if self.tolerant:
return False
else:
return True
class StatementDelAttribute(StatementChildrenHavingBase):
kind = "STATEMENT_DEL_ATTRIBUTE"
named_children = ( "expression", )
def __init__(self, expression, attribute_name, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"expression" : expression
},
source_ref = source_ref
)
self.attribute_name = attribute_name
def getDetails(self):
return { "attribute" : self.attribute_name }
def getDetail(self):
return "to attribute %s" % self.attribute_name
def getAttributeName(self):
return self.attribute_name
def setAttributeName(self, attribute_name):
self.attribute_name = attribute_name
getLookupSource = StatementChildrenHavingBase.childGetter( "expression" )
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getLookupSource() )
lookup_source = self.getLookupSource()
if lookup_source.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementExpressionOnlyReplacementNode
return makeStatementExpressionOnlyReplacementNode(
expression = lookup_source,
node = self
)
return self, None, None
class StatementDelSubscript(StatementChildrenHavingBase):
kind = "STATEMENT_DEL_SUBSCRIPT"
named_children = ( "expression", "subscript" )
def __init__(self, expression, subscript, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"expression" : expression,
"subscript" : subscript
},
source_ref = source_ref
)
getSubscribed = StatementChildrenHavingBase.childGetter( "expression" )
getSubscript = StatementChildrenHavingBase.childGetter( "subscript" )
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getSubscribed() )
subscribed = self.getSubscribed()
if subscribed.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = subscribed,
node = self
)
return result, "new_raise", """\
Subscript del raises exception in subscribed value, removed del"""
constraint_collection.onExpression( self.getSubscript() )
subscript = self.getSubscript()
if subscript.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
subscribed,
subscript
)
)
return result, "new_raise", """\
Subscript del raises exception in subscribt value, removed del"""
return self, None, None
class StatementDelSlice(StatementChildrenHavingBase):
kind = "STATEMENT_DEL_SLICE"
named_children = ( "expression", "lower", "upper" )
def __init__(self, expression, lower, upper, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"expression" : expression,
"lower" : lower,
"upper" : upper
},
source_ref = source_ref
)
getLookupSource = StatementChildrenHavingBase.childGetter( "expression" )
getLower = StatementChildrenHavingBase.childGetter( "lower" )
getUpper = StatementChildrenHavingBase.childGetter( "upper" )
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getLookupSource() )
lookup_source = self.getLookupSource()
if lookup_source.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = lookup_source,
node = self
)
return result, "new_raise", """\
Slice del raises exception in sliced value, removed del"""
constraint_collection.onExpression( self.getLower(), allow_none = True )
lower = self.getLower()
if lower is not None and lower.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
lookup_source,
lower
)
)
return result, "new_raise", """
Slice del raises exception in lower slice boundary value, removed del"""
constraint_collection.onExpression( self.getUpper(), allow_none = True )
upper = self.getUpper()
if upper is not None and upper.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
lookup_source,
lower,
upper
)
)
return result, "new_raise", """
Slice del raises exception in upper slice boundary value, removed del"""
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/SliceNodes.py 0000644 0001750 0001750 00000006516 12265264105 020360 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Slice nodes.
Slices are important when working with lists. Tracking them can allow to achieve more
compact code, or predict results at compile time.
There will be a method "computeExpressionSlice" to aid predicting them.
"""
from .NodeBases import ExpressionChildrenHavingBase
from .NodeMakingHelpers import convertNoneConstantToNone
class ExpressionSliceLookup(ExpressionChildrenHavingBase):
kind = "EXPRESSION_SLICE_LOOKUP"
named_children = (
"expression",
"lower",
"upper"
)
checkers = {
"upper" : convertNoneConstantToNone,
"lower" : convertNoneConstantToNone
}
def __init__(self, expression, lower, upper, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"expression" : expression,
"upper" : upper,
"lower" : lower
},
source_ref = source_ref
)
getLookupSource = ExpressionChildrenHavingBase.childGetter( "expression" )
getLower = ExpressionChildrenHavingBase.childGetter("lower")
setLower = ExpressionChildrenHavingBase.childSetter("lower")
getUpper = ExpressionChildrenHavingBase.childGetter("upper")
setUpper = ExpressionChildrenHavingBase.childSetter("upper")
def computeExpression(self, constraint_collection):
lookup_source = self.getLookupSource()
return lookup_source.computeExpressionSlice(
lookup_node = self,
lower = self.getLower(),
upper = self.getUpper(),
constraint_collection = constraint_collection
)
def isKnownToBeIterable(self, count):
# TODO: Should ask SlicetRegistry
return None
class ExpressionSliceObject(ExpressionChildrenHavingBase):
kind = "EXPRESSION_SLICE_OBJECT"
named_children = ( "lower", "upper", "step" )
def __init__(self, lower, upper, step, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"upper" : upper,
"lower" : lower,
"step" : step
},
source_ref = source_ref
)
getLower = ExpressionChildrenHavingBase.childGetter( "lower" )
getUpper = ExpressionChildrenHavingBase.childGetter( "upper" )
getStep = ExpressionChildrenHavingBase.childGetter( "step" )
def computeExpression(self, constraint_collection):
# TODO: Not much to do, potentially simplify to slice instead?
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/ParameterSpecs.py 0000644 0001750 0001750 00000042117 12265264105 021243 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" This module maintains the parameter specification classes.
These are used for function, lambdas, generators. They are also a factory
for the respective variable objects. One of the difficulty of Python and
its parameter parsing is that they are allowed to be nested like this:
(a,b), c
Much like in assignments, which are very similar to parameters, except
that parameters may also be assigned from a dictionary, they are no less
flexible.
"""
class TooManyArguments(Exception):
def __init__(self, real_exception):
Exception.__init__( self )
self.real_exception = real_exception
def getRealException(self):
return self.real_exception
from nuitka import Variables, Utils
class ParameterSpecTuple:
def __init__(self, normal_args, nest_count = 1):
self.normal_args = tuple( normal_args )
self.nest_count = nest_count
self.owner = None
self.normal_variables = None
def getNormalParameters(self):
return self.normal_variables
def setOwner(self, owner):
assert self.owner is None
self.owner = owner
self.normal_variables = []
for count, normal_arg in enumerate( self.normal_args ):
if type( normal_arg ) == str:
normal_variable = Variables.ParameterVariable(
owner = self.owner,
parameter_name = normal_arg,
kw_only = False
)
elif type( normal_arg ) == tuple:
sub_parameter_spec = ParameterSpecTuple(
normal_args = normal_arg,
nest_count = self.nest_count + 1
)
sub_parameter_spec.setOwner( self.owner )
sub_parameter_name = "Unpackable_%s_%s" % (
self.nest_count,
count+1
)
normal_variable = Variables.NestedParameterVariable(
owner = self.owner,
parameter_name = sub_parameter_name,
parameter_spec = sub_parameter_spec
)
else:
assert False, normal_arg
self.normal_variables.append( normal_variable )
def getVariables(self):
result = []
for variable in self.normal_variables:
if variable.isNestedParameterVariable():
result += variable.getVariables()
else:
result.append( variable )
return result
def hasNestedParameterVariables(self):
for variable in self.normal_variables:
if variable.isNestedParameterVariable():
return True
else:
return False
def getAllVariables(self):
result = self.normal_variables[:]
for variable in self.normal_variables:
if variable.isNestedParameterVariable():
result += variable.getAllVariables()
return result
def getAllNames(self):
result = []
def extractArg(normal_arg):
if type( normal_arg ) is str:
result.append( normal_arg )
elif type( normal_arg ) is tuple:
for normal_arg in normal_arg:
extractArg( normal_arg )
else:
assert False
for normal_arg in self.normal_args:
extractArg( normal_arg )
return result
def getTopLevelVariables(self):
return self.normal_variables
def getParameterNames(self):
return Variables.getNames( self.getVariables() )
class ParameterSpec(ParameterSpecTuple):
def __init__( self, name, normal_args, kw_only_args, list_star_arg,
dict_star_arg, default_count ):
assert None not in normal_args
self.name = name
self.nest_count = 1
ParameterSpecTuple.__init__( self, normal_args )
assert list_star_arg is None or type( list_star_arg ) is str, \
list_star_arg
assert dict_star_arg is None or type( dict_star_arg ) is str, \
dict_star_arg
self.list_star_arg = list_star_arg
self.dict_star_arg = dict_star_arg
self.list_star_variable = None
self.dict_star_variable = None
self.default_count = default_count
self.kw_only_args = tuple( kw_only_args )
self.kw_only_variables = None
def checkValid(self):
arg_names = self.getAllNames()
# Check for duplicate arguments, could happen.
for arg_name in arg_names:
if arg_names.count( arg_name ) != 1:
return "duplicate argument '%s' in function definition" % arg_name
else:
return None
def __repr__(self):
parts = [ str(normal_arg) for normal_arg in self.normal_args ]
if self.list_star_arg is not None:
parts.append( "*%s" % self.list_star_arg )
if self.dict_star_variable is not None:
parts.append( "**%s" % self.dict_star_variable )
if parts:
return "" % ",".join( parts )
else:
return ""
def getArgumentCount(self):
return len( self.normal_args )
def setOwner(self, owner):
if self.owner is not None:
return
ParameterSpecTuple.setOwner( self, owner )
if self.list_star_arg:
self.list_star_variable = Variables.ParameterVariable( owner, self.list_star_arg, False )
else:
self.list_star_variable = None
if self.dict_star_arg:
self.dict_star_variable = Variables.ParameterVariable(
owner = owner,
parameter_name = self.dict_star_arg,
kw_only = False
)
else:
self.dict_star_variable = None
self.kw_only_variables = [
Variables.ParameterVariable( self.owner, kw_only_arg, True )
for kw_only_arg in
self.kw_only_args
]
def isEmpty(self):
return len( self.normal_args ) == 0 and self.list_star_arg is None and \
self.dict_star_arg is None and len( self.kw_only_args ) == 0
def getDefaultParameterVariables(self):
result = ParameterSpecTuple.getTopLevelVariables( self )
return result[ len( self.normal_args ) - self.default_count : ]
def getDefaultParameterNames(self):
return self.normal_args[ \
len( self.normal_args ) - self.default_count : ]
def getDefaultCount(self):
return self.default_count
def hasDefaultParameters(self):
return self.getDefaultCount() > 0
def getVariables(self):
result = ParameterSpecTuple.getVariables( self )[:]
if self.list_star_variable is not None:
result.append( self.list_star_variable )
if self.dict_star_variable is not None:
result.append( self.dict_star_variable )
return result + self.kw_only_variables
def getTopLevelVariables(self):
result = ParameterSpecTuple.getTopLevelVariables( self )
return result + self.kw_only_variables
def getAllVariables(self):
result = ParameterSpecTuple.getAllVariables( self )[:]
if self.list_star_variable is not None:
result.append( self.list_star_variable )
if self.dict_star_variable is not None:
result.append( self.dict_star_variable )
return result + self.kw_only_variables
def getAllNames(self):
result = ParameterSpecTuple.getAllNames( self )[:]
if self.list_star_arg is not None:
result.append( self.list_star_arg )
if self.dict_star_arg is not None:
result.append( self.dict_star_arg )
return result + list( self.kw_only_args )
def getStarListArgumentName(self):
return self.list_star_arg
def getListStarArgVariable(self):
return self.list_star_variable
def getStarDictArgumentName(self):
return self.dict_star_arg
def getDictStarArgVariable(self):
return self.dict_star_variable
def getKwOnlyVariables(self):
return self.kw_only_variables
def allowsKeywords(self):
# Abstract method, pylint: disable=R0201
return True
def getKeywordRefusalText(self):
return "%s() takes no keyword arguments" % self.name
def getArgumentNames(self):
return self.normal_args
def getKwOnlyParameterNames(self):
return self.kw_only_args
def getKwOnlyParameterCount(self):
return len( self.kw_only_args )
def getCoArgNames(self):
result = []
for count, variable in enumerate( self.getTopLevelVariables() ):
if variable.isNestedParameterVariable():
result.append( ".%d" % count )
else:
result.append( variable.getName() )
if self.list_star_variable is not None:
result.append( self.list_star_arg )
if self.dict_star_variable is not None:
result.append( self.dict_star_arg )
return result
# Note: Based loosley on "inspect.getcallargs" with corrections.
def matchCall(func_name, args, star_list_arg, star_dict_arg, num_defaults, positional, pairs, improved = False ):
# This is of incredible code complexity, but there really is no other way to
# express this with less statements, branches, or variables.
# pylint: disable=R0914,R0912,R0915
assert type( positional ) is tuple
assert type( pairs ) in ( tuple, list )
# Make a copy, we are going to modify it.
pairs = list( pairs )
result = {}
assigned_tuple_params = []
def assign(arg, value):
if type( arg ) is str:
# Normal case:
result[ arg ] = value
else:
# Tuple argument case:
assigned_tuple_params.append( arg )
value = iter( value )
for i, subarg in enumerate( arg ):
try:
subvalue = next( value )
except StopIteration:
raise TooManyArguments(
ValueError(
"need more than %d %s to unpack" % (
i,
"values" if i > 1 else "value"
)
)
)
# Recurse into tuple argument values, could be more tuples.
assign( subarg, subvalue )
# Check that not too many values we provided.
try:
next( value )
except StopIteration:
pass
else:
raise TooManyArguments(
ValueError( "too many values to unpack" )
)
def isAssigned(arg):
if type( arg ) is str:
return arg in result
return arg in assigned_tuple_params
num_pos = len( positional )
num_total = num_pos + len( pairs )
num_args = len( args )
for arg, value in zip( args, positional ):
assign( arg, value )
# Python3 does this check earlier.
if Utils.python_version >= 300 and not star_dict_arg:
for pair in pairs:
if pair[0] not in args:
message = "'%s' is an invalid keyword argument for this function" % pair[0]
raise TooManyArguments(
TypeError( message )
)
if star_list_arg:
if num_pos > num_args:
assign( star_list_arg, positional[ -(num_pos-num_args) : ] )
else:
assign( star_list_arg, () )
elif 0 < num_args < num_total:
if num_defaults == 0:
if num_args != 1:
raise TooManyArguments(
TypeError(
"%s expected %d arguments, got %d" % (
func_name,
num_args,
num_total
)
)
)
raise TooManyArguments(
TypeError(
"%s() takes exactly %s (%d given)" % (
func_name,
"one argument" if num_args == 1 else "%d arguments" % num_args,
num_total
)
)
)
else:
raise TooManyArguments(
TypeError(
"%s() takes at most %d %s (%d given)" % (
func_name,
num_args,
"argument" if num_args == 1 else "arguments",
num_total
)
)
)
elif num_args == 0 and num_total:
if star_dict_arg:
if num_pos:
# Could use num_pos, but Python also uses num_total.
raise TooManyArguments(
TypeError(
"%s() takes exactly 0 arguments (%d given)" % (
func_name,
num_total
)
)
)
else:
raise TooManyArguments(
TypeError(
"%s() takes no arguments (%d given)" % (
func_name,
num_total
)
)
)
named_argument_names = [
pair[0]
for pair in
pairs
]
for arg in args:
if type( arg ) is str and arg in named_argument_names:
if isAssigned( arg ):
raise TooManyArguments(
TypeError(
"%s() got multiple values for keyword argument '%s'" % (
func_name,
arg
)
)
)
else:
new_pairs = []
for pair in pairs:
if arg == pair[0]:
assign( arg, pair[1] )
else:
new_pairs.append( pair )
assert len( new_pairs ) == len( pairs ) - 1
pairs = new_pairs
# Fill in any missing values with the None to indicate "default".
if num_defaults > 0:
for arg in args[ -num_defaults : ]:
if not isAssigned( arg ):
assign( arg, None )
if star_dict_arg:
assign( star_dict_arg, pairs )
elif pairs:
unexpected = next( iter( dict( pairs ) ) )
if improved:
message = "%s() got an unexpected keyword argument '%s'" % (
func_name,
unexpected
)
else:
message = "'%s' is an invalid keyword argument for this function" % unexpected
raise TooManyArguments(
TypeError( message )
)
unassigned = num_args - len(
[
arg
for arg in args
if isAssigned( arg )
]
)
if unassigned:
num_required = num_args - num_defaults
if num_required > 0 or improved:
if num_defaults == 0 and num_args != 1:
raise TooManyArguments(
TypeError(
"%s expected %d arguments, got %d" % (
func_name,
num_args,
num_total
)
)
)
if num_required == 1:
arg_desc = "1 argument"
else:
arg_desc = "%d arguments" % num_required
raise TooManyArguments(
TypeError(
"%s() takes %s %s (%d given)" % (
func_name,
"at least" if num_defaults > 0 else "exactly",
arg_desc,
num_total
)
)
)
else:
raise TooManyArguments(
TypeError(
"%s expected %s%s, got %d" % (
func_name,
( "at least " if Utils.python_version < 300 else "" )
if num_defaults > 0
else "exactly ",
"%d arguments" % num_required,
num_total
)
)
)
return result
Nuitka-0.5.0.1/nuitka/nodes/ContainerOperationNodes.py 0000644 0001750 0001750 00000013500 12265264105 023113 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Operations on Containers.
"""
from .NodeBases import (
ExpressionChildrenHavingBase,
StatementChildrenHavingBase
)
class ExpressionListOperationAppend(ExpressionChildrenHavingBase):
kind = "EXPRESSION_LIST_OPERATION_APPEND"
named_children = ( "list", "value" )
def __init__(self, liste, value, source_ref):
assert liste is not None
assert value is not None
ExpressionChildrenHavingBase.__init__(
self,
values = {
"list" : liste,
"value" : value
},
source_ref = source_ref
)
getList = ExpressionChildrenHavingBase.childGetter( "list" )
getValue = ExpressionChildrenHavingBase.childGetter( "value" )
def computeExpression(self, constraint_collection):
constraint_collection.removeKnowledge( self.getList() )
return self, None, None
class ExpressionSetOperationAdd(ExpressionChildrenHavingBase):
kind = "EXPRESSION_SET_OPERATION_ADD"
named_children = ( "set", "value" )
def __init__(self, sete, value, source_ref):
assert sete is not None
assert value is not None
ExpressionChildrenHavingBase.__init__(
self,
values = {
"set" : sete,
"value" : value
},
source_ref = source_ref
)
getSet = ExpressionChildrenHavingBase.childGetter( "set" )
getValue = ExpressionChildrenHavingBase.childGetter( "value" )
def computeExpression(self, constraint_collection):
constraint_collection.removeKnowledge( self.getSet() )
return self, None, None
class ExpressionDictOperationSet(ExpressionChildrenHavingBase):
kind = "EXPRESSION_DICT_OPERATION_SET"
named_children = ( "dict", "key", "value" )
def __init__(self, dicte, key, value, source_ref):
assert dicte is not None
assert key is not None
assert value is not None
ExpressionChildrenHavingBase.__init__(
self,
values = {
"dict" : dicte,
"key" : key,
"value" : value
},
source_ref = source_ref
)
getDict = ExpressionChildrenHavingBase.childGetter( "dict" )
getKey = ExpressionChildrenHavingBase.childGetter( "key" )
getValue = ExpressionChildrenHavingBase.childGetter( "value" )
def computeExpression(self, constraint_collection):
constraint_collection.removeKnowledge( self.getDict() )
return self, None, None
class StatementDictOperationRemove(StatementChildrenHavingBase):
kind = "STATEMENT_DICT_OPERATION_REMOVE"
named_children = ( "dict", "key" )
def __init__(self, dicte, key, source_ref):
assert dicte is not None
assert key is not None
StatementChildrenHavingBase.__init__(
self,
values = {
"dict" : dicte,
"key" : key,
},
source_ref = source_ref
)
getDict = StatementChildrenHavingBase.childGetter( "dict" )
getKey = StatementChildrenHavingBase.childGetter( "key" )
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getDict() )
dicte = self.getDict()
if dicte.willRaiseException( BaseException ):
from .NodeMakingHelpers import \
makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = dicte,
node = self
)
return result, "new_raise", """\
Dictionary remove already raises implicitely accessing dictionary."""
constraint_collection.onExpression( self.getKey() )
key = self.getKey()
if key.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
dicte,
key
)
)
return result, "new_node", """
Dictionary remove already raises implicitely building key."""
# TODO: Be less lossly about it.
constraint_collection.removeKnowledge( dicte )
return self, None, None
class ExpressionDictOperationGet(ExpressionChildrenHavingBase):
kind = "EXPRESSION_DICT_OPERATION_GET"
named_children = ( "dict", "key" )
def __init__(self, dicte, key, source_ref):
assert dicte is not None
assert key is not None
ExpressionChildrenHavingBase.__init__(
self,
values = {
"dict" : dicte,
"key" : key,
},
source_ref = source_ref
)
getDict = ExpressionChildrenHavingBase.childGetter( "dict" )
getKey = ExpressionChildrenHavingBase.childGetter( "key" )
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/BuiltinDictNodes.py 0000644 0001750 0001750 00000006342 12265264105 021530 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Node the calls to the 'dict' builtin.
"""
from .NodeBases import ExpressionChildrenHavingBase
from .ConstantRefNodes import ExpressionConstantRef
from .ContainerMakingNodes import ExpressionKeyValuePair
from nuitka.optimizations.BuiltinOptimization import builtin_dict_spec
class ExpressionBuiltinDict(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_DICT"
named_children = ( "pos_arg", "pairs" )
def __init__(self, pos_arg, pairs, source_ref):
assert type( pos_arg ) not in ( tuple, list ), source_ref
assert type( pairs ) in ( tuple, list ), source_ref
ExpressionChildrenHavingBase.__init__(
self,
values = {
"pos_arg" : pos_arg,
"pairs" : tuple(
ExpressionKeyValuePair(
ExpressionConstantRef( key, source_ref ),
value,
value.getSourceReference()
)
for key, value in
pairs
)
},
source_ref = source_ref
)
getPositionalArgument = ExpressionChildrenHavingBase.childGetter( "pos_arg" )
getNamedArgumentPairs = ExpressionChildrenHavingBase.childGetter( "pairs" )
def hasOnlyConstantArguments(self):
pos_arg = self.getPositionalArgument()
if pos_arg is not None and not pos_arg.isCompileTimeConstant():
return False
for arg_pair in self.getNamedArgumentPairs():
if not arg_pair.getKey().isCompileTimeConstant():
return False
if not arg_pair.getValue().isCompileTimeConstant():
return False
return True
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
if self.hasOnlyConstantArguments():
pos_arg = self.getPositionalArgument()
if pos_arg is not None:
pos_args = ( pos_arg, )
else:
pos_args = None
from .NodeMakingHelpers import getComputationResult
return getComputationResult(
node = self,
computation = lambda : builtin_dict_spec.simulateCall(
( pos_args, self.getNamedArgumentPairs() )
),
description = "Replace dict call with constant arguments."
)
else:
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/TypeNodes.py 0000644 0001750 0001750 00000010454 12265264105 020236 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" The type1 node.
This one just determines types. It's great for optimization. We may be able to
predict its value, but knowing it. In that case, we have a builtin name
reference for that type to convert to, or when checking the result of it, we
will then know it's limited after the fact.
"""
from .NodeBases import (
ExpressionBuiltinSingleArgBase,
ExpressionChildrenHavingBase
)
from nuitka.Builtins import builtin_names
class ExpressionBuiltinType1(ExpressionBuiltinSingleArgBase):
kind = "EXPRESSION_BUILTIN_TYPE1"
def computeExpression(self, constraint_collection):
value = self.getValue()
if value.isCompileTimeConstant():
value = value.getCompileTimeConstant()
type_name = value.__class__.__name__
from .BuiltinRefNodes import (
ExpressionBuiltinAnonymousRef,
ExpressionBuiltinRef
)
if type_name in builtin_names:
new_node = ExpressionBuiltinRef(
builtin_name = type_name,
source_ref = self.getSourceReference()
)
else:
new_node = ExpressionBuiltinAnonymousRef(
builtin_name = type_name,
source_ref = self.getSourceReference()
)
return (
new_node,
"new_builtin",
"Replaced predictable type lookup with builtin type '%s'." % (
type_name
)
)
return self, None, None
def computeExpressionDrop(self, statement, constraint_collection):
from .NodeMakingHelpers import \
makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = self.getValue(),
node = statement
)
return result, "new_statements", """\
Removed type taking for unused result."""
class ExpressionBuiltinSuper(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_SUPER"
named_children = ( "type", "object" )
def __init__(self, super_type, super_object, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"type" : super_type,
"object" : super_object
},
source_ref = source_ref )
getType = ExpressionChildrenHavingBase.childGetter( "type" )
getObject = ExpressionChildrenHavingBase.childGetter( "object" )
def computeExpression(self, constraint_collection):
# TODO: Quite some cases should be possible to predict.
return self, None, None
class ExpressionBuiltinIsinstance(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_ISINSTANCE"
named_children = ( "instance", "cls" )
def __init__(self, instance, cls, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"instance" : instance,
"cls" : cls
},
source_ref = source_ref )
getInstance = ExpressionChildrenHavingBase.childGetter( "instance" )
getCls = ExpressionChildrenHavingBase.childGetter( "cls" )
def computeExpression(self, constraint_collection):
# TODO: Quite some cases should be possible to predict.
return self, None, None
def mayProvideReference(self):
# Dedicated code returns "True" or "False" only, which requires no reference,
# except for rich comparisons, which do.
return False
Nuitka-0.5.0.1/nuitka/nodes/ExecEvalNodes.py 0000644 0001750 0001750 00000022477 12265264105 021021 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Nodes concern with exec and eval builtins.
These are the dynamic codes, and as such rather difficult. We would like
to eliminate or limit their impact as much as possible, but it's difficult
to do.
"""
from nuitka import Utils
from .NodeBases import (
ExpressionChildrenHavingBase,
StatementChildrenHavingBase,
)
# Delayed import into multiple branches is not an issue, pylint: disable=W0404
class ExpressionBuiltinEval(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_EVAL"
named_children = ( "source", "globals", "locals" )
def __init__(self, source_code, globals_arg, locals_arg, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"source" : source_code,
"globals" : globals_arg,
"locals" : locals_arg,
},
source_ref = source_ref
)
getSourceCode = ExpressionChildrenHavingBase.childGetter( "source" )
getGlobals = ExpressionChildrenHavingBase.childGetter( "globals" )
getLocals = ExpressionChildrenHavingBase.childGetter( "locals" )
def computeExpression(self, constraint_collection):
# TODO: Attempt for constant values to do it.
return self, None, None
# Note: Python3 only so far.
if Utils.python_version >= 300:
class ExpressionBuiltinExec(ExpressionBuiltinEval):
kind = "EXPRESSION_BUILTIN_EXEC"
def __init__(self, source_code, globals_arg, locals_arg, source_ref):
ExpressionBuiltinEval.__init__(
self,
source_code = source_code,
globals_arg = globals_arg,
locals_arg = locals_arg,
source_ref = source_ref
)
def needsLocalsDict(self):
return True
def computeExpression(self, constraint_collection):
# TODO: Attempt for constant values to do it.
return self, None, None
def computeExpressionDrop(self, statement, constraint_collection):
if self.getParentVariableProvider().isEarlyClosure():
from .ExecEvalNodes import StatementExec
result = StatementExec(
source_code = self.getSourceCode(),
globals_arg = self.getGlobals(),
locals_arg = self.getLocals(),
source_ref = self.getSourceReference()
)
return result, "new_statements", """\
Replaced builtin exec call to exec statement in early closure context."""
else:
return statement, None, None
# Note: Python2 only
if Utils.python_version < 300:
class ExpressionBuiltinExecfile(ExpressionBuiltinEval):
kind = "EXPRESSION_BUILTIN_EXECFILE"
named_children = ( "source", "globals", "locals" )
def __init__(self, source_code, globals_arg, locals_arg, source_ref):
ExpressionBuiltinEval.__init__(
self,
source_code = source_code,
globals_arg = globals_arg,
locals_arg = locals_arg,
source_ref = source_ref
)
def needsLocalsDict(self):
return True
def computeExpressionDrop(self, statement, constraint_collection):
# In this case, the copy-back must be done and will only be done
# correctly by the code for exec statements.
provider = self.getParentVariableProvider()
if provider.isExpressionFunctionBody() and \
provider.isClassDictCreation():
from .ExecEvalNodes import StatementExec
result = StatementExec(
source_code = self.getSourceCode(),
globals_arg = self.getGlobals(),
locals_arg = self.getLocals(),
source_ref = self.getSourceReference()
)
return result, "new_statements", """\
Changed execfile to exec on class level."""
else:
return statement, None, None
# TODO: Find a place for this. Potentially as an attribute of nodes themselves.
def _couldBeNone(node):
if node is None:
return True
elif node.isExpressionMakeDict():
return False
elif node.isExpressionBuiltinGlobals() or \
node.isExpressionBuiltinLocals() or \
node.isExpressionBuiltinDir0() or \
node.isExpressionBuiltinVars():
return False
else:
# assert False, node
return True
class StatementExec(StatementChildrenHavingBase):
kind = "STATEMENT_EXEC"
named_children = ( "source", "globals", "locals" )
def __init__(self, source_code, globals_arg, locals_arg, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"globals" : globals_arg,
"locals" : locals_arg,
"source" : source_code
},
source_ref = source_ref,
)
def setChild(self, name, value):
if name in ( "globals", "locals" ):
from .NodeMakingHelpers import convertNoneConstantToNone
value = convertNoneConstantToNone( value )
return StatementChildrenHavingBase.setChild( self, name, value )
getSourceCode = StatementChildrenHavingBase.childGetter( "source" )
getGlobals = StatementChildrenHavingBase.childGetter( "globals" )
getLocals = StatementChildrenHavingBase.childGetter( "locals" )
def needsLocalsDict(self):
return _couldBeNone( self.getGlobals() ) or \
self.getGlobals().isExpressionBuiltinLocals() or \
( self.getLocals() is not None and \
self.getLocals().isExpressionBuiltinLocals()
)
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getSourceCode() )
source_code = self.getSourceCode()
if source_code.willRaiseException( BaseException ):
result = source_code
return (
result,
"new_raise",
"""\
Exec statement raises implicitely when determining source code argument."""
)
constraint_collection.onExpression(
expression = self.getGlobals(),
allow_none = True
)
globals_arg = self.getGlobals()
if globals_arg is not None and \
globals_arg.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
source_code,
globals_arg
)
)
return (
result,
"new_raise",
"""\
Exec statement raises implicitely when determining globals argument."""
)
constraint_collection.onExpression(
expression = self.getLocals(),
allow_none = True
)
locals_arg = self.getLocals()
if locals_arg is not None and \
locals_arg.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = (
source_code,
globals_arg,
locals_arg
)
)
return (
result,
"new_raise",
"""\
Exec statement raises implicitely when determining locals argument."""
)
str_value = self.getSourceCode().getStrValue()
# TODO: This is not yet completely working
if False and str_value is not None:
from nuitka.tree.Building import (
buildParseTree,
completeVariableClosures
)
exec_body = buildParseTree(
provider = self.getParentVariableProvider(),
source_code = str_value.getConstant(),
source_ref = str_value.getSourceReference().getExecReference(
value = True
),
is_module = False,
is_main = False
)
# Need to re-visit things.
self.replaceWith( exec_body )
completeVariableClosures( self.getParentModule() )
return (
exec_body,
"new_statements",
"Inlined constant exec statement."
)
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/PrintNodes.py 0000644 0001750 0001750 00000015763 12265264105 020421 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Print nodes.
Right now there is only the print statement, but in principle, there should also be the
print function here. These perform output, which can be combined if possible, and could be
detected to fail, which would be perfect.
Predicting the behavior of 'print' is not trivial at all, due to many special cases.
"""
from .NodeBases import StatementChildrenHavingBase
# Delayed import into multiple branches is not an issue, pylint: disable=W0404
class StatementPrint(StatementChildrenHavingBase):
kind = "STATEMENT_PRINT"
named_children = ( "dest", "values" )
def __init__(self, dest, values, newline, source_ref):
assert values or newline
StatementChildrenHavingBase.__init__(
self,
values = {
"values" : tuple( values ),
"dest" : dest
},
source_ref = source_ref
)
self.newline = newline
def isNewlinePrint(self):
return self.newline
def removeNewlinePrint(self):
self.newline = False
getDestination = StatementChildrenHavingBase.childGetter( "dest" )
getValues = StatementChildrenHavingBase.childGetter( "values" )
setValues = StatementChildrenHavingBase.childSetter( "values" )
def computeStatement(self, constraint_collection):
constraint_collection.onExpression( self.getDestination(), allow_none = True )
dest = self.getDestination()
if dest is not None and dest.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = dest,
node = self
)
return result, "new_raise", """\
Known exception raise in print statement destination converted to explicit raise."""
for count, value in enumerate( self.getValues() ):
constraint_collection.onExpression( value )
value = self.getValues()[ count ]
if value.willRaiseException( BaseException ):
# Trim to values up to this from this statement.
values = self.getValues()
values = list( values )[ : values.index( value ) ]
from .NodeMakingHelpers import (
makeStatementExpressionOnlyReplacementNode,
makeStatementsSequenceReplacementNode
)
if values:
result = makeStatementsSequenceReplacementNode(
statements = (
StatementPrint(
dest = self.getDestination(),
values = values,
newline = False,
source_ref = self.source_ref
),
makeStatementExpressionOnlyReplacementNode(
expression = value,
node = self
)
),
node = self
)
else:
result = makeStatementExpressionOnlyReplacementNode(
expression = value,
node = self
)
return result, "new_raise", """\
Known exception raise in print statement arguments converted to explicit raise."""
printeds = self.getValues()
for count in range( len( printeds ) - 1 ):
if printeds[ count ].isExpressionConstantRef():
new_value = printeds[ count ].getConstant()
# Above code should have replaced this already.
assert type( new_value ) is str, self
stop_count = count + 1
while True:
candidate = printeds[ stop_count ]
if candidate.isExpressionConstantRef() and candidate.isStringConstant():
if not new_value.endswith( "\t" ):
new_value += " "
new_value += candidate.getConstant()
stop_count += 1
if stop_count >= len( printeds ):
break
else:
break
if stop_count != count + 1:
from .NodeMakingHelpers import makeConstantReplacementNode
new_node = makeConstantReplacementNode(
constant = new_value,
node = printeds[ count ]
)
new_printeds = printeds[ : count ] + ( new_node, ) + printeds[ stop_count: ]
self.setValues( new_printeds )
constraint_collection.signalChange(
"new_expression",
printeds[ count ].getSourceReference(),
"Combined print string arguments at compile time"
)
break
if dest is None:
values = self.getValues()
if values:
if values[0].isExpressionSideEffects():
from .NodeMakingHelpers import (
makeStatementExpressionOnlyReplacementNode,
makeStatementsSequenceReplacementNode
)
statements = [
makeStatementExpressionOnlyReplacementNode(
side_effect,
self
)
for side_effect in
values[0].getSideEffects()
]
statements.append( self )
self.setValues(
( values[0].getExpression(), ) + values[ 1: ]
)
result = makeStatementsSequenceReplacementNode(
statements = statements,
node = self,
)
return result, "new_statements", """\
Side effects first printed item promoted to statements."""
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/BuiltinVarsNodes.py 0000644 0001750 0001750 00000003113 12265264105 021551 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Builtin vars node.
Not used much, esp. not in the form with arguments. Maybe used in some meta programming,
and hopefully can be predicted, because at run time, it is hard to support.
"""
from .NodeBases import ExpressionChildrenHavingBase
class ExpressionBuiltinVars(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_VARS"
named_children = ( "source", )
def __init__(self, source, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"source" : source,
},
source_ref = source_ref
)
getSource = ExpressionChildrenHavingBase.childGetter( "source" )
def computeExpression(self, constraint_collection):
# TODO: Should be possible. pylint: disable=W0613
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/BuiltinOpenNodes.py 0000644 0001750 0001750 00000003716 12265264105 021550 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Node the calls to the 'open' builtin.
This is a rather two sided beast, as it may be read or write. And we would like to be able
to track it, so we can include files into the executable, or write more efficiently.
"""
from .NodeBases import ExpressionChildrenHavingBase
class ExpressionBuiltinOpen(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_OPEN"
named_children = ( "filename", "mode", "buffering" )
def __init__(self, filename, mode, buffering, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"filename" : filename,
"mode" : mode,
"buffering" : buffering
},
source_ref = source_ref
)
getFilename = ExpressionChildrenHavingBase.childGetter( "filename" )
getMode = ExpressionChildrenHavingBase.childGetter( "mode" )
getBuffering = ExpressionChildrenHavingBase.childGetter( "buffering" )
def computeExpression(self, constraint_collection):
# Note: Quite impossible to predict without further assumptions, but we could look
# at the arguments at least, pylint: disable=W0613
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/ComparisonNodes.py 0000644 0001750 0001750 00000016610 12265264105 021427 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Nodes for comparisons.
"""
from .NodeBases import ExpressionChildrenHavingBase
from nuitka import PythonOperators
# Delayed import into multiple branches is not an issue, pylint: disable=W0404
class ExpressionComparison(ExpressionChildrenHavingBase):
kind = "EXPRESSION_COMPARISON"
named_children = ( "left", "right" )
def __init__(self, left, right, comparator, source_ref):
assert left.isExpression()
assert right.isExpression()
assert type( comparator ) is str, comparator
assert comparator in PythonOperators.all_comparison_functions
ExpressionChildrenHavingBase.__init__(
self,
values = {
"left" : left,
"right" : right
},
source_ref = source_ref
)
self.comparator = comparator
if comparator in ( "Is", "IsNot" ):
assert self.__class__ is not ExpressionComparison
def getOperands(self):
return (
self.getLeft(),
self.getRight()
)
getLeft = ExpressionChildrenHavingBase.childGetter( "left" )
getRight = ExpressionChildrenHavingBase.childGetter( "right" )
def getComparator(self):
return self.comparator
def getDetails(self):
return { "comparator" : self.comparator }
def getSimulator(self):
return PythonOperators.all_comparison_functions[self.comparator]
def computeExpression(self, constraint_collection):
# Left and right is all we need, pylint: disable=W0613
left, right = self.getOperands()
if left.isCompileTimeConstant() and right.isCompileTimeConstant():
left_value = left.getCompileTimeConstant()
right_value = right.getCompileTimeConstant()
from .NodeMakingHelpers import getComputationResult
return getComputationResult(
node = self,
computation = lambda : self.getSimulator()(
left_value,
right_value
),
description = "Comparison with constant arguments."
)
return self, None, None
def computeExpressionOperationNot(self, not_node, constraint_collection):
if self.comparator in PythonOperators.comparison_inversions:
left, right = self.getOperands()
from .NodeMakingHelpers import makeComparisonNode
result = makeComparisonNode(
left = left,
right = right,
comparator = PythonOperators.comparison_inversions[
self.comparator
],
source_ref = self.source_ref
)
return result, "new_expression", """\
Replaced negated comparison with inverse comparision."""
return not_node, None, None
def mayProvideReference(self):
# Dedicated code returns "True" or "False" only, which requires no
# reference, except for rich comparisons, which do.
return self.comparator in PythonOperators.rich_comparison_functions
class ExpressionComparisonIsIsNotBase(ExpressionComparison):
def __init__(self, left, right, comparator, source_ref):
ExpressionComparison.__init__(
self,
left = left,
right = right,
comparator = comparator,
source_ref = source_ref
)
assert comparator in ( "Is", "IsNot" )
self.match_value = comparator == "Is"
def isExpressionComparison(self):
# Virtual method, pylint: disable=R0201
return True
def computeExpression(self, constraint_collection):
left, right = self.getOperands()
if constraint_collection.mustAlias( left, right ):
from .NodeMakingHelpers import (
makeConstantReplacementNode,
wrapExpressionWithSideEffects
)
result = makeConstantReplacementNode(
constant = self.match_value,
node = self
)
if left.mayHaveSideEffects() or right.mayHaveSideEffects():
result = wrapExpressionWithSideEffects(
side_effects = self.extractSideEffects(),
old_node = self,
new_node = result
)
return result, "new_constant", """\
Determined values to alias and therefore result of %s comparison.""" % (
self.comparator
)
if constraint_collection.mustNotAlias( left, right ):
from .NodeMakingHelpers import (
makeConstantReplacementNode,
wrapExpressionWithSideEffects
)
result = makeConstantReplacementNode(
constant = not self.match_value,
node = self
)
if left.mayHaveSideEffects() or right.mayHaveSideEffects():
result = wrapExpressionWithSideEffects(
side_effects = self.extractSideEffects(),
old_node = self,
new_node = result
)
return result, "new_constant", """\
Determined values to not alias and therefore result of %s comparison.""" % (
self.comparator
)
return ExpressionComparison.computeExpression(
self,
constraint_collection = constraint_collection
)
def extractSideEffects(self):
left, right = self.getOperands()
return left.extractSideEffects() + right.extractSideEffects()
def computeExpressionDrop(self, statement, constraint_collection):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = self.getOperands()
)
return result, "new_statements", """\
Removed %s comparison for unused result.""" % self.comparator
class ExpressionComparisonIs(ExpressionComparisonIsIsNotBase):
kind = "EXPRESSION_COMPARISON_IS"
def __init__(self, left, right, source_ref):
ExpressionComparisonIsIsNotBase.__init__(
self,
left = left,
right = right,
comparator = "Is",
source_ref = source_ref
)
class ExpressionComparisonIsNOT(ExpressionComparisonIsIsNotBase):
kind = "EXPRESSION_COMPARISON_IS_NOT"
def __init__(self, left, right, source_ref):
ExpressionComparisonIsIsNotBase.__init__(
self,
left = left,
right = right,
comparator = "IsNot",
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/nodes/BuiltinTypeNodes.py 0000644 0001750 0001750 00000016631 12265264105 021570 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Builtin type nodes tuple/list/float/int etc.
These are all very simple and have predictable properties, because we know their type and
that should allow some important optimizations.
"""
from .NodeBases import (
ExpressionSpecBasedComputationMixin,
ExpressionBuiltinSingleArgBase,
ChildrenHavingMixin,
NodeBase
)
from nuitka.optimizations import BuiltinOptimization
from nuitka.Utils import python_version
class ExpressionBuiltinTypeBase(ExpressionBuiltinSingleArgBase):
pass
class ExpressionBuiltinTuple(ExpressionBuiltinTypeBase):
kind = "EXPRESSION_BUILTIN_TUPLE"
builtin_spec = BuiltinOptimization.builtin_tuple_spec
class ExpressionBuiltinList(ExpressionBuiltinTypeBase):
kind = "EXPRESSION_BUILTIN_LIST"
builtin_spec = BuiltinOptimization.builtin_list_spec
class ExpressionBuiltinSet(ExpressionBuiltinTypeBase):
kind = "EXPRESSION_BUILTIN_SET"
builtin_spec = BuiltinOptimization.builtin_set_spec
class ExpressionBuiltinFloat(ExpressionBuiltinTypeBase):
kind = "EXPRESSION_BUILTIN_FLOAT"
builtin_spec = BuiltinOptimization.builtin_float_spec
class ExpressionBuiltinBool(ExpressionBuiltinTypeBase):
kind = "EXPRESSION_BUILTIN_BOOL"
builtin_spec = BuiltinOptimization.builtin_bool_spec
def mayProvideReference(self):
# Dedicated code returns "True" or "False" only, which requires no reference
return False
def computeExpression(self, constraint_collection):
value = self.getValue()
if value is not None:
truth_value = self.getValue().getTruthValue()
if truth_value is not None:
from .NodeMakingHelpers import wrapExpressionWithNodeSideEffects, makeConstantReplacementNode
result = wrapExpressionWithNodeSideEffects(
new_node = makeConstantReplacementNode(
constant = truth_value,
node = self,
),
old_node = self.getValue()
)
return result, "new_constant", "Predicted truth value of builtin bool argument"
return ExpressionBuiltinTypeBase.computeExpression( self, constraint_collection )
class ExpressionBuiltinIntLongBase( ChildrenHavingMixin, NodeBase,
ExpressionSpecBasedComputationMixin ):
named_children = ( "value", "base" )
try:
int( base = 2 )
except TypeError:
base_only_value = False
else:
base_only_value = True
def __init__(self, value, base, source_ref):
from .NodeMakingHelpers import makeConstantReplacementNode
NodeBase.__init__( self, source_ref = source_ref )
if value is None and self.base_only_value:
value = makeConstantReplacementNode(
constant = "0",
node = self
)
ChildrenHavingMixin.__init__(
self,
values = {
"value" : value,
"base" : base
}
)
getValue = ChildrenHavingMixin.childGetter( "value" )
getBase = ChildrenHavingMixin.childGetter( "base" )
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
value = self.getValue()
base = self.getBase()
given_values = []
if value is None:
if base is not None:
if not self.base_only_value:
from .NodeMakingHelpers import getComputationResult
return getComputationResult(
node = self,
computation = lambda : int( base = 2 ),
description = "int builtin call with only base argument"
)
given_values = ()
elif base is None:
given_values = ( value, )
else:
given_values = ( value, base )
return self.computeBuiltinSpec( given_values )
class ExpressionBuiltinInt(ExpressionBuiltinIntLongBase):
kind = "EXPRESSION_BUILTIN_INT"
builtin_spec = BuiltinOptimization.builtin_int_spec
class ExpressionBuiltinUnicodeBase( ChildrenHavingMixin, NodeBase,
ExpressionSpecBasedComputationMixin ):
named_children = ( "value", "encoding", "errors" )
def __init__(self, value, encoding, errors, source_ref):
NodeBase.__init__( self, source_ref = source_ref )
ChildrenHavingMixin.__init__(
self,
values = {
"value" : value,
"encoding" : encoding,
"errors" : errors
}
)
getValue = ChildrenHavingMixin.childGetter( "value" )
getEncoding = ChildrenHavingMixin.childGetter( "encoding" )
getErrors = ChildrenHavingMixin.childGetter( "errors" )
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
args = [
self.getValue(),
self.getEncoding(),
self.getErrors()
]
while args and args[-1] is None:
del args[-1]
return self.computeBuiltinSpec( tuple( args ) )
if python_version < 300:
class ExpressionBuiltinStr(ExpressionBuiltinTypeBase):
kind = "EXPRESSION_BUILTIN_STR"
builtin_spec = BuiltinOptimization.builtin_str_spec
def computeExpression(self, constraint_collection):
new_node, change_tags, change_desc = ExpressionBuiltinTypeBase.computeExpression(
self,
constraint_collection
)
if new_node is self:
str_value = self.getValue().getStrValue()
if str_value is not None:
from .NodeMakingHelpers import wrapExpressionWithNodeSideEffects
new_node = wrapExpressionWithNodeSideEffects(
new_node = str_value,
old_node = self.getValue()
)
change_tags = "new_expression"
change_desc = "Predicted str builtin result"
return new_node, change_tags, change_desc
class ExpressionBuiltinLong(ExpressionBuiltinIntLongBase):
kind = "EXPRESSION_BUILTIN_LONG"
builtin_spec = BuiltinOptimization.builtin_long_spec
class ExpressionBuiltinUnicode(ExpressionBuiltinUnicodeBase):
kind = "EXPRESSION_BUILTIN_UNICODE"
builtin_spec = BuiltinOptimization.builtin_unicode_spec
else:
class ExpressionBuiltinStr(ExpressionBuiltinUnicodeBase):
kind = "EXPRESSION_BUILTIN_STR"
builtin_spec = BuiltinOptimization.builtin_str_spec
Nuitka-0.5.0.1/nuitka/nodes/NodeBases.py 0000644 0001750 0001750 00000112767 12265264105 020201 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Node base classes.
These classes provide the generic base classes available for nodes.
"""
from nuitka.odict import OrderedDict
from nuitka.oset import OrderedSet
from nuitka import (
Variables,
Tracing,
TreeXML,
Options
)
from nuitka.__past__ import iterItems
lxml = TreeXML.lxml
class NodeCheckMetaClass(type):
kinds = set()
def __new__(mcs, name, bases, dictionary):
assert len( bases ) == len( set( bases ) )
# Uncomment this for debug view of class tags.
# print name, dictionary[ "tags" ]
return type.__new__( mcs, name, bases, dictionary )
def __init__(mcs, name, bases, dictionary):
if not name.endswith( "Base" ):
assert ( "kind" in dictionary ), name
kind = dictionary[ "kind" ]
assert type( kind ) is str, name
assert kind not in NodeCheckMetaClass.kinds, name
NodeCheckMetaClass.kinds.add( kind )
def convert(value):
if value in ( "AND", "OR", "NOT" ):
return value
else:
return value.title()
kind_to_name_part = "".join(
[ convert( x ) for x in kind.split( "_" ) ]
)
assert name.endswith( kind_to_name_part ), ( name, kind_to_name_part )
# Automatically add checker methods for everything to the common base class
checker_method = "is" + kind_to_name_part
def checkKind(self):
return self.kind == kind
if not hasattr( NodeBase, checker_method ):
setattr( NodeBase, checker_method, checkKind )
type.__init__( mcs, name, bases, dictionary )
# For every node type, there is a test, and then some more members,
# pylint: disable=R0904
# For Python2/3 compatible source, we create a base class that has the metaclass
# used and doesn't require making a choice.
NodeMetaClassBase = NodeCheckMetaClass( "NodeMetaClassBase", (object, ), {} )
class NodeBase(NodeMetaClassBase):
kind = None
# Must be overloaded by expressions.
value_friend_maker = None
def __init__(self, source_ref):
assert source_ref is not None
assert source_ref.line is not None
self.parent = None
self.source_ref = source_ref
def isNode(self):
# Virtual method, pylint: disable=R0201
return True
def __repr__(self):
# This is to avoid crashes, because of bugs in detail.
# pylint: disable=W0702
try:
detail = self.getDetail()
except:
detail = "detail raises exception"
if not detail:
return "" % self.getDescription()
else:
return "" % ( self.getDescription(), detail )
def getDescription(self):
""" Description of the node, intented for use in __repr__ and
graphical display.
"""
return "%s at %s" % ( self.kind, self.source_ref.getAsString() )
def getDetails(self):
""" Details of the node, intended for use in __repr__ and dumps.
"""
# Virtual method, pylint: disable=R0201
return {}
def getDetail(self):
""" Details of the node, intended for use in __repr__ and graphical
display.
"""
# Virtual method, pylint: disable=R0201
return str( self.getDetails() )[1:-1]
def getParent(self):
""" Parent of the node. Every node except modules have to have a parent.
"""
if self.parent is None and not self.isPythonModule():
assert False, ( self, self.source_ref )
return self.parent
def getParents(self):
""" Parents of the node. Up to module level.
"""
result = []
current = self
while True:
current = current.getParent()
result.append( current )
if current.isPythonModule() or current.isExpressionFunctionBody():
break
assert None not in result, self
result.reverse()
return result
def getParentFunction(self):
""" Return the parent that is a function.
"""
parent = self.getParent()
while parent is not None and not parent.isExpressionFunctionBody():
parent = parent.getParent()
return parent
def getParentModule(self):
""" Return the parent that is module.
"""
parent = self
while not parent.isPythonModule():
if hasattr( parent, "provider" ):
# After we checked, we can use it, will be much faster,
# pylint: disable=E1101
parent = parent.provider
else:
parent = parent.getParent()
return parent
def isParentVariableProvider(self):
# Check if it's a closure giver, in which cases it can provide variables,
# pylint: disable=E1101
return isinstance( self, ClosureGiverNodeBase )
def getParentVariableProvider(self):
parent = self.getParent()
while not parent.isParentVariableProvider():
parent = parent.getParent()
return parent
def getParentStatementsFrame(self):
current = self.getParent()
while True:
if current.isStatementsFrame():
return current
if current.isParentVariableProvider():
return None
current = current.getParent()
def getSourceReference(self):
return self.source_ref
def asXml(self):
result = lxml.etree.Element(
"node",
kind = self.__class__.__name__,
line = "%s" % self.getSourceReference().getLineNumber()
)
for key, value in iterItems( self.getDetails() ):
value = str( value )
if value.startswith( "<" ) and value.endswith( ">" ):
value = value[1:-1]
result.set( key, str( value ) )
for name, children in self.getVisitableNodesNamed():
if type( children ) not in ( list, tuple ):
children = ( children, )
role = lxml.etree.Element(
"role",
name = name
)
result.append( role )
for child in children:
if child is not None:
role.append(
child.asXml()
)
return result
def dump(self, level = 0):
Tracing.printIndented( level, self )
Tracing.printSeparator( level )
for visitable in self.getVisitableNodes():
visitable.dump( level + 1 )
Tracing.printSeparator( level )
def isExpression(self):
return self.kind.startswith( "EXPRESSION_" )
def isStatement(self):
return self.kind.startswith( "STATEMENT_" )
def isExpressionBuiltin(self):
return self.kind.startswith( "EXPRESSION_BUILTIN_" )
def isOperation(self):
return self.kind.startswith( "EXPRESSION_OPERATION_" )
def isExpressionOperationBool2(self):
return self.kind.startswith( "EXPRESSION_BOOL_" )
def isExpressionMakeSequence(self):
# Virtual method, pylint: disable=R0201,W0613
return False
def isIteratorMaking(self):
# Virtual method, pylint: disable=R0201,W0613
return False
def isNumberConstant(self):
# Virtual method, pylint: disable=R0201,W0613
return False
def isExpressionCall(self):
# Virtual method, pylint: disable=R0201,W0613
return False
def visit(self, context, visitor):
visitor( self )
for visitable in self.getVisitableNodes():
visitable.visit( context, visitor )
def getVisitableNodes(self):
# Virtual method, pylint: disable=R0201,W0613
return ()
def getVisitableNodesNamed(self):
# Virtual method, pylint: disable=R0201
return ()
def replaceWith(self, new_node):
self.parent.replaceChild(
old_node = self,
new_node = new_node
)
def discard(self):
""" The node has become unused. """
# print "Discarding", self
if Options.isExperimental():
self.parent = None
def getName(self):
# Virtual method, pylint: disable=R0201,W0613
return None
def mayHaveSideEffects(self):
""" Unless we are told otherwise, everything may have a side effect. """
# Virtual method, pylint: disable=R0201,W0613
return True
def isOrderRelevant(self):
return self.mayHaveSideEffects()
def mayHaveSideEffectsBool(self):
""" Unless we are told otherwise, everything may have a side effect. """
# Virtual method, pylint: disable=R0201,W0613
return True
def extractSideEffects(self):
""" Unless defined otherwise, the expression is the side effect. """
# Virtual method, pylint: disable=R0201,W0613
return ( self, )
def mayRaiseException(self, exception_type):
""" Unless we are told otherwise, everything may raise everything. """
# Virtual method, pylint: disable=R0201,W0613
return True
def willRaiseException(self, exception_type):
""" Unless we are told otherwise, nothing may raise anything. """
# Virtual method, pylint: disable=R0201,W0613
return False
def needsLineNumber(self):
return self.mayRaiseException( BaseException )
def isIndexable(self):
""" Unless we are told otherwise, it's not indexable. """
# Virtual method, pylint: disable=R0201,W0613
return False
def isStatementAborting(self):
""" Is the node aborting, control flow doesn't continue after this node. """
# Virtual method, pylint: disable=R0201
assert self.isStatement(), self.kind
return False
def needsLocalsDict(self):
""" Node requires a locals dictionary by provider. """
# Virtual method, pylint: disable=R0201,W0613
return False
def getIntegerValue(self):
""" Node as integer value, if possible."""
# Virtual method, pylint: disable=R0201,W0613
return None
class CodeNodeBase(NodeBase):
def __init__(self, name, code_prefix, source_ref):
assert name is not None
assert " " not in name, name
assert "<" not in name, name
NodeBase.__init__( self, source_ref = source_ref )
self.name = name
self.code_prefix = code_prefix
# The code name is determined on demand only.
self.code_name = None
# The "UID" values of children kinds are kept here.
self.uids = {}
def getName(self):
return self.name
def getFullName(self):
result = self.getName()
current = self
while True:
current = current.getParent()
if current is None:
break
name = current.getName()
if name is not None:
result = "%s__%s" % ( name, result )
assert "<" not in result, result
return result
def getCodeName(self):
if self.code_name is None:
provider = self.getParentVariableProvider()
parent_name = provider.getCodeName()
uid = "_%d" % provider.getChildUID( self )
assert isinstance( self, CodeNodeBase )
if self.name:
name = uid + "_" + self.name
else:
name = uid
self.code_name = "%s%s_of_%s" % ( self.code_prefix, name, parent_name )
return self.code_name
def getChildUID(self, node):
if node.kind not in self.uids:
self.uids[ node.kind ] = 0
self.uids[ node.kind ] += 1
return self.uids[ node.kind ]
class ChildrenHavingMixin:
named_children = ()
checkers = {}
def __init__(self, values):
assert len(self.named_children)
assert type(self.named_children) is tuple
for key in values.keys():
assert key in self.named_children, key
# Default non-given values to None. TODO: Good idea? Better check for
# completeness instead.
self.child_values = dict.fromkeys(self.named_children)
self.child_values.update(values)
for key, value in self.child_values.items():
if key in self.checkers:
value = self.child_values[key] = self.checkers[key](value)
assert type(value) is not list, key
if type( value ) is tuple:
assert None not in value, key
for val in value:
val.parent = self
elif value is not None:
value.parent = self
def setChild(self, name, value):
""" Set a child value.
Do not overload, provider self.checkers instead.
"""
# Only accept legal child names
assert name in self.child_values, name
# Lists as inputs are OK, but turn them into tuples.
if type(value) is list:
value = tuple(value)
if name in self.checkers:
value = self.checkers[name](value)
# Reparent value to us.
if type(value) is tuple:
for val in value:
val.parent = self
elif value is not None:
value.parent = self
# Determine old value, and inform it about loosing its parent.
old_value = self.child_values[name]
assert old_value is not value, value
self.child_values[name] = value
# TODO: Enable this
if old_value is not None:
if type(old_value) is tuple:
for val in old_value:
if val not in value:
val.discard()
else:
old_value.discard()
def getChild(self, name):
# Only accept legal child names
assert name in self.child_values, name
return self.child_values[name]
def hasChild(self, name):
return name in self.child_values
@staticmethod
def childGetter(name):
def getter(self):
return self.getChild(name)
return getter
@staticmethod
def childSetter(name):
def setter(self, value):
self.setChild(name, value)
return setter
def getVisitableNodes(self):
result = []
for name in self.named_children:
value = self.child_values[ name ]
if value is None:
pass
elif type( value ) is tuple:
result += list( value )
elif isinstance( value, NodeBase ):
result.append( value )
else:
raise AssertionError(
self,
"has illegal child", name, value, value.__class__
)
return tuple( result )
def getVisitableNodesNamed(self):
result = []
for name in self.named_children:
value = self.child_values[ name ]
result.append( ( name, value ) )
return result
def replaceChild(self, old_node, new_node):
if new_node is not None and not isinstance(new_node, NodeBase):
raise AssertionError(
"Cannot replace with", new_node, "old", old_node, "in", self
)
# Find the replaced node, as an added difficulty, what might be
# happening, is that the old node is an element of a tuple, in which we
# may also remove that element, by setting it to None.
for key, value in self.child_values.items():
if value is None:
pass
elif type(value) is tuple:
if old_node in value:
if new_node is not None:
self.setChild(
key,
tuple(
(val if val is not old_node else new_node)
for val in
value
)
)
else:
self.setChild(
key,
tuple(
val
for val in
value
if val is not old_node
)
)
break
elif isinstance(value, NodeBase):
if old_node is value:
self.setChild(key, new_node)
break
else:
assert False, ( key, value, value.__class__ )
else:
raise AssertionError(
"Didn't find child",
old_node,
"in",
self
)
return key
def makeCloneAt(self, source_ref):
values = {}
for key, value in self.child_values.items():
assert type( value ) is not list, key
if value is None:
values[ key ] = None
elif type( value ) is tuple:
values[ key ] = tuple(
v.makeCloneAt(
source_ref = v.getSourceReference()
)
for v in
value
)
else:
values[ key ] = value.makeCloneAt(
value.getSourceReference()
)
values.update( self.getDetails() )
try:
return self.__class__(
source_ref = source_ref,
**values
)
except TypeError as e:
print( "Problem cloning", self.__class__ )
raise
class ClosureGiverNodeBase(CodeNodeBase):
""" Mixin for nodes that provide variables for closure takers. """
def __init__(self, name, code_prefix, source_ref):
CodeNodeBase.__init__(
self,
name = name,
code_prefix = code_prefix,
source_ref = source_ref
)
self.providing = OrderedDict()
self.keeper_variables = OrderedSet()
self.temp_variables = OrderedDict()
self.temp_scopes = OrderedDict()
def hasProvidedVariable(self, variable_name):
return variable_name in self.providing
def getProvidedVariable(self, variable_name):
if variable_name not in self.providing:
self.providing[ variable_name ] = self.createProvidedVariable(
variable_name = variable_name
)
return self.providing[ variable_name ]
def createProvidedVariable(self, variable_name):
# Virtual method, pylint: disable=R0201,W0613
assert type( variable_name ) is str
return None
def registerProvidedVariables(self, variables):
for variable in variables:
self.registerProvidedVariable( variable )
def registerProvidedVariable(self, variable):
assert variable is not None
self.providing[ variable.getName() ] = variable
def getProvidedVariables(self):
return self.providing.values()
def allocateTempKeeperVariable(self):
name = "keeper_%d" % len( self.keeper_variables )
result = Variables.TempKeeperVariable(
owner = self,
variable_name = name
)
self.keeper_variables.add( result )
return result
def getTempKeeperVariables(self):
return self.keeper_variables
def removeTempKeeperVariable(self, variable):
self.keeper_variables.discard( variable )
def allocateTempScope(self, name, allow_closure = False):
self.temp_scopes[ name ] = self.temp_scopes.get( name, 0 ) + 1
# TODO: Instead of using overly long code name, could just visit parents
# and make sure to allocate the scope at the top.
if allow_closure:
return "%s_%s_%d" % (
self.getCodeName(),
name,
self.temp_scopes[ name ]
)
else:
return "%s_%d" % ( name, self.temp_scopes[ name ] )
def allocateTempVariable(self, temp_scope, name):
if temp_scope is not None:
full_name = "%s__%s" % ( temp_scope, name )
else:
full_name = name
del name
assert full_name not in self.temp_variables, full_name
result = Variables.TempVariable(
owner = self,
variable_name = full_name
)
self.temp_variables[ full_name ] = result
return result
def getTempVariable(self, temp_scope, name):
if temp_scope is not None:
full_name = "%s__%s" % ( temp_scope, name )
else:
full_name = name
return self.temp_variables[ full_name ]
def getTempVariables(self):
return tuple( self.temp_variables.values() )
def removeTempVariable(self, variable):
del self.temp_variables[ variable.getName() ]
class ParameterHavingNodeBase(ClosureGiverNodeBase):
def __init__(self, name, code_prefix, parameters, source_ref):
ClosureGiverNodeBase.__init__(
self,
name = name,
code_prefix = code_prefix,
source_ref = source_ref
)
self.parameters = parameters
self.parameters.setOwner( self )
self.registerProvidedVariables(
variables = self.parameters.getVariables()
)
def getParameters(self):
return self.parameters
class ClosureTakerMixin:
""" Mixin for nodes that accept variables from closure givers. """
def __init__(self, provider, early_closure):
assert provider.isParentVariableProvider(), provider
self.provider = provider
self.early_closure = early_closure
self.taken = set()
self.temp_variables = set()
def getParentVariableProvider(self):
return self.provider
def getClosureVariable(self, variable_name):
result = self.provider.getVariableForClosure(
variable_name = variable_name
)
assert result is not None, variable_name
# There is no maybe with closures. It means, it is closure variable in
# this case.
if result.isMaybeLocalVariable():
# This mixin is used with nodes only, but doesn't want to inherit
# from it, pylint: disable=E1101
result = self.getParentModule().getVariableForClosure(
variable_name = variable_name
)
return self.addClosureVariable( result )
def addClosureVariable(self, variable):
variable = variable.makeReference( self )
self.taken.add( variable )
return variable
def getClosureVariables(self):
return tuple(
sorted(
[
take
for take in
self.taken
if take.isClosureReference()
],
key = lambda x : x.getName()
)
)
def hasTakenVariable(self, variable_name):
for variable in self.taken:
if variable.getName() == variable_name:
return True
else:
return False
def getTakenVariable(self, variable_name):
for variable in self.taken:
if variable.getName() == variable_name:
return variable
else:
return None
def isEarlyClosure(self):
""" Early closure taking means immediate binding of references.
Normally it's good to lookup name references immediately, but not for
functions. In case of a function body it is not allowed to do that,
because a later assignment needs to be queried first. Nodes need to
indicate via this if they would like to resolve references at the same
time as assignments.
"""
return self.early_closure
class ExpressionMixin:
def isCompileTimeConstant(self):
""" Has a value that we can use at compile time.
Yes or no. If it has such a value, simulations can be applied at
compile time and e.g. operations or conditions, or even calls may
be executed against it.
"""
# Virtual method, pylint: disable=R0201
return False
def getCompileTimeConstant(self):
assert self.isCompileTimeConstant(), self
assert False
def getTruthValue(self):
""" Return known truth value. The "None" value indicates unknown. """
if self.isCompileTimeConstant():
return bool( self.getCompileTimeConstant() )
else:
return None
def isKnownToBeIterable(self, count):
""" Can be iterated at all (count is None) or exactly count times.
Yes or no. If it can be iterated a known number of times, it may
be asked to unpack itself.
"""
# Virtual method, pylint: disable=R0201,W0613
return False
def isKnownToBeIterableAtMin(self, count):
# Virtual method, pylint: disable=R0201,W0613
return False
def isKnownToBeIterableAtMax(self, count):
# Virtual method, pylint: disable=R0201,W0613
return False
def mayProvideReference(self):
""" May at run time produce a reference.
This then would have to be consumed or released in a reliable way.
"""
# Virtual method, pylint: disable=R0201
return True
def getIterationLength(self):
""" Value that "len" or "PyObject_Size" would give, if known.
Otherwise it is "None" to indicate unknown.
"""
# Virtual method, pylint: disable=R0201
return None
def getStringValue(self):
""" Node as integer value, if possible."""
# Virtual method, pylint: disable=R0201,W0613
return None
def getStrValue(self):
""" Value that "str" or "PyObject_Str" would give, if known.
Otherwise it is "None" to indicate unknown.
"""
string_value = self.getStringValue()
if string_value is not None:
from .NodeMakingHelpers import makeConstantReplacementNode
# TODO: Side effects should be considered, getStringValue may be
# omitting effects.
return makeConstantReplacementNode(
node = self,
constant = string_value
)
return None
def onRelease(self, constraint_collection):
# print "onRelease", self
pass
def computeExpressionRaw(self, constraint_collection):
""" Compute an expression.
Default behavior is to just visit the child expressions first, and
then the node "computeExpression". For a few cases this needs to
be overloaded, e.g. conditional expressions.
"""
# First apply the sub-expressions, as they are evaluated before.
sub_expressions = self.getVisitableNodes()
for sub_expression in sub_expressions:
constraint_collection.onExpression(
expression = sub_expression
)
# Then ask ourselves to work on it.
return self.computeExpression(
constraint_collection = constraint_collection
)
def computeExpressionAttribute(self, lookup_node, attribute_name,
constraint_collection):
# By default, an attribute lookup may change everything about the lookup
# source. Virtual method, pylint: disable=R0201,W0613
constraint_collection.removeKnowledge( lookup_node )
return lookup_node, None, None
def computeExpressionSubscript(self, lookup_node, subscript,
constraint_collection):
# By default, an subscript may change everything about the lookup
# source.
constraint_collection.removeKnowledge( lookup_node )
return lookup_node, None, None
def computeExpressionSlice(self, lookup_node, lower, upper,
constraint_collection):
# By default, a slicing may change everything about the lookup source.
constraint_collection.removeKnowledge( lookup_node )
return lookup_node, None, None
def computeExpressionCall(self, call_node, constraint_collection):
call_node.getCalled().onContentEscapes(constraint_collection)
return call_node, None, None
def computeExpressionIter1(self, iter_node, constraint_collection):
iter_node.getValue().onContentEscapes(constraint_collection)
return iter_node, None, None
def computeExpressionOperationNot(self, not_node, constraint_collection):
constraint_collection.removeKnowledge(not_node)
return not_node, None, None
def computeExpressionDrop(self, statement, constraint_collection):
if not self.mayHaveSideEffects():
return None, "new_statements", "Removed statement without effect."
return statement, None, None
def onContentEscapes(self, constraint_collection):
pass
class CompileTimeConstantExpressionMixin(ExpressionMixin):
def __init__(self):
self.computed_attribute = False
def isCompileTimeConstant(self):
""" Has a value that we can use at compile time.
Yes or no. If it has such a value, simulations can be applied at
compile time and e.g. operations or conditions, or even calls may
be executed against it.
"""
# Virtual method, pylint: disable=R0201
return True
def mayHaveSideEffects(self):
return False
def mayHaveSideEffectsBool(self):
return False
def computeExpressionOperationNot(self, not_node, constraint_collection):
from .NodeMakingHelpers import getComputationResult
return getComputationResult(
node = not_node,
computation = lambda : not self.getCompileTimeConstant(),
description = """\
Compile time constant negation truth value precomputed."""
)
def computeExpressionAttribute(self, lookup_node, attribute_name, constraint_collection):
if self.computed_attribute:
return lookup_node, None, None
value = self.getCompileTimeConstant()
from .NodeMakingHelpers import getComputationResult, isCompileTimeConstantValue
if not hasattr( value, attribute_name ) or isCompileTimeConstantValue( getattr( value, attribute_name ) ):
return getComputationResult(
node = lookup_node,
computation = lambda : getattr( value, attribute_name ),
description = "Attribute lookup to %s precomputed." % (
attribute_name
)
)
self.computed_attribute = True
return lookup_node, None, None
def computeExpressionSubscript(self, lookup_node, subscript, constraint_collection):
from .NodeMakingHelpers import getComputationResult
if subscript.isCompileTimeConstant():
return getComputationResult(
node = lookup_node,
computation = lambda : self.getCompileTimeConstant()[ subscript.getCompileTimeConstant() ],
description = "Subscript of constant with constant value."
)
return lookup_node, None, None
def computeExpressionSlice(self, lookup_node, lower, upper, constraint_collection):
from .NodeMakingHelpers import getComputationResult
# TODO: Could be happy with predictable index values and not require
# constants.
if lower is not None:
if upper is not None:
if lower.isCompileTimeConstant() and upper.isCompileTimeConstant():
return getComputationResult(
node = lookup_node,
computation = lambda : self.getCompileTimeConstant()[
lower.getCompileTimeConstant() : upper.getCompileTimeConstant()
],
description = """\
Slicing of constant with constant indexes."""
)
else:
if lower.isCompileTimeConstant():
return getComputationResult(
node = lookup_node,
computation = lambda : self.getCompileTimeConstant()[
lower.getCompileTimeConstant() :
],
description = """\
Slicing of constant with constant lower index only."""
)
else:
if upper is not None:
if upper.isCompileTimeConstant():
return getComputationResult(
node = lookup_node,
computation = lambda : self.getCompileTimeConstant()[
: upper.getCompileTimeConstant()
],
description = """\
Slicing of constant with constant upper index only."""
)
else:
return getComputationResult(
node = lookup_node,
computation = lambda : self.getCompileTimeConstant()[ : ],
description = "Slicing of constant with no indexes."
)
return lookup_node, None, None
class ExpressionSpecBasedComputationMixin(ExpressionMixin):
builtin_spec = None
def computeBuiltinSpec(self, given_values):
assert self.builtin_spec is not None, self
for value in given_values:
if value is not None and not value.isCompileTimeConstant():
return self, None, None
if not self.builtin_spec.isCompileTimeComputable(given_values):
return self, None, None
from .NodeMakingHelpers import getComputationResult
return getComputationResult(
node = self,
computation = lambda : self.builtin_spec.simulateCall(given_values),
description = "Builtin call to '%s' precomputed." % (
self.builtin_spec.getName()
)
)
class ExpressionChildrenHavingBase(ChildrenHavingMixin, NodeBase,
ExpressionMixin):
def __init__(self, values, source_ref):
NodeBase.__init__(
self,
source_ref = source_ref
)
ChildrenHavingMixin.__init__(
self,
values = values
)
class StatementChildrenHavingBase(ChildrenHavingMixin, NodeBase):
def __init__(self, values, source_ref):
NodeBase.__init__( self, source_ref = source_ref )
ChildrenHavingMixin.__init__(
self,
values = values
)
class ExpressionBuiltinNoArgBase(NodeBase, ExpressionMixin):
def __init__(self, builtin_function, source_ref):
NodeBase.__init__(
self,
source_ref = source_ref
)
self.builtin_function = builtin_function
def computeExpression(self, constraint_collection):
from .NodeMakingHelpers import getComputationResult
# The lamba is there for make sure that no argument parsing will reach
# the builtin function at all, pylint: disable=W0108
return getComputationResult(
node = self,
computation = lambda : self.builtin_function(),
description = "No arg %s builtin" % self.builtin_function.__name__
)
class ExpressionBuiltinSingleArgBase(ExpressionChildrenHavingBase,
ExpressionSpecBasedComputationMixin ):
named_children = (
"value",
)
def __init__(self, value, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"value" : value,
},
source_ref = source_ref
)
getValue = ExpressionChildrenHavingBase.childGetter( "value" )
def computeExpression(self, constraint_collection):
value = self.getValue()
assert self.builtin_spec is not None, self
if value is None:
return self.computeBuiltinSpec(
given_values = ()
)
else:
if value.willRaiseException(BaseException):
return value, "new_raise", """\
Builtin call raises exception while building argument."""
return self.computeBuiltinSpec(
given_values = (value,)
)
class SideEffectsFromChildrenMixin:
def mayHaveSideEffects(self):
for child in self.getVisitableNodes():
if child.mayHaveSideEffects():
return True
else:
return False
def extractSideEffects(self):
# No side effects at all but from the children.
result = []
for child in self.getVisitableNodes():
result.extend(
child.extractSideEffects()
)
return tuple(result)
Nuitka-0.5.0.1/nuitka/nodes/ClassNodes.py 0000644 0001750 0001750 00000005002 12265264105 020353 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Nodes for classes and their creations.
The classes are are at the core of the language and have their complexities.
"""
from .NodeBases import ExpressionChildrenHavingBase
class ExpressionSelectMetaclass(ExpressionChildrenHavingBase):
kind = "EXPRESSION_SELECT_METACLASS"
named_children = ( "metaclass", "bases", )
def __init__(self, metaclass, bases, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"metaclass" : metaclass,
"bases" : bases
},
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
# TODO: Meta class selection is very computable, and should be done.
return self, None, None
getMetaclass = ExpressionChildrenHavingBase.childGetter("metaclass")
getBases = ExpressionChildrenHavingBase.childGetter( "bases" )
class ExpressionBuiltinType3(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_TYPE3"
named_children = ( "type_name", "bases", "dict" )
def __init__(self, type_name, bases, type_dict, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"type_name" : type_name,
"bases" : bases,
"dict" : type_dict
},
source_ref = source_ref
)
getTypeName = ExpressionChildrenHavingBase.childGetter( "type_name" )
getBases = ExpressionChildrenHavingBase.childGetter( "bases" )
getDict = ExpressionChildrenHavingBase.childGetter( "dict" )
def computeExpression(self, constraint_collection):
# TODO: Should be compile time computable if bases and dict are.
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/SideEffectNodes.py 0000644 0001750 0001750 00000007172 12265264105 021321 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Node that models side effects.
Sometimes, the effect of an expression needs to be had, but the value itself
does not matter at all.
"""
from .NodeBases import ExpressionChildrenHavingBase
def checkSideEffects(value):
real_value = []
for child in value:
if child.isExpressionSideEffects():
real_value.extend(child.getSideEffects())
real_value.append(child.getExpression())
else:
assert child.isExpression()
real_value.append( child )
return tuple(real_value)
class ExpressionSideEffects(ExpressionChildrenHavingBase):
kind = "EXPRESSION_SIDE_EFFECTS"
named_children = ( "side_effects", "expression" )
checkers = {
"side_effects" : checkSideEffects
}
def __init__(self, side_effects, expression, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"side_effects" : tuple( side_effects ),
"expression" : expression
},
source_ref = source_ref
)
getSideEffects = ExpressionChildrenHavingBase.childGetter( "side_effects" )
setSideEffects = ExpressionChildrenHavingBase.childSetter("side_effects")
getExpression = ExpressionChildrenHavingBase.childGetter( "expression" )
def computeExpression(self, constraint_collection):
side_effects = self.getSideEffects()
new_side_effects = []
for side_effect in side_effects:
if side_effect.mayHaveSideEffects():
new_side_effects.append( side_effect )
expression = self.getExpression()
if expression.isExpressionSideEffects():
new_side_effects.extend( expression.getSideEffects() )
expression.setSideEffects( new_side_effects )
return expression, "new_expression", "Remove nested side effects"
if new_side_effects != side_effects:
self.setSideEffects( new_side_effects )
if not new_side_effects:
return expression, "new_expression", "Removed empty side effects."
return self, None, None
def willRaiseException(self, exception_type):
for child in self.getVisitableNodes():
if child.willRaiseException(exception_type):
return True
else:
return False
def getTruthValue(self):
return self.getExpression().getTruthValue()
def computeExpressionDrop(self, statement, constraint_collection):
# Side effects can become statements.
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
expressions = self.getSideEffects() + (self.getExpression(),)
result = makeStatementOnlyNodesFromExpressions(
expressions = expressions
)
return result, "new_statements", """\
Turned side effects of expression only statement into statements."""
Nuitka-0.5.0.1/nuitka/nodes/CallNodes.py 0000644 0001750 0001750 00000011474 12265264105 020173 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Call node
Function calls and generally calling expressions are the same thing. This is
very important, because it allows to predict most things, and avoid expensive
operations like parameter parsing at run time.
There will be a method "computeExpressionCall" to aid predicting them.
"""
from .NodeBases import ExpressionChildrenHavingBase
from .ConstantRefNodes import ExpressionConstantRef
class ExpressionCall(ExpressionChildrenHavingBase):
kind = "EXPRESSION_CALL"
named_children = ( "called", "args", "kw" )
def __init__(self, called, args, kw, source_ref):
assert called.isExpression()
assert args.isExpression()
assert kw.isExpression()
ExpressionChildrenHavingBase.__init__(
self,
values = {
"called" : called,
"args" : args,
"kw" : kw,
},
source_ref = source_ref
)
getCalled = ExpressionChildrenHavingBase.childGetter( "called" )
getCallArgs = ExpressionChildrenHavingBase.childGetter( "args" )
getCallKw = ExpressionChildrenHavingBase.childGetter( "kw" )
def isExpressionCall(self):
return True
def computeExpression(self, constraint_collection):
called = self.getCalled()
if called.willRaiseException(BaseException):
return called, "new_raise", "Called expression raises exception"
args = self.getCallArgs()
from .NodeMakingHelpers import wrapExpressionWithSideEffects
if args.willRaiseException(BaseException):
result = wrapExpressionWithSideEffects(
side_effects = (called,),
old_node = self,
new_node = args
)
return result, "new_raise", "Call arguments raise exception"
kw = self.getCallKw()
if kw.willRaiseException(BaseException):
result = wrapExpressionWithSideEffects(
side_effects = (called, args),
old_node = self,
new_node = kw
)
return result, "new_raise", "Call keyword arguments raise exception"
return called.computeExpressionCall(
call_node = self,
constraint_collection = constraint_collection
)
def extractPreCallSideEffects(self):
args = self.getCallArgs()
kw = self.getCallKw()
return args.extractSideEffects() + kw.extractSideEffects()
class ExpressionCallNoKeywords(ExpressionCall):
kind = "EXPRESSION_CALL_NO_KEYWORDS"
named_children = ( "called", "args", "kw" )
def __init__(self, called, args, source_ref):
assert called.isExpression()
ExpressionCall.__init__(
self,
called = called,
args = args,
kw = ExpressionConstantRef(
constant = {},
source_ref = source_ref,
),
source_ref = source_ref
)
class ExpressionCallKeywordsOnly(ExpressionCall):
kind = "EXPRESSION_CALL_KEYWORDS_ONLY"
named_children = ( "called", "args", "kw" )
def __init__(self, called, kw, source_ref):
assert called.isExpression()
ExpressionCall.__init__(
self,
called = called,
args = ExpressionConstantRef(
constant = (),
source_ref = source_ref,
),
kw = kw,
source_ref = source_ref
)
class ExpressionCallEmpty(ExpressionCall):
kind = "EXPRESSION_CALL_EMPTY"
named_children = ( "called", "args", "kw" )
def __init__(self, called, source_ref):
assert called.isExpression()
ExpressionCall.__init__(
self,
called = called,
args = ExpressionConstantRef(
constant = (),
source_ref = source_ref
),
kw = ExpressionConstantRef(
constant = {},
source_ref = source_ref,
),
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/nodes/FunctionNodes.py 0000644 0001750 0001750 00000047750 12265264105 021113 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Nodes for functions and their creations.
Lambdas are functions too. The functions are at the core of the language and
have their complexities.
"""
from .NodeBases import (
ExpressionChildrenHavingBase,
SideEffectsFromChildrenMixin,
ParameterHavingNodeBase,
ExpressionMixin,
ChildrenHavingMixin,
ClosureTakerMixin,
NodeBase
)
from .IndicatorMixins import (
MarkUnoptimizedFunctionIndicator,
MarkContainsTryExceptIndicator,
MarkLocalsDictIndicator,
MarkGeneratorIndicator
)
from .ParameterSpecs import TooManyArguments, matchCall
from nuitka import Variables, Utils
from nuitka.__past__ import iterItems
class ExpressionFunctionBody( ClosureTakerMixin, ChildrenHavingMixin,
ParameterHavingNodeBase, ExpressionMixin,
MarkContainsTryExceptIndicator,
MarkGeneratorIndicator,
MarkLocalsDictIndicator,
MarkUnoptimizedFunctionIndicator ):
# We really want these many ancestors, as per design, we add properties via
# base class mixins a lot, pylint: disable=R0901
kind = "EXPRESSION_FUNCTION_BODY"
named_children = ( "body", )
def __init__( self, provider, name, doc, parameters, source_ref,
is_class = False ):
# Register ourselves immediately with the module.
provider.getParentModule().addFunction( self )
if is_class:
code_prefix = "class"
else:
code_prefix = "function"
if name == "":
name = "lambda"
code_prefix = name
self.is_lambda = True
else:
self.is_lambda = False
if name == "":
code_prefix = "listcontr"
name = ""
self.local_locals = Utils.python_version >= 300
else:
self.local_locals = True
if name == "":
code_prefix = "setcontr"
name = ""
if name == "":
code_prefix = "dictcontr"
name = ""
if name == "":
code_prefix = "genexpr"
name = ""
self.is_genexpr = True
else:
self.is_genexpr = False
self.non_local_declarations = []
ClosureTakerMixin.__init__(
self,
provider = provider,
early_closure = is_class
)
ParameterHavingNodeBase.__init__(
self,
name = name,
code_prefix = code_prefix,
parameters = parameters,
source_ref = source_ref
)
ChildrenHavingMixin.__init__(
self,
values = {}
)
MarkContainsTryExceptIndicator.__init__( self )
MarkGeneratorIndicator.__init__( self )
MarkLocalsDictIndicator.__init__( self )
MarkUnoptimizedFunctionIndicator.__init__( self )
self.is_class = is_class
self.doc = doc
# Indicator, if this is a function that uses "super", because if it
# does, it would like to get the final "__class__" attached.
self.has_super = False
# Indicator if the return value exception might be required.
self.return_exception = False
# Indicator if the function needs to be created as a function object.
self.needs_creation = False
# Indicator if the function is called directly.
self.needs_direct = False
# Indicator if the function is used outside of where it's defined.
self.cross_module_use = False
def getDetails(self):
return {
"name" : self.getFunctionName(),
"ref_name" : self.getCodeName(),
"parameters" : self.getParameters(),
"provider" : self.provider.getCodeName(),
"doc" : self.doc
}
def getDetail(self):
return "named %s with %s" % ( self.name, self.parameters )
def getParent(self):
assert False
def isClassDictCreation(self):
return self.is_class
def getContainingClassDictCreation(self):
current = self
while not current.isPythonModule():
if current.isClassDictCreation():
return current
current = current.getParentVariableProvider()
return None
def getFunctionName(self):
if self.is_lambda:
return ""
elif self.is_genexpr:
return ""
else:
return self.name
def getFunctionQualname(self):
""" Function __qualname__ new in CPython3.3
Should contain some kind of full name descriptions for the closure to
recognize and will be used for outputs.
"""
function_name = self.getFunctionName()
provider = self.getParentVariableProvider()
if provider.isPythonModule():
return function_name
elif provider.isClassDictCreation():
return provider.getFunctionQualname() + "." + function_name
else:
return provider.getFunctionQualname() + ".." + function_name
def getDoc(self):
return self.doc
def getLocalVariableNames(self):
return Variables.getNames( self.getLocalVariables() )
def getLocalVariables(self):
return [
variable for
variable in
self.providing.values()
if variable.isLocalVariable()
]
def getUserLocalVariables(self):
return tuple(
variable for
variable in
self.providing.values()
if variable.isLocalVariable() and not variable.isParameterVariable()
)
def getVariables(self):
return self.providing.values()
def removeVariable(self, variable):
assert variable.getOwner() is self
assert variable in self.providing.values(), ( self.providing, variable )
assert not variable.getReferences()
del self.providing[ variable.getName() ]
assert not variable.isParameterVariable()
self.taken.remove( variable )
def getVariableForAssignment(self, variable_name):
# print ( "ASS func", self, variable_name )
if self.hasTakenVariable( variable_name ):
result = self.getTakenVariable( variable_name )
if self.isClassDictCreation():
if result.isModuleVariableReference() and \
not result.isFromGlobalStatement():
result = self.getProvidedVariable( variable_name )
if result.isModuleVariableReference():
del self.providing[ variable_name ]
result = self.getProvidedVariable( variable_name )
else:
result = self.getProvidedVariable( variable_name )
return result
def getVariableForReference(self, variable_name):
# print ( "REF func", self, variable_name )
if self.hasProvidedVariable( variable_name ):
result = self.getProvidedVariable( variable_name )
else:
# For exec containing/star import containing, get a closure variable
# and if it is a module variable, only then make it a maybe local
# variable.
result = self.getClosureVariable(
variable_name = variable_name
)
if self.isUnoptimized() and result.isModuleVariable():
result = Variables.MaybeLocalVariable(
owner = self,
variable_name = variable_name
)
# Remember that we need that closure for something.
self.registerProvidedVariable( result )
return result
def getVariableForClosure(self, variable_name):
# print( "getVariableForClosure", self, variable_name )
# The class bodies provide no closure, except under CPython3.x, there
# they provide "__class__" and nothing else.
if self.isClassDictCreation():
if variable_name == "__class__":
if Utils.python_version < 300:
return self.provider.getVariableForReference(
variable_name
)
elif Utils.python_version >= 340:
result = self.getTempVariable(
temp_scope = None,
name = "__class__"
)
return result.makeReference( self )
else:
return self.provider.getVariableForReference(
variable_name
)
if self.hasProvidedVariable( variable_name ):
return self.getProvidedVariable( variable_name )
else:
return self.provider.getVariableForClosure( variable_name )
def createProvidedVariable(self, variable_name):
# print( "createProvidedVariable", self, variable_name )
if self.local_locals:
if self.isClassDictCreation():
return Variables.ClassVariable(
owner = self,
variable_name = variable_name
)
else:
return Variables.LocalVariable(
owner = self,
variable_name = variable_name
)
else:
# Make sure the provider knows it has to provide a variable of this
# name for the assigment.
self.provider.getVariableForAssignment(
variable_name = variable_name
)
return self.getClosureVariable(
variable_name = variable_name
)
def addNonlocalsDeclaration(self, names, source_ref):
self.non_local_declarations.append( ( names, source_ref ) )
def getNonlocalDeclarations(self):
return self.non_local_declarations
getBody = ChildrenHavingMixin.childGetter( "body" )
setBody = ChildrenHavingMixin.childSetter( "body" )
def needsCreation(self):
# TODO: This looks kind of arbitrary, the users should decide, if they
# need it.
return self.needs_creation
def markAsNeedsCreation(self):
self.needs_creation = True
def needsDirectCall(self):
return self.needs_direct
def markAsDirectlyCalled(self):
self.needs_direct = True
def isCrossModuleUsed(self):
return self.cross_module_use
def markAsCrossModuleUsed(self):
self.cross_module_use = True
def computeExpression(self, constraint_collection):
assert False
# Function body is quite irreplacable.
return self, None, None
def computeExpressionCall(self, call_node, constraint_collection):
# TODO: Until we have something to re-order the arguments, we need to
# skip this. For the immediate need, we avoid this complexity, as a
# re-ordering will be needed.
if call_node.getNamedArgumentPairs():
return call_node, None, None
call_spec = self.getParameters()
try:
args_dict = matchCall(
func_name = self.getName(),
args = call_spec.getArgumentNames(),
star_list_arg = call_spec.getStarListArgumentName(),
star_dict_arg = call_spec.getStarDictArgumentName(),
num_defaults = call_spec.getDefaultCount(),
positional = call_node.getPositionalArguments(),
pairs = ()
)
values = []
for positional_arg in call_node.getPositionalArguments():
for _arg_name, arg_value in iterItems( args_dict ):
if arg_value is positional_arg:
values.append( arg_value )
result = ExpressionFunctionCall(
function_body = self,
values = values,
source_ref = call_node.getSourceReference()
)
return (
result,
"new_statements", # TODO: More appropiate tag maybe.
"""Replaced call to created function body '%s' with direct \
function call""" % self.getName()
)
except TooManyArguments as e:
from .NodeMakingHelpers import (
makeRaiseExceptionReplacementExpressionFromInstance,
wrapExpressionWithSideEffects
)
result = wrapExpressionWithSideEffects(
new_node = makeRaiseExceptionReplacementExpressionFromInstance(
expression = call_node,
exception = e.getRealException()
),
old_node = call_node,
side_effects = call_node.extractPreCallSideEffects()
)
return (
result,
"new_statements,new_raise", # TODO: More appropiate tag maybe.
"""Replaced call to created function body '%s' to argument \
error""" % self.getName()
)
def isCompileTimeConstant(self):
# TODO: It's actually pretty much compile time accessible mayhaps.
return None
def mayHaveSideEffects(self):
# The function definition has no side effects, calculating the defaults
# would be, but that is done outside of this.
return False
def markAsClassClosureTaker(self):
self.has_super = True
def isClassClosureTaker(self):
return self.has_super
def makeCloneAt(self, source_ref):
result = self.__class__(
provider = self.provider,
name = self.name,
doc = self.name,
# TODO: Clone parameters too, when we start to mutate them.
parameters = self.parameters,
source_ref = source_ref
)
result.setBody(
self.getBody().makeCloneAt( source_ref )
)
return result
def markAsExceptionReturnValue(self):
self.return_exception = True
def needsExceptionReturnValue(self):
return self.return_exception
class ExpressionFunctionCreation( SideEffectsFromChildrenMixin,
ExpressionChildrenHavingBase ):
kind = "EXPRESSION_FUNCTION_CREATION"
# Note: The order of evaluation for these is a bit unexpected, but
# true. Keyword defaults go first, then normal defaults, and annotations of
# all kinds go last.
named_children = (
"kw_defaults", "defaults", "annotations", "function_ref"
)
def __init__( self, function_ref, defaults, kw_defaults, annotations,
source_ref ):
assert kw_defaults is None or kw_defaults.isExpression()
assert annotations is None or annotations.isExpression()
assert function_ref.isExpressionFunctionRef()
ExpressionChildrenHavingBase.__init__(
self,
values = {
"function_ref" : function_ref,
"defaults" : tuple(defaults),
"kw_defaults" : kw_defaults,
"annotations" : annotations
},
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
# TODO: Function body may know something.
return self, None, None
getFunctionRef = ExpressionChildrenHavingBase.childGetter( "function_ref" )
getDefaults = ExpressionChildrenHavingBase.childGetter( "defaults" )
getKwDefaults = ExpressionChildrenHavingBase.childGetter( "kw_defaults" )
getAnnotations = ExpressionChildrenHavingBase.childGetter( "annotations" )
def mayRaiseException(self, exception_type):
for default in self.getDefaults():
result = default.mayRaiseException(exception_type)
if result is True or result is None:
return result
kw_defaults = self.getKwDefaults()
if kw_defaults is not None:
result = kw_defaults.mayRaiseException(exception_type)
if result is True or result is None:
return result
annotations = self.getAnnotations()
if annotations is not None:
result = annotations.mayRaiseException( exception_type )
if result is True or result is None:
return result
return False
class ExpressionFunctionRef(NodeBase, ExpressionMixin):
kind = "EXPRESSION_FUNCTION_REF"
def __init__(self, function_body, source_ref):
assert function_body.isExpressionFunctionBody()
NodeBase.__init__(
self,
source_ref = source_ref
)
self.function_body = function_body
# SSA trace based information about the function.
self.collection = None
def getDetails(self):
return {
"function" : self.function_body.getCodeName()
}
def makeCloneAt(self, source_ref):
return ExpressionFunctionRef(
function_body = self.function_body,
source_ref = source_ref
)
def getFunctionBody(self):
return self.function_body
def computeExpressionRaw(self, constraint_collection):
function_body = self.getFunctionBody()
owning_module = function_body.getParentModule()
from nuitka.ModuleRegistry import addUsedModule
addUsedModule( owning_module )
owning_module.addUsedFunction( function_body )
from nuitka.optimizations.ConstraintCollections import ConstraintCollectionFunction
collection = ConstraintCollectionFunction(
parent = constraint_collection,
function_body = function_body
)
function_body.collection = collection
# TODO: Function collection may now know something.
return self, None, None
def mayHaveSideEffects(self):
# Using a function has no side effects.
return False
class ExpressionFunctionCall(ExpressionChildrenHavingBase):
kind = "EXPRESSION_FUNCTION_CALL"
named_children = ( "function", "values" )
def __init__(self, function, values, source_ref):
assert function.isExpressionFunctionCreation()
ExpressionChildrenHavingBase.__init__(
self,
values = {
"function" : function,
"values" : tuple( values ),
},
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
function = self.getFunction()
if function.willRaiseException( BaseException ):
return function, "new_raise", "Called function is a raise"
values = self.getArgumentValues()
for count, value in enumerate( values ):
if value.willRaiseException( BaseException ):
from .NodeMakingHelpers import wrapExpressionWithSideEffects
result = wrapExpressionWithSideEffects(
side_effects = [ function ] + list( values[ : count ] ),
new_node = value,
old_node = self
)
return result, "new_raise", "Called function arguments raise"
return self, None, None
getFunction = ExpressionChildrenHavingBase.childGetter( "function" )
getArgumentValues = ExpressionChildrenHavingBase.childGetter( "values" )
Nuitka-0.5.0.1/nuitka/nodes/StatementNodes.py 0000644 0001750 0001750 00000030636 12265264105 021265 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Nodes for statements.
"""
from .NodeBases import StatementChildrenHavingBase
from nuitka.Utils import python_version
def checkStatements(value):
""" Check that statements list value propert.
Must not be None, must not contain None, and of course only statements,
may be empty.
"""
assert value is not None
assert None not in value
for statement in value:
assert statement.isStatement() or statement.isStatementsFrame(), \
statement
return tuple(value)
class StatementsSequence(StatementChildrenHavingBase):
kind = "STATEMENTS_SEQUENCE"
named_children = (
"statements",
)
checkers = {
"statements" : checkStatements
}
def __init__(self, statements, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"statements" : statements
},
source_ref = source_ref
)
getStatements = StatementChildrenHavingBase.childGetter("statements")
setStatements = StatementChildrenHavingBase.childSetter("statements")
def getDetails(self):
if self.getStatements():
return {
"statement_count" : len( self.getStatements() )
}
else:
return {
"statement_count" : 0
}
# Overloading name based automatic check, so that derived ones know it too.
def isStatementsSequence(self):
# Virtual method, pylint: disable=R0201,W0613
return True
def trimStatements(self, statement):
assert statement.parent is self
old_statements = list(self.getStatements())
assert statement in old_statements, \
(statement, self)
new_statements = old_statements[ : old_statements.index(statement)+1 ]
self.setChild("statements", new_statements)
def removeStatement(self, statement):
assert statement.parent is self
statements = list(self.getStatements())
statements.remove(statement)
self.setChild("statements", statements)
def mergeStatementsSequence(self, statement_sequence):
assert statement_sequence.parent is self
old_statements = list(self.getStatements())
assert statement_sequence in old_statements, \
(statement_sequence, self)
merge_index = old_statements.index(statement_sequence)
new_statements = tuple(old_statements[ : merge_index ]) + \
statement_sequence.getStatements() + \
tuple(old_statements[ merge_index+1 : ])
self.setChild("statements", new_statements)
def mayHaveSideEffects(self):
# Statement sequences have a side effect if one of the statements does.
for statement in self.getStatements():
if statement.mayHaveSideEffects():
return True
else:
return False
def isStatementAborting(self):
return self.getStatements()[-1].isStatementAborting()
def computeStatement(self, constraint_collection):
# Don't want to be called like this.
assert False
def computeStatementsSequence(self, constraint_collection):
# Expect to be overloaded.
assert not self.isStatementsFrame(), self
new_statements = []
statements = self.getStatements()
assert statements, self
for count, statement in enumerate(statements):
# May be frames embedded.
if statement.isStatementsFrame():
new_statement = statement.computeStatementsSequence(
constraint_collection
)
else:
new_statement = constraint_collection.onStatement(
statement = statement
)
if new_statement is not None:
if new_statement.isStatementsSequence() and \
not new_statement.isStatementsFrame():
new_statements.extend(
new_statement.getStatements()
)
else:
new_statements.append(
new_statement
)
if statement is not statements[-1] and \
new_statement.isStatementAborting():
constraint_collection.signalChange(
"new_statements",
statements[count + 1].getSourceReference(),
"Removed dead statements."
)
break
if statements != new_statements:
if new_statements:
self.setStatements(new_statements)
return self
else:
return None
else:
return self
def checkFrameStatements(value):
""" Check that frames statements list value proper.
Must not be None, must not contain None, and of course only statements
sequences, or statements, may be empty.
"""
assert value is not None
assert None not in value
for statement in value:
assert statement.isStatement() or statement.isStatementsFrame(), \
statement
return tuple(value)
class StatementsFrame(StatementsSequence):
kind = "STATEMENTS_FRAME"
checkers = {
"statements" : checkFrameStatements
}
def __init__(self, statements, guard_mode, code_name, var_names, arg_count,
kw_only_count, has_starlist, has_stardict, source_ref):
StatementsSequence.__init__(
self,
statements = statements,
source_ref = source_ref
)
self.var_names = tuple( var_names )
self.code_name = code_name
self.kw_only_count = kw_only_count
self.arg_count = arg_count
self.guard_mode = guard_mode
self.has_starlist = has_starlist
self.has_stardict = has_stardict
self.needs_frame_exception_preserve = False
def getDetails(self):
result = {
"code_name" : self.code_name,
"var_names" : ", ".join( self.var_names ),
"guard_mode" : self.guard_mode
}
if python_version >= 300:
result["kw_only_count"] = self.kw_only_count
result.update(StatementsSequence.getDetails(self))
return result
def needsLineNumber(self):
return False
def getGuardMode(self):
return self.guard_mode
def getVarNames(self):
return self.var_names
def getCodeObjectName(self):
return self.code_name
def getKwOnlyParameterCount(self):
return self.kw_only_count
def getArgumentCount(self):
return self.arg_count
def makeCloneAt(self, source_ref):
assert False
def markAsFrameExceptionPreserving(self):
self.needs_frame_exception_preserve = True
def needsFrameExceptionPreversing(self):
return self.needs_frame_exception_preserve
def getCodeObjectHandle(self, context):
provider = self.getParentVariableProvider()
# TODO: Why do this accessing a node, do this outside.
from nuitka.codegen.CodeObjectCodes import getCodeObjectHandle
return getCodeObjectHandle(
context = context,
filename = self.source_ref.getFilename(),
var_names = self.getVarNames(),
arg_count = self.getArgumentCount(),
kw_only_count = self.getKwOnlyParameterCount(),
line_number = 0
if provider.isPythonModule() else
self.source_ref.getLineNumber(),
code_name = self.getCodeObjectName(),
is_generator = provider.isExpressionFunctionBody() and \
provider.isGenerator(),
is_optimized = not provider.isPythonModule() and \
not provider.isClassDictCreation() and \
not context.hasLocalsDict(),
has_starlist = self.has_starlist,
has_stardict = self.has_stardict
)
def computeStatementsSequence(self, constraint_collection):
new_statements = []
statements = self.getStatements()
for count, statement in enumerate(statements):
# May be frames embedded.
if statement.isStatementsFrame():
new_statement = statement.computeStatementsSequence(
constraint_collection = constraint_collection
)
else:
new_statement = constraint_collection.onStatement(
statement = statement
)
if new_statement is not None:
if new_statement.isStatementsSequence() and \
not new_statement.isStatementsFrame():
new_statements.extend(new_statement.getStatements())
else:
new_statements.append(new_statement)
if statement is not statements[-1] and \
new_statement.isStatementAborting():
constraint_collection.signalChange(
"new_statements",
statements[count+1].getSourceReference(),
"Removed dead statements."
)
break
if not new_statements:
return None
# Determine statements inside the frame, that need not be in a frame,
# because they wouldn't raise an exception.
outside_pre = []
while new_statements and \
not new_statements[0].mayRaiseException(BaseException):
outside_pre.append(new_statements[0])
del new_statements[0]
outside_post = []
while new_statements and \
not new_statements[-1].mayRaiseException(BaseException):
outside_post.insert(0, new_statements[-1])
del new_statements[-1]
if outside_pre or outside_post:
from .NodeMakingHelpers import makeStatementsSequenceReplacementNode
if new_statements:
self.setStatements(new_statements)
return makeStatementsSequenceReplacementNode(
statements = outside_pre + [self] + \
outside_post,
node = self
)
else:
return makeStatementsSequenceReplacementNode(
statements = outside_pre + outside_post,
node = self
)
else:
if statements != new_statements:
self.setStatements(new_statements)
return self
class StatementExpressionOnly(StatementChildrenHavingBase):
kind = "STATEMENT_EXPRESSION_ONLY"
named_children = ("expression",)
def __init__(self, expression, source_ref):
assert expression.isExpression()
StatementChildrenHavingBase.__init__(
self,
values = {
"expression" : expression
},
source_ref = source_ref
)
def getDetail(self):
return "expression %s" % self.getExpression()
def mayHaveSideEffects(self):
return self.getExpression().mayHaveSideEffects()
getExpression = StatementChildrenHavingBase.childGetter(
"expression"
)
def computeStatement(self, constraint_collection):
constraint_collection.onExpression(
expression = self.getExpression()
)
expression = self.getExpression()
return expression.computeExpressionDrop(
statement = self,
constraint_collection = constraint_collection
)
Nuitka-0.5.0.1/nuitka/nodes/GlobalsLocalsNodes.py 0000644 0001750 0001750 00000010367 12265264105 022041 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Globals/locals/dir0/dir1 nodes
These nodes give access to variables, highly problematic, because using them,
the code may change or access anything about them, so nothing can be trusted
anymore, if we start to not know where their value goes.
"""
from .NodeBases import (
ExpressionBuiltinSingleArgBase,
StatementChildrenHavingBase,
ExpressionMixin,
NodeBase
)
class ExpressionBuiltinGlobals(NodeBase, ExpressionMixin):
kind = "EXPRESSION_BUILTIN_GLOBALS"
def __init__(self, source_ref):
NodeBase.__init__( self, source_ref = source_ref )
def computeExpression(self, constraint_collection):
return self, None, None
class ExpressionBuiltinLocals(NodeBase, ExpressionMixin):
kind = "EXPRESSION_BUILTIN_LOCALS"
def __init__(self, source_ref):
NodeBase.__init__( self, source_ref = source_ref )
def computeExpression(self, constraint_collection):
# Just inform the collection that all escaped.
for variable_ref in constraint_collection.getActiveVariables():
variable = variable_ref
while variable.isReference():
variable = variable.getReferenced()
# TODO: Currently this is a bit difficult to express in a positive
# way
if not variable.isTempKeeperVariable() and not variable.isTempVariable():
variable_trace = constraint_collection.getVariableCurrentTrace(
variable_ref
)
variable_trace.addUsage( self )
return self, None, None
def needsLocalsDict(self):
return self.getParentVariableProvider().isEarlyClosure() and \
( not self.getParent().isStatementReturn() or self.getParent().isExceptionDriven() )
class StatementSetLocals(StatementChildrenHavingBase):
kind = "STATEMENT_SET_LOCALS"
named_children = ( "new_locals", )
def __init__(self, new_locals, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"new_locals" : new_locals,
},
source_ref = source_ref
)
def needsLocalsDict(self):
return True
getNewLocals = StatementChildrenHavingBase.childGetter( "new_locals" )
def computeStatement(self, constraint_collection):
# Make sure that we don't even assume "unset" of things not set yet for anything.
constraint_collection.removeAllKnowledge()
constraint_collection.onExpression( self.getNewLocals() )
new_locals = self.getNewLocals()
if new_locals.willRaiseException( BaseException ):
from .NodeMakingHelpers import makeStatementExpressionOnlyReplacementNode
result = makeStatementExpressionOnlyReplacementNode(
expression = new_locals,
node = self
)
return result, "new_raise", "Setting locals already raises implicitely building new locals."
return self, None, None
class ExpressionBuiltinDir0(NodeBase, ExpressionMixin):
kind = "EXPRESSION_BUILTIN_DIR0"
def __init__(self, source_ref):
NodeBase.__init__( self, source_ref = source_ref )
def computeExpression(self, constraint_collection):
return self, None, None
class ExpressionBuiltinDir1(ExpressionBuiltinSingleArgBase):
kind = "EXPRESSION_BUILTIN_DIR1"
def computeExpression(self, constraint_collection):
# TODO: Quite some cases should be possible to predict.
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/VariableRefNodes.py 0000644 0001750 0001750 00000022564 12265264105 021504 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Node for variable references.
These represent all variable references in the node tree. Can be in assignments
and its expressions, changing the meaning of course dramatically.
"""
from nuitka import Variables, Builtins
from .NodeBases import (
ExpressionMixin,
NodeBase
)
from .ConstantRefNodes import ExpressionConstantRef
def _isReadOnlyUnterdeterminedModuleVariable(variable):
return variable.isModuleVariable() and \
variable.getReadOnlyIndicator() is None
def _isReadOnlyModuleVariable(variable):
return (
variable.isModuleVariable() and \
variable.getReadOnlyIndicator() is True
) or variable.isMaybeLocalVariable()
class ExpressionVariableRef(NodeBase, ExpressionMixin):
kind = "EXPRESSION_VARIABLE_REF"
def __init__(self, variable_name, source_ref):
NodeBase.__init__( self, source_ref = source_ref )
self.variable_name = variable_name
self.variable = None
def getDetails(self):
if self.variable is None:
return { "name" : self.variable_name }
else:
return { "name" : self.variable_name, "variable" : self.variable }
def getDetail(self):
if self.variable is None:
return self.variable_name
else:
return repr( self.variable )
def makeCloneAt(self, source_ref):
result = self.__class__(
variable_name = self.variable_name,
source_ref = source_ref
)
result.variable = self.variable
return result
def isTargetVariableRef(self):
return False
def getVariableName(self):
return self.variable_name
def getVariable(self):
return self.variable
def setVariable(self, variable):
assert isinstance( variable, Variables.Variable ), repr( variable )
assert self.variable is None
self.variable = variable
def computeExpression(self, constraint_collection):
assert self.variable is not None
if _isReadOnlyUnterdeterminedModuleVariable( self.variable ):
constraint_collection.assumeUnclearLocals()
constraint_collection.signalChange(
"new_expression",
self.source_ref,
"Unclear module variable delays processing."
)
if _isReadOnlyModuleVariable( self.variable ):
if self.variable_name in Builtins.builtin_exception_names:
from .BuiltinRefNodes import ExpressionBuiltinExceptionRef
new_node = ExpressionBuiltinExceptionRef(
exception_name = self.variable_name,
source_ref = self.getSourceReference()
)
# TODO: More like "removed_variable and new_constant" probably
change_tags = "new_builtin"
change_desc = """\
Module variable '%s' found to be builtin exception reference.""" % (
self.variable_name
)
elif self.variable_name in Builtins.builtin_names:
from .BuiltinRefNodes import ExpressionBuiltinRef
new_node = ExpressionBuiltinRef(
builtin_name = self.variable_name,
source_ref = self.getSourceReference()
)
# TODO: More like "removed_variable and new_constant" probably
change_tags = "new_builtin"
change_desc = """\
Module variable '%s' found to be builtin reference.""" % (
self.variable_name
)
elif self.variable_name == "__name__":
new_node = ExpressionConstantRef(
constant = self.variable.getReferenced().getOwner().\
getFullName(),
source_ref = self.getSourceReference()
)
change_tags = "new_constant"
change_desc = """\
Replaced read-only module attribute '__name__' with constant value."""
elif self.variable_name == "__package__":
new_node = ExpressionConstantRef(
constant = self.variable.getReferenced().getOwner().\
getPackage(),
source_ref = self.getSourceReference()
)
change_tags = "new_constant"
change_desc = """\
Replaced read-only module attribute '__package__' with constant value."""
else:
# Probably should give a warning once about it.
new_node = self
change_tags = None
change_desc = None
return new_node, change_tags, change_desc
return self, None, None
def onContentEscapes(self, constraint_collection):
constraint_collection.onVariableContentEscapes( self.variable )
def isKnownToBeIterable(self, count):
return None
def mayProvideReference(self):
# Variables are capable of "asObject0".
return False
def mayHaveSideEffects(self):
# TODO: Remembered traced could tell better.
return True
class ExpressionTargetVariableRef(ExpressionVariableRef):
kind = "EXPRESSION_TARGET_VARIABLE_REF"
def __init__(self, variable_name, source_ref):
ExpressionVariableRef.__init__( self, variable_name, source_ref )
self.variable_version = None
def getDetails(self):
if self.variable is None:
return { "name" : self.variable_name }
else:
return {
"name" : self.variable_name,
"variable" : self.variable,
"version" : self.variable_version
}
def makeCloneAt(self, source_ref):
result = self.__class__(
variable_name = self.variable_name,
source_ref = source_ref
)
if self.variable is not None:
result.setVariable( self.variable )
return result
def computeExpression(self, constraint_collection):
assert False
def isTargetVariableRef(self):
return True
def getVariableVersion(self):
assert self.variable_version is not None, self
return self.variable_version
def setVariable(self, variable):
ExpressionVariableRef.setVariable( self, variable )
self.variable_version = variable.allocateTargetNumber()
assert self.variable_version is not None
class ExpressionTempVariableRef(NodeBase, ExpressionMixin):
kind = "EXPRESSION_TEMP_VARIABLE_REF"
def __init__(self, variable, source_ref):
NodeBase.__init__( self, source_ref = source_ref )
self.variable = variable
def getDetails(self):
return { "name" : self.variable.getName() }
def getDetail(self):
return self.variable.getName()
def makeCloneAt(self, source_ref):
return self.__class__(
variable = self.variable,
source_ref = source_ref
)
def getVariableName(self):
return self.variable.getName()
def getVariable(self):
return self.variable
def isTargetVariableRef(self):
return False
def computeExpression(self, constraint_collection):
constraint_collection.onVariableUsage( self )
# Nothing to do here.
return self, None, None
def onContentEscapes(self, constraint_collection):
constraint_collection.onVariableContentEscapes( self.variable )
def mayRaiseException(self, exception_type):
# Can't happen
return False
def isKnownToBeIterableAtMin(self, count):
# TODO: See through the variable current trace.
return None
def isKnownToBeIterableAtMax(self, count):
# TODO: See through the variable current trace.
return None
# Python3 only, it updates temporary variables that are closure variables.
def setVariable(self, variable):
self.variable = variable
class ExpressionTargetTempVariableRef(ExpressionTempVariableRef):
kind = "EXPRESSION_TARGET_TEMP_VARIABLE_REF"
def __init__(self, variable, source_ref):
ExpressionTempVariableRef.__init__( self, variable, source_ref )
self.variable_version = variable.allocateTargetNumber()
def computeExpression(self, constraint_collection):
assert False, self.parent
def isTargetVariableRef(self):
return True
def getVariableVersion(self):
return self.variable_version
# Python3 only, it updates temporary variables that are closure variables.
def setVariable(self, variable):
ExpressionTempVariableRef.setVariable( self, variable )
self.variable_version = self.variable.allocateTargetNumber()
Nuitka-0.5.0.1/nuitka/nodes/ConstantRefNodes.py 0000644 0001750 0001750 00000020251 12265264105 021537 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Node for constant expressions. Can be any builtin type.
"""
from .NodeBases import NodeBase, CompileTimeConstantExpressionMixin
from nuitka.Constants import (
getConstantIterationLength,
isIterableConstant,
isIndexConstant,
isNumberConstant,
isConstant,
isMutable
)
from nuitka.Options import isDebug
# pylint: disable=W0622
from nuitka.__past__ import iterItems, unicode
# pylint: enable=W0622
from logging import warning
class ExpressionConstantRef(CompileTimeConstantExpressionMixin, NodeBase):
kind = "EXPRESSION_CONSTANT_REF"
def __init__(self, constant, source_ref, user_provided = False):
NodeBase.__init__( self, source_ref = source_ref )
CompileTimeConstantExpressionMixin.__init__( self )
assert isConstant( constant ), constant
self.constant = constant
self.user_provided = user_provided
if not user_provided and isDebug():
try:
size = len( constant )
if type( constant ) in ( str, unicode ):
max_size = 1000
else:
max_size = 256
if size > max_size:
warning(
"Too large constant (%s %d) encountered at %s.",
type( constant ),
size,
source_ref.getAsString()
)
except TypeError:
pass
# TODO: Make this a warning, and cover all constant types.
# assert type( constant ) is not str or len( constant ) < 30000
def __repr__(self):
return "" % ( self.kind, self.constant, self.source_ref, self.user_provided )
def makeCloneAt(self, source_ref):
return self.__class__(self.constant, source_ref)
def getDetails(self):
return { "value" : repr( self.constant ), "user_provided" : self.user_provided }
def getDetail(self):
return repr( self.constant )
def computeExpression(self, constraint_collection):
# No need to check anything, pylint: disable=W0613
# Cannot compute any further, this is already the best.
return self, None, None
def computeExpressionCall(self, call_node, constraint_collection):
from .NodeMakingHelpers import makeRaiseExceptionReplacementExpression, wrapExpressionWithSideEffects
new_node = wrapExpressionWithSideEffects(
new_node = makeRaiseExceptionReplacementExpression(
expression = self,
exception_type = "TypeError",
exception_value = "'%s' object is not callable" % type( self.constant ).__name__
),
old_node = call_node,
side_effects = call_node.extractPreCallSideEffects()
)
return new_node, "new_raise", "Predicted call of constant value to exception raise."
def getCompileTimeConstant(self):
return self.constant
getConstant = getCompileTimeConstant
def isMutable(self):
return isMutable( self.constant )
def isNumberConstant(self):
return isNumberConstant( self.constant )
def isIndexConstant(self):
return isIndexConstant( self.constant )
def isStringConstant(self):
return type( self.constant ) is str
def isIndexable(self):
return self.constant is None or self.isNumberConstant()
def isKnownToBeIterable(self, count):
if isIterableConstant( self.constant ):
return count is None or getConstantIterationLength( self.constant ) == count
else:
return False
def isKnownToBeIterableAtMin(self, count):
length = self.getIterationLength()
return length is not None and length >= count
def canPredictIterationValues(self):
return self.isKnownToBeIterable( None )
def getIterationValue(self, count):
assert count < len( self.constant )
return ExpressionConstantRef( self.constant[ count ], self.source_ref )
def getIterationValues(self):
source_ref = self.getSourceReference()
return tuple(
ExpressionConstantRef(
constant = value,
source_ref = source_ref
)
for value in
self.constant
)
def isMapping(self):
return type( self.constant ) is dict
def isMappingWithConstantStringKeys(self):
assert self.isMapping()
for key in self.constant:
if type( key ) not in ( str, unicode ):
return False
else:
return True
def getMappingPairs(self):
assert self.isMapping()
pairs = []
source_ref = self.getSourceReference()
for key, value in iterItems( self.constant ):
pairs.append(
ExpressionConstantRef(
constant = key,
source_ref = source_ref
),
ExpressionConstantRef(
constant = value,
source_ref = source_ref
)
)
return pairs
def getMappingStringKeyPairs(self):
assert self.isMapping()
pairs = []
source_ref = self.getSourceReference()
for key, value in iterItems( self.constant ):
pairs.append(
(
key,
ExpressionConstantRef(
constant = value,
source_ref = source_ref
)
)
)
return pairs
def isBoolConstant(self):
return type( self.constant ) is bool
def mayHaveSideEffects(self):
# Constants have no side effects
return False
def extractSideEffects(self):
# Constants have no side effects
return ()
def mayRaiseException(self, exception_type):
# Virtual method, pylint: disable=R0201,W0613
# Constants won't raise any kind of exception.
return False
def mayProvideReference(self):
return self.isMutable()
def getIntegerValue(self):
if self.isNumberConstant():
return int( self.constant )
else:
return None
def getStringValue(self):
if self.isStringConstant():
return self.constant
else:
return None
def getIterationLength(self):
if isIterableConstant( self.constant ):
return getConstantIterationLength( self.constant )
else:
return None
def isIterableConstant(self):
return isIterableConstant( self.constant )
def getStrValue(self):
if type( self.constant ) is str:
return self
else:
return ExpressionConstantRef(
constant = str( self.constant ),
source_ref = self.getSourceReference()
)
def computeExpressionIter1(self, iter_node, constraint_collection):
if type(self.constant) in (list, set, frozenset, dict):
result = ExpressionConstantRef(
constant = tuple(self.constant),
user_provided = self.user_provided,
source_ref = self.getSourceReference()
)
self.replaceWith(result)
return iter_node, "new_constant", """
Iteration over constant %s changed to tuple.""" % type(self.constant).__name__
return iter_node, None, None
Nuitka-0.5.0.1/nuitka/nodes/IndicatorMixins.py 0000644 0001750 0001750 00000006355 12265264105 021435 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Module for node class mixins that indicate runtime determined facts about a node.
These come into play after finalization only. All of the these attributes (and we could
use properties instead) are determined once or from a default and then used like this.
"""
class MarkContainsTryExceptIndicator:
""" Mixin for indication that a module, class or function contains a try/except.
"""
def __init__(self):
self.try_except_containing = False
self.try_finally_containing = False
self.raise_containing = False
def markAsTryExceptContaining(self):
self.try_except_containing = True
def isTryExceptContaining(self):
return self.try_except_containing
def markAsTryFinallyContaining(self):
self.try_finally_containing = True
def isTryFinallyContaining(self):
return self.try_finally_containing
def markAsRaiseContaining(self):
self.raise_containing = True
def isRaiseContaining(self):
return self.raise_containing
class MarkLocalsDictIndicator:
def __init__(self):
self.needs_locals_dict = False
def hasLocalsDict(self):
return self.needs_locals_dict
def markAsLocalsDict(self):
self.needs_locals_dict = True
class MarkGeneratorIndicator:
""" Mixin for indication that a function/lambda is a generator.
"""
def __init__(self):
self.is_generator = False
def markAsGenerator(self):
self.is_generator = True
def isGenerator(self):
return self.is_generator
class MarkUnoptimizedFunctionIndicator:
""" Mixin for indication that a function contains an exec or star import.
These do not access global variables directly, but check a locals dictionary
first, because they do.
"""
def __init__(self):
self.unoptimized_locals = False
self.unqualified_exec = False
self.exec_source_ref = None
def markAsExecContaining(self):
self.unoptimized_locals = True
def markAsUnqualifiedExecContaining(self, source_ref):
self.unqualified_exec = True
# Let the first one win.
if self.exec_source_ref is None:
self.exec_source_ref = source_ref
markAsStarImportContaining = markAsExecContaining
def isUnoptimized(self):
return self.unoptimized_locals
def isUnqualifiedExec(self):
return self.unoptimized_locals and self.unqualified_exec
def getExecSourceRef(self):
return self.exec_source_ref
Nuitka-0.5.0.1/nuitka/nodes/__init__.py 0000644 0001750 0001750 00000001501 12265264105 020054 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Dummy file to make this directory a package. """
Nuitka-0.5.0.1/nuitka/nodes/BuiltinRangeNodes.py 0000644 0001750 0001750 00000025710 12265264105 021701 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Node the calls to the 'range' builtin.
This is a rather complex beast as it has many cases, is difficult to know if
it's sizable enough to compute, and there are complex cases, where the bad
result of it can be predicted still, and these are interesting for warnings.
"""
from .NodeBases import (
ExpressionChildrenHavingBase,
ExpressionBuiltinNoArgBase
)
from nuitka.optimizations import BuiltinOptimization
from nuitka.Utils import python_version
import math
class ExpressionBuiltinRange0(ExpressionBuiltinNoArgBase):
kind = "EXPRESSION_BUILTIN_RANGE0"
def __init__(self, source_ref):
ExpressionBuiltinNoArgBase.__init__(
self,
builtin_function = range,
source_ref = source_ref
)
def mayHaveSideEffects(self):
return False
class ExpressionBuiltinRangeBase(ExpressionChildrenHavingBase):
""" Base class for range nodes with 1/2/3 arguments. """
builtin_spec = BuiltinOptimization.builtin_range_spec
def __init__(self, values, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = values,
source_ref = source_ref
)
def getTruthValue(self):
length = self.getIterationLength()
if length is None:
return None
else:
return length > 0
def mayHaveSideEffects(self):
for child in self.getVisitableNodes():
if child.mayHaveSideEffects():
return True
if child.getIntegerValue() is None:
return True
if python_version >= 270 and \
child.isExpressionConstantRef() and \
type(child.getConstant()) is float:
return True
else:
return False
def computeBuiltinSpec(self, given_values):
assert self.builtin_spec is not None, self
if not self.builtin_spec.isCompileTimeComputable(given_values):
return self, None, None
from .NodeMakingHelpers import getComputationResult
return getComputationResult(
node = self,
computation = lambda : self.builtin_spec.simulateCall(
given_values
),
description = "Builtin call to %s precomputed." % (
self.builtin_spec.getName()
)
)
def computeExpressionIter1(self, iter_node, constraint_collection):
# TODO: Support Python3 range objects too.
if python_version >= 300:
return iter_node, None, None
iteration_length = self.getIterationLength()
if iteration_length is not None and iteration_length > 256:
result = ExpressionBuiltinXrange(
low = self.getLow(),
high = self.getHigh(),
step = self.getStep(),
source_ref = self.getSourceReference()
)
self.replaceWith(result)
return iter_node, "new_expression", "Replaced range with xrange."
return iter_node, None, None
def getHigh(self):
return None
def getStep(self):
return None
class ExpressionBuiltinRange1(ExpressionBuiltinRangeBase):
kind = "EXPRESSION_BUILTIN_RANGE1"
named_children = ( "low", )
def __init__(self, low, source_ref):
assert low is not None
ExpressionBuiltinRangeBase.__init__(
self,
values = {
"low" : low,
},
source_ref = source_ref
)
getLow = ExpressionChildrenHavingBase.childGetter( "low" )
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
# TODO: Support Python3 range objects too.
if python_version >= 300:
return self, None, None
low = self.getLow()
return self.computeBuiltinSpec(
given_values = (
low,
)
)
def getIterationLength(self):
low = self.getLow().getIntegerValue()
if low is None:
return None
return max(0, low)
def canPredictIterationValues(self):
return self.getIterationLength() is not None
def getIterationValue(self, element_index):
length = self.getIterationLength()
if length is None:
return None
if element_index > length:
return None
from .NodeMakingHelpers import makeConstantReplacementNode
# TODO: Make sure to cast element_index to what CPython will give, for
# now a downcast will do.
return makeConstantReplacementNode(
constant = int(element_index),
node = self
)
def isKnownToBeIterable(self, count):
return count is None or count == self.getIterationLength()
class ExpressionBuiltinRange2(ExpressionBuiltinRangeBase):
kind = "EXPRESSION_BUILTIN_RANGE2"
named_children = ( "low", "high" )
def __init__(self, low, high, source_ref):
ExpressionBuiltinRangeBase.__init__(
self,
values = {
"low" : low,
"high" : high
},
source_ref = source_ref
)
getLow = ExpressionChildrenHavingBase.childGetter( "low" )
getHigh = ExpressionChildrenHavingBase.childGetter( "high" )
builtin_spec = BuiltinOptimization.builtin_range_spec
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
if python_version >= 300:
return self, None, None
low = self.getLow()
high = self.getHigh()
return self.computeBuiltinSpec( ( low, high ) )
def getIterationLength(self):
low = self.getLow()
high = self.getHigh()
low = low.getIntegerValue()
if low is None:
return None
high = high.getIntegerValue()
if high is None:
return None
return max( 0, high - low )
def canPredictIterationValues(self):
return self.getIterationLength() is not None
def getIterationValue(self, element_index):
low = self.getLow()
high = self.getHigh()
low = low.getIntegerValue()
if low is None:
return None
high = high.getIntegerValue()
if high is None:
return None
result = low + element_index
if result >= high:
return None
else:
from .NodeMakingHelpers import makeConstantReplacementNode
return makeConstantReplacementNode(
constant = result,
node = self
)
def isKnownToBeIterable(self, count):
return count is None or count == self.getIterationLength()
class ExpressionBuiltinRange3(ExpressionBuiltinRangeBase):
kind = "EXPRESSION_BUILTIN_RANGE3"
named_children = ( "low", "high", "step" )
def __init__(self, low, high, step, source_ref):
ExpressionBuiltinRangeBase.__init__(
self,
values = {
"low" : low,
"high" : high,
"step" : step
},
source_ref = source_ref
)
getLow = ExpressionChildrenHavingBase.childGetter( "low" )
getHigh = ExpressionChildrenHavingBase.childGetter( "high" )
getStep = ExpressionChildrenHavingBase.childGetter( "step" )
builtin_spec = BuiltinOptimization.builtin_range_spec
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
if python_version >= 300:
return self, None, None
low = self.getLow()
high = self.getHigh()
step = self.getStep()
return self.computeBuiltinSpec( ( low, high, step ) )
def getIterationLength(self):
low = self.getLow()
high = self.getHigh()
step = self.getStep()
low = low.getIntegerValue()
if low is None:
return None
high = high.getIntegerValue()
if high is None:
return None
step = step.getIntegerValue()
if step is None:
return None
# Give up on this, will raise ValueError.
if step == 0:
return None
if low < high:
if step < 0:
estimate = 0
else:
estimate = math.ceil( float( high - low ) / step )
else:
if step > 0:
estimate = 0
else:
estimate = math.ceil( float( high - low ) / step )
estimate = round( estimate )
assert not estimate < 0
return int( estimate )
def canPredictIterationValues(self):
return self.getIterationLength() is not None
def getIterationValue(self, element_index):
low = self.getLow().getIntegerValue()
if low is None:
return None
high = self.getHigh().getIntegerValue()
if high is None:
return None
step = self.getStep().getIntegerValue()
result = low + step * element_index
if result >= high:
return None
else:
from .NodeMakingHelpers import makeConstantReplacementNode
return makeConstantReplacementNode(
constant = result,
node = self
)
def isKnownToBeIterable(self, count):
return count is None or count == self.getIterationLength()
class ExpressionBuiltinXrange(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_XRANGE"
named_children = ("low", "high", "step")
def __init__(self, low, high, step, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"low" : low,
"high" : high,
"step" : step
},
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
return self, None, None
getLow = ExpressionChildrenHavingBase.childGetter( "low" )
getHigh = ExpressionChildrenHavingBase.childGetter( "high" )
getStep = ExpressionChildrenHavingBase.childGetter( "step" )
Nuitka-0.5.0.1/nuitka/nodes/ContainerMakingNodes.py 0000644 0001750 0001750 00000026673 12265264105 022400 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Nodes that build containers.
"""
from .NodeBases import (
ExpressionChildrenHavingBase,
SideEffectsFromChildrenMixin
)
from nuitka import Constants
class ExpressionMakeSequenceBase(SideEffectsFromChildrenMixin,
ExpressionChildrenHavingBase):
named_children = ("elements",)
def __init__(self, sequence_kind, elements, source_ref):
assert sequence_kind in ( "TUPLE", "LIST", "SET" ), sequence_kind
for element in elements:
assert element.isExpression(), element
self.sequence_kind = sequence_kind.lower()
ExpressionChildrenHavingBase.__init__(
self,
values = {
"elements" : tuple( elements ),
},
source_ref = source_ref
)
def isExpressionMakeSequence(self):
return True
def getSequenceKind(self):
return self.sequence_kind
getElements = ExpressionChildrenHavingBase.childGetter( "elements" )
def getSimulator(self):
# Abstract method, pylint: disable=R0201,W0613
return None
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
elements = self.getElements()
for count, element in enumerate( elements ):
if element.willRaiseException( BaseException ):
from .NodeMakingHelpers import wrapExpressionWithSideEffects
result = wrapExpressionWithSideEffects(
side_effects = elements[ : count ],
new_node = element,
old_node = self
)
return result, "new_raise", "Sequence creation raises exception"
# TODO: CompileTimeConstant should be good enough.
for element in elements:
if not element.isExpressionConstantRef():
return self, None, None
simulator = self.getSimulator()
assert simulator is not None
from .NodeMakingHelpers import getComputationResult
# The simulator is in fact callable if not None, pylint: disable=E1102
return getComputationResult(
node = self,
computation = lambda : simulator(
element.getConstant()
for element in
self.getElements()
),
description = "%s with constant arguments." % simulator
)
def mayHaveSideEffectsBool(self):
return False
def isKnownToBeIterable(self, count):
return count is None or count == len( self.getElements() )
def getIterationValue(self, count):
return self.getElements()[ count ]
def getIterationLength(self):
return len( self.getElements() )
def canPredictIterationValues(self):
return True
def getIterationValues(self):
return self.getElements()
def getTruthValue(self):
return self.getIterationLength() > 0
def computeExpressionDrop(self, statement, constraint_collection):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
result = makeStatementOnlyNodesFromExpressions(
expressions = self.getElements()
)
return result, "new_statements", """\
Removed sequence creation for unused sequence."""
class ExpressionMakeTuple(ExpressionMakeSequenceBase):
kind = "EXPRESSION_MAKE_TUPLE"
def __init__(self, elements, source_ref):
ExpressionMakeSequenceBase.__init__(
self,
sequence_kind = "TUPLE",
elements = elements,
source_ref = source_ref
)
def getSimulator(self):
return tuple
class ExpressionMakeList(ExpressionMakeSequenceBase):
kind = "EXPRESSION_MAKE_LIST"
def __init__(self, elements, source_ref):
ExpressionMakeSequenceBase.__init__(
self,
sequence_kind = "LIST",
elements = elements,
source_ref = source_ref
)
def getSimulator(self):
return list
def computeExpressionIter1(self, iter_node, constraint_collection):
result = ExpressionMakeTuple(
elements = self.getElements(),
source_ref = self.source_ref
)
self.replaceWith(result)
return iter_node, "new_expression", """\
Iteration of list reduced to tuple."""
class ExpressionMakeSet(ExpressionMakeSequenceBase):
kind = "EXPRESSION_MAKE_SET"
def __init__(self, elements, source_ref):
ExpressionMakeSequenceBase.__init__(
self,
sequence_kind = "SET",
elements = elements,
source_ref = source_ref
)
def getSimulator(self):
return set
def computeExpressionIter1(self, iter_node, constraint_collection):
result = ExpressionMakeTuple(
elements = self.getElements(),
source_ref = self.source_ref
)
self.replaceWith(result)
return iter_node, "new_expression", """\
Iteration of set reduced to tuple."""
class ExpressionKeyValuePair(SideEffectsFromChildrenMixin,
ExpressionChildrenHavingBase):
kind = "EXPRESSION_KEY_VALUE_PAIR"
named_children = ( "key", "value" )
def __init__(self, key, value, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"key" : key,
"value" : value
},
source_ref = source_ref
)
getKey = ExpressionChildrenHavingBase.childGetter( "key" )
getValue = ExpressionChildrenHavingBase.childGetter( "value" )
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
key = self.getKey()
if key.willRaiseException(BaseException):
return key, "new_raise", "Dictionary key raises exception"
value = self.getValue()
if value.willRaiseException(BaseException):
from .NodeMakingHelpers import wrapExpressionWithNodeSideEffects
result = wrapExpressionWithNodeSideEffects(
new_node = value,
old_node = key
)
return result, "new_raise", "Dictionary value raises exception"
return self, None, None
class ExpressionMakeDict(SideEffectsFromChildrenMixin,
ExpressionChildrenHavingBase):
kind = "EXPRESSION_MAKE_DICT"
named_children = ( "pairs", )
def __init__(self, pairs, lazy_order, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"pairs" : tuple( pairs ),
},
source_ref = source_ref
)
self.lazy_order = lazy_order
getPairs = ExpressionChildrenHavingBase.childGetter( "pairs" )
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
pairs = self.getPairs()
for count, pair in enumerate( pairs ):
if pair.willRaiseException( BaseException ):
from .NodeMakingHelpers import wrapExpressionWithSideEffects
result = wrapExpressionWithSideEffects(
side_effects = pairs[ : count ],
new_node = pair,
old_node = self
)
return result, "new_raise", "Dict creation raises exception"
for pair in pairs:
key = pair.getKey()
# TODO: Mutable key should cause something problematic.
if not key.isExpressionConstantRef() or key.isMutable():
return self, None, None
value = pair.getValue()
if not value.isExpressionConstantRef():
return self, None, None
constant_value = Constants.createConstantDict(
keys = [
pair.getKey().getConstant()
for pair in
pairs
],
values = [
pair.getValue().getConstant()
for pair in
pairs
],
lazy_order = self.lazy_order
)
from .NodeMakingHelpers import makeConstantReplacementNode
new_node = makeConstantReplacementNode(
constant = constant_value,
node = self
)
return new_node, "new_constant", """\
Created dictionary found to be constant."""
def mayHaveSideEffectsBool(self):
return False
def isKnownToBeIterable(self, count):
return count is None or count == len( self.getPairs() )
def getIterationLength(self):
return len( self.getPairs() )
def canPredictIterationValues(self):
# Dictionaries are fully predictable, pylint: disable=R0201
return True
def getIterationValue(self, count):
return self.getPairs()[ count ].getKey()
def getTruthValue(self):
return self.getIterationLength() > 0
def isMapping(self):
# Dictionaries are always mappings, but this is a virtual method,
# pylint: disable=R0201
return True
def isMappingWithConstantStringKeys(self):
for pair in self.getPairs():
key = pair.getKey()
if not key.isExpressionConstantRef() or not key.isStringConstant():
return False
else:
return True
def getMappingStringKeyPairs(self):
return [
(
pair.getKey().getConstant(),
pair.getValue()
)
for pair in
self.getPairs()
]
def getMappingPairs(self):
return self.getPairs()
# TODO: Missing computeExpressionIter1 here. For now it would require us to
# add lots of temporary variables for keys, which then becomes the tuple,
# but for as long as we don't have efficient forward propagation of these,
# we won't do that. Otherwise we loose execution order of values with them
# remaining as side effects. We could limit ourselves to cases where
# isMappingWithConstantStringKeys is true, or keys had no side effects, but
# that feels wasted effort as we are going to have full propagation.
def computeExpressionDrop(self, statement, constraint_collection):
from .NodeMakingHelpers import makeStatementOnlyNodesFromExpressions
expressions = []
for pair in self.getPairs():
expressions.append(pair.getValue())
expressions.append(pair.getKey())
result = makeStatementOnlyNodesFromExpressions(
expressions = expressions
)
return result, "new_statements", """\
Removed sequence creation for unused sequence."""
Nuitka-0.5.0.1/nuitka/nodes/LoopNodes.py 0000644 0001750 0001750 00000013520 12265264105 020223 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Loop nodes.
There are for and loop nodes, but both are reduced to loops with break/continue
statements for it. These reformulations require that optimization of loops has
to be very general, yet the node type for loop, becomes very simple.
"""
from .NodeBases import (
StatementChildrenHavingBase,
NodeBase
)
from nuitka.tree.Extractions import getVariablesWritten
class StatementLoop(StatementChildrenHavingBase):
kind = "STATEMENT_LOOP"
named_children = (
"frame",
)
def __init__(self, body, source_ref):
StatementChildrenHavingBase.__init__(
self,
values = {
"frame" : body
},
source_ref = source_ref
)
self.break_exception = False
self.continue_exception = False
getLoopBody = StatementChildrenHavingBase.childGetter("frame")
def markAsExceptionContinue(self):
self.continue_exception = True
def markAsExceptionBreak(self):
self.break_exception = True
def needsExceptionContinue(self):
return self.continue_exception
def needsExceptionBreak(self):
return self.break_exception
def computeStatement(self, constraint_collection):
loop_body = self.getLoopBody()
if loop_body is not None:
# Look ahead. what will be written.
variable_writes = getVariablesWritten( loop_body )
# Mark all variables as unknown that are written in the loop body,
# so it destroys the assumptions for loop turn around.
for variable, _variable_version in variable_writes:
constraint_collection.markActiveVariableAsUnknown(
variable = variable
)
result = loop_body.computeStatementsSequence(
constraint_collection = constraint_collection
)
# Might be changed.
if result is not loop_body:
loop_body.replaceWith( result )
loop_body = result
# Consider trailing "continue" statements, these have no effect, so we
# can remove them.
if loop_body is not None:
assert loop_body.isStatementsSequence()
statements = loop_body.getStatements()
assert statements # Cannot be empty
last_statement = statements[-1]
if last_statement.isStatementContinueLoop():
loop_body.removeStatement( last_statement )
statements = loop_body.getStatements()
if not statements:
loop_body.replaceWith( None )
loop_body = None
constraint_collection.signalChange(
"new_statements",
last_statement.getSourceReference(),
"Removed continue as last statement of loop."
)
# Consider leading "break" statements, they should be the only, and
# should lead to removing the whole loop statement. Trailing "break"
# statements could also be handled, but that would need to consider if
# there are other "break" statements too. Numbering loop exits is
# nothing we have yet.
if loop_body is not None:
assert loop_body.isStatementsSequence()
statements = loop_body.getStatements()
assert statements # Cannot be empty
if len( statements ) == 1 and statements[-1].isStatementBreakLoop():
return None, "new_statements", """\
Removed loop immediately broken."""
return self, None, None
def needsLineNumber(self):
# The loop itself cannot fail, the first statement will set the line
# number if necessary.
return False
class StatementContinueLoop(NodeBase):
kind = "STATEMENT_CONTINUE_LOOP"
def __init__(self, source_ref):
NodeBase.__init__( self, source_ref = source_ref )
self.exception_driven = False
def isStatementAborting(self):
return True
def markAsExceptionDriven(self):
self.exception_driven = True
def isExceptionDriven(self):
return self.exception_driven
def computeStatement(self, constraint_collection):
# This statement being aborting, will already tell everything. TODO: The
# fine difference that this jumps to loop start for sure, should be
# represented somehow one day.
return self, None, None
class StatementBreakLoop(NodeBase):
kind = "STATEMENT_BREAK_LOOP"
def __init__(self, source_ref):
NodeBase.__init__( self, source_ref = source_ref )
self.exception_driven = False
def isStatementAborting(self):
return True
def markAsExceptionDriven(self):
self.exception_driven = True
def isExceptionDriven(self):
return self.exception_driven
def computeStatement(self, constraint_collection):
# This statement being aborting, will already tell everything. TODO: The
# fine difference that this exits the loop for sure, should be
# represented somehow one day.
return self, None, None
Nuitka-0.5.0.1/nuitka/nodes/AttributeNodes.py 0000644 0001750 0001750 00000017275 12265264105 021270 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Attribute node
Knowing attributes of an object is very important, esp. when it comes to 'self' and
objects and classes.
There will be a method "computeExpressionAttribute" to aid predicting them.
"""
from .NodeBases import ExpressionChildrenHavingBase
class ExpressionAttributeLookup(ExpressionChildrenHavingBase):
kind = "EXPRESSION_ATTRIBUTE_LOOKUP"
named_children = ( "expression", )
def __init__(self, expression, attribute_name, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"expression" : expression
},
source_ref = source_ref
)
self.attribute_name = attribute_name
def getAttributeName(self):
return self.attribute_name
def setAttributeName(self, attribute_name):
self.attribute_name = attribute_name
def getDetails(self):
return { "attribute" : self.getAttributeName() }
def getDetail(self):
return "attribute %s from %s" % ( self.getAttributeName(), self.getLookupSource() )
getLookupSource = ExpressionChildrenHavingBase.childGetter( "expression" )
def makeCloneAt(self, source_ref):
return ExpressionAttributeLookup(
expression = self.getLookupSource().makeCloneAt( source_ref ),
attribute_name = self.getAttributeName(),
source_ref = source_ref
)
def computeExpression(self, constraint_collection):
lookup_source = self.getLookupSource()
if lookup_source.willRaiseException( BaseException ):
return lookup_source, "new_raise", "Attribute lookup source raises exception."
return lookup_source.computeExpressionAttribute(
lookup_node = self,
attribute_name = self.getAttributeName(),
constraint_collection = constraint_collection
)
def isKnownToBeIterable(self, count):
# TODO: Could be known.
return None
class ExpressionSpecialAttributeLookup(ExpressionAttributeLookup):
kind = "EXPRESSION_SPECIAL_ATTRIBUTE_LOOKUP"
# TODO: Special lookups should be treated somehow different.
def computeExpression(self, constraint_collection):
lookup_source = self.getLookupSource()
if lookup_source.willRaiseException( BaseException ):
return lookup_source, "new_raise", "Special attribute lookup source raises exception."
# TODO: Special lookups may reuse "computeExpressionAttribute"
return self, None, None
class ExpressionBuiltinGetattr(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_GETATTR"
named_children = ( "source", "attribute", "default" )
# Need to accept 'object' keyword argument, that is just the API of getattr,
# pylint: disable=W0622
def __init__(self, object, name, default, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"source" : object,
"attribute" : name,
"default" : default
},
source_ref = source_ref
)
getLookupSource = ExpressionChildrenHavingBase.childGetter( "source" )
getAttribute = ExpressionChildrenHavingBase.childGetter( "attribute" )
getDefault = ExpressionChildrenHavingBase.childGetter( "default" )
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
default = self.getDefault()
if default is None:
attribute = self.getAttribute()
attribute_name = attribute.getStringValue()
if attribute_name is not None:
source = self.getLookupSource()
# If source has sideeffects, it must be evaluated, before the lookup,
# meaning, a temporary variable should be assigned. For now, we give up in
# this case. TODO: Replace source with a temporary variable assignment as
# a side effect.
side_effects = source.extractSideEffects()
if not side_effects:
result = ExpressionAttributeLookup(
expression = source,
attribute_name = attribute_name,
source_ref = self.source_ref
)
from .NodeMakingHelpers import wrapExpressionWithNodeSideEffects
result = wrapExpressionWithNodeSideEffects(
new_node = result,
old_node = attribute
)
return (
result,
"new_expression",
"""Replaced call to built-in 'getattr' with constant \
attribute '%s' to mere attribute lookup""" % attribute_name
)
return self, None, None
class ExpressionBuiltinSetattr(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_SETATTR"
named_children = ( "source", "attribute", "value" )
# Need to accept 'object' keyword argument, that is just the API of setattr,
# pylint: disable=W0622
def __init__(self, object, name, value, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"source" : object,
"attribute" : name,
"value" : value
},
source_ref = source_ref
)
getLookupSource = ExpressionChildrenHavingBase.childGetter( "source" )
getAttribute = ExpressionChildrenHavingBase.childGetter( "attribute" )
getValue = ExpressionChildrenHavingBase.childGetter( "value" )
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
# Note: Might be possible to predict or downgrade to mere attribute set.
return self, None, None
class ExpressionBuiltinHasattr(ExpressionChildrenHavingBase):
kind = "EXPRESSION_BUILTIN_HASATTR"
named_children = ( "source", "attribute" )
# Need to accept object keyword argument, that is just the API of hasattr,
# pylint: disable=W0622
def __init__(self, object, name, source_ref):
ExpressionChildrenHavingBase.__init__(
self,
values = {
"source" : object,
"attribute" : name,
},
source_ref = source_ref
)
getLookupSource = ExpressionChildrenHavingBase.childGetter( "source" )
getAttribute = ExpressionChildrenHavingBase.childGetter( "attribute" )
def computeExpression(self, constraint_collection):
# Children can tell all we need to know, pylint: disable=W0613
# Note: Might be possible to predict or downgrade to mere attribute check.
return self, None, None
def mayProvideReference(self):
# Dedicated code returns "True" or "False" only, which requires no reference.
return False
Nuitka-0.5.0.1/nuitka/nodes/ModuleNodes.py 0000644 0001750 0001750 00000030517 12265270513 020544 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Module/Package nodes
The top of the tree. Packages are also modules. Modules are what hold a program
together and cross-module optimizations are the most difficult to tackle.
"""
from .NodeBases import (
ClosureGiverNodeBase,
ChildrenHavingMixin,
NodeBase
)
from .IndicatorMixins import MarkContainsTryExceptIndicator
from nuitka.SourceCodeReferences import SourceCodeReference
from nuitka.nodes.FutureSpecs import FutureSpec
from nuitka import Variables, Importing, Utils
from nuitka.oset import OrderedSet
import re
class PythonModuleMixin:
def __init__(self, name, package_name):
assert type(name) is str, type(name)
assert "." not in name, name
assert package_name is None or \
(type( package_name ) is str and package_name != "")
self.name = name
self.package_name = package_name
self.package = None
def getName(self):
return self.name
def getPackage(self):
return self.package_name
def getFullName(self):
if self.package_name:
return self.package_name + "." + self.getName()
else:
return self.getName()
def isMainModule(self):
return False
def isInternalModule(self):
return False
def attemptRecursion(self):
# Make sure the package is recursed to.
from nuitka.tree import Recursion
from nuitka import Importing
# Return the list of newly added modules.
result = []
if self.package_name is not None and self.package is None:
package_package, _package_module_name, package_filename = \
Importing.findModule(
source_ref = self.getSourceReference(),
module_name = self.package_name,
parent_package = None,
level = 1,
warn = Utils.python_version < 330
)
# TODO: Temporary, if we can't find the package for Python3.3 that
# is semi-OK, maybe.
if Utils.python_version >= 330 and not package_filename:
return []
imported_module, is_added = Recursion.recurseTo(
module_package = package_package,
module_filename = package_filename,
module_relpath = Utils.relpath(package_filename),
module_kind = "py",
reason = "Containing package of recursed module.",
)
self.package = imported_module
if is_added:
result.append(imported_module)
if self.package:
from nuitka.ModuleRegistry import addUsedModule
addUsedModule(self.package)
# print "Recursed to package", self.package_name
result.extend(self.package.attemptRecursion())
return result
def checkModuleBody(value):
assert value is None or value.isStatementsSequence()
return value
class PythonModule(PythonModuleMixin, ChildrenHavingMixin,
ClosureGiverNodeBase, MarkContainsTryExceptIndicator):
""" Module
The module is the only possible root of a tree. When there are many
modules they form a forrest.
"""
kind = "PYTHON_MODULE"
named_children = (
"body",
)
checkers = {
"body": checkModuleBody
}
def __init__(self, name, package_name, source_ref):
ClosureGiverNodeBase.__init__(
self,
name = name,
code_prefix = "module",
source_ref = source_ref
)
ChildrenHavingMixin.__init__(
self,
values = {},
)
MarkContainsTryExceptIndicator.__init__( self )
PythonModuleMixin.__init__(
self,
name = name,
package_name = package_name
)
self.variables = set()
# The list functions contained in that module.
self.functions = OrderedSet()
self.active_functions = OrderedSet()
# SSA trace based information about the module.
self.collection = None
def getDetails(self):
return {
"filename" : self.source_ref.getFilename(),
"package" : self.package_name,
"name" : self.name
}
def asXml(self):
# The class is new style, false alarm: pylint: disable=E1002
result = super( PythonModule, self ).asXml()
for function_body in self.functions:
result.append( function_body.asXml() )
return result
getBody = ChildrenHavingMixin.childGetter("body")
setBody = ChildrenHavingMixin.childSetter("body")
def isPythonModule(self):
return True
def getParent(self):
assert False
def getParentVariableProvider(self):
return None
def getVariables(self):
return self.variables
def getFilename(self):
return self.source_ref.getFilename()
def getVariableForAssignment(self, variable_name):
result = self.getProvidedVariable(variable_name)
return result.makeReference(self)
def getVariableForReference(self, variable_name):
result = self.getProvidedVariable(variable_name)
return result.makeReference(self)
def getVariableForClosure(self, variable_name):
return self.getProvidedVariable(
variable_name = variable_name
)
def createProvidedVariable(self, variable_name):
result = Variables.ModuleVariable(
module = self,
variable_name = variable_name
)
assert result not in self.variables
self.variables.add(result)
return result
def isEarlyClosure(self):
# Modules should immediately closure variables on use.
# pylint: disable=R0201
return True
def getCodeName(self):
def r(match):
c = match.group()
if c == '.':
return "$"
else:
return "$$%d$" % ord(c)
return "module_" + \
"".join(re.sub("[^a-zA-Z0-9_]", r ,c) for c in self.getFullName())
def addFunction(self, function_body):
assert function_body not in self.functions
self.functions.add( function_body )
def getFunctions(self):
return self.functions
def startTraversal(self):
self.active_functions = OrderedSet()
def addUsedFunction(self, function_body):
assert function_body in self.functions
assert function_body.isExpressionFunctionBody()
if function_body not in self.active_functions:
self.active_functions.add(function_body)
def getUsedFunctions(self):
return self.active_functions
def getOutputFilename(self):
main_filename = self.getFilename()
if main_filename.endswith(".py"):
return main_filename[:-3]
else:
return main_filename
class SingleCreationMixin:
created = set()
def __init__(self):
assert self.__class__ not in self.created
self.created.add( self.__class__ )
class PythonMainModule(PythonModule, SingleCreationMixin):
kind = "PYTHON_MAIN_MODULE"
def __init__(self, main_added, source_ref):
PythonModule.__init__(
self,
name = "__main__",
package_name = None,
source_ref = source_ref
)
SingleCreationMixin.__init__( self )
self.main_added = main_added
def isMainModule(self):
return True
def getOutputFilename(self):
if self.main_added:
return Utils.dirname(self.getFilename())
else:
return PythonModule.getOutputFilename(self)
class PythonInternalModule(PythonModule, SingleCreationMixin):
kind = "PYTHON_INTERNAL_MODULE"
def __init__(self):
PythonModule.__init__(
self,
name = "__internal__",
package_name = None,
source_ref = SourceCodeReference.fromFilenameAndLine(
filename = "internal",
line = 0,
future_spec = FutureSpec(),
inside_exec = False
)
)
SingleCreationMixin.__init__( self )
def isInternalModule(self):
return True
def getOutputFilename(self):
return "__internal"
class PythonPackage(PythonModule):
kind = "PYTHON_PACKAGE"
def __init__(self, name, package_name, source_ref):
assert name
PythonModule.__init__(
self,
name = name,
package_name = package_name,
source_ref = source_ref
)
def getOutputFilename(self):
return Utils.dirname( self.getFilename() )
class PythonShlibModule(PythonModuleMixin, NodeBase):
kind = "PYTHON_SHLIB_MODULE"
def __init__(self, name, package_name, source_ref):
NodeBase.__init__(
self,
source_ref = source_ref
)
PythonModuleMixin.__init__(
self,
name = name,
package_name = package_name
)
def getDetails(self):
return {
"name" : self.name,
"package_name" : self.package_name
}
def getFilename(self):
return self.getSourceReference().getFilename()
def startTraversal(self):
pass
def getImplicitImports(self):
if self.getFullName() == "PyQt4.QtCore":
return (
("atexit", None),
("sip", None)
)
elif self.getFullName() == "lxml.etree":
return (
("gzip", None),
)
else:
return ()
def considerImplicitImports(self, signal_change):
for module_name, module_package in self.getImplicitImports():
_module_package, _module_name, module_filename = \
Importing.findModule(
source_ref = self.source_ref,
module_name = module_name,
parent_package = module_package,
level = -1,
warn = True
)
if Utils.isDir(module_filename):
module_kind = "py"
elif module_filename.endswith(".py"):
module_kind = "py"
elif module_filename.endswith(".so"):
module_kind = "shlib"
elif module_filename.endswith(".pyd"):
module_kind = "shlib"
else:
assert False
from nuitka.tree import Recursion
decision, reason = Recursion.decideRecursion(
module_filename = module_filename,
module_name = module_name,
module_package = module_package,
module_kind = module_kind
)
assert decision, reason
if decision:
module_relpath = Utils.relpath(module_filename)
imported_module, added_flag = Recursion.recurseTo(
module_package = module_package,
module_filename = module_filename,
module_relpath = module_relpath,
module_kind = module_kind,
reason = reason
)
from nuitka.ModuleRegistry import addUsedModule
addUsedModule(imported_module)
if added_flag:
signal_change(
"new_code",
imported_module.getSourceReference(),
"Recursed to module."
)
Nuitka-0.5.0.1/nuitka/ModuleRegistry.py 0000644 0001750 0001750 00000004365 12265264105 020176 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" This to keep track of used modules.
There is a set of root modules, which are user specified, and must be
processed. As they go, they add more modules to active modules list
and move done modules out of it.
That process can be restarted and modules will be fetched back from
the existing set of modules.
"""
from nuitka.oset import OrderedSet
root_modules = OrderedSet()
def addRootModule(module):
root_modules.add(module)
def getRootModules():
return root_modules
active_modules = OrderedSet()
done_modules = OrderedSet()
def startTraversal():
# Using global here, as this is really a singleton, in the form of a module,
# pylint: disable=W0603
global active_modules, done_modules
active_modules = OrderedSet(root_modules)
done_modules = OrderedSet()
for active_module in active_modules:
active_module.startTraversal()
def addUsedModule(module):
if module not in done_modules and module not in active_modules:
active_modules.add(module)
module.startTraversal()
def nextModule():
if active_modules:
result = active_modules.pop()
done_modules.add(result)
return result
else:
return None
def remainingCount():
return len(active_modules)
def getDoneModules():
return list(done_modules)
def getDoneUserModules():
return [
module
for module in
done_modules
if not module.isInternalModule()
if not module.isMainModule()
]
Nuitka-0.5.0.1/nuitka/SourceCodeReferences.py 0000644 0001750 0001750 00000006526 12265264105 021256 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Source code reference record.
All the information to lookup line and file of a code location, together with
the future flags in use there.
"""
class SourceCodeReference:
@classmethod
def fromFilenameAndLine(cls, filename, line, future_spec, inside_exec):
result = cls()
result.filename = filename
result.line = line
result.future_spec = future_spec
result.inside_exec = inside_exec
return result
def __init__(self):
self.line = None
self.filename = None
self.future_spec = None
self.inside_exec = False
self.set_line = True
def __repr__(self):
return "<%s to %s:%s>" % ( self.__class__.__name__, self.filename, self.line )
def clone(self, line):
result = SourceCodeReference.fromFilenameAndLine(
filename = self.filename,
line = line,
future_spec = self.future_spec,
inside_exec = self.inside_exec
)
result.set_line = self.set_line
return result
def atLineNumber(self, line):
assert int( line ) == line
return self.clone( line )
def getLineNumber(self):
return self.line
def getFilename(self):
return self.filename
def getFutureSpec(self):
return self.future_spec
def getAsString(self):
return "%s:%s" % ( self.filename, self.line )
def getExecReference(self, value):
if self.inside_exec != value:
return self.__class__.fromFilenameAndLine(
filename = self.filename,
line = self.line,
future_spec = self.future_spec.clone(),
inside_exec = value
)
else:
return self
def isExecReference(self):
return self.inside_exec
def shallSetCurrentLine(self):
return self.set_line
def __cmp__(self, other):
if other is None:
return -1
assert isinstance( other, SourceCodeReference ), other
result = cmp( self.filename, other.filename)
if result == 0:
result = cmp( self.line, other.line )
return result
def atInternal(self):
if self.set_line:
result = self.clone( self.line )
result.set_line = False
return result
else:
return self
def fromFilename(filename, future_spec):
return SourceCodeReference.fromFilenameAndLine(
filename = filename,
line = 1,
future_spec = future_spec,
inside_exec = False
)
Nuitka-0.5.0.1/nuitka/Tracing.py 0000644 0001750 0001750 00000002662 12265264105 016605 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Outputs to the user.
Printing with intends or plain, mostly a compensation for the print strageness. I want to
avoid "from __future__ import print_function" in every file out there, which makes adding
another print rather tedious. This should cover all calls/uses of "print" we have to do,
and the make it easy to simply to "print for_debug" without much hassle (braces).
"""
from __future__ import print_function
import sys
def printIndented(level, *what):
print( " " * level, *what )
def printSeparator(level = 0):
print( " " * level, "*" * 10 )
def printLine(*what):
print( *what )
def printError(message):
print( message, file=sys.stderr )
Nuitka-0.5.0.1/nuitka/codegen/ 0000755 0001750 0001750 00000000000 12265271051 016240 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/codegen/CodeTemplates.py 0000644 0001750 0001750 00000006073 12265264105 021353 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code templates one stop access. """
# Wildcard imports are here to centralize the templates for access through one
# module name, this one, they are not used here though.
# pylint: disable=W0401,W0614
from .templates.CodeTemplatesMain import *
from .templates.CodeTemplatesConstants import *
from .templates.CodeTemplatesFunction import *
from .templates.CodeTemplatesGeneratorFunction import *
from .templates.CodeTemplatesParameterParsing import *
from .templates.CodeTemplatesExceptions import *
from .templates.CodeTemplatesImporting import *
from .templates.CodeTemplatesPrinting import *
from .templates.CodeTemplatesBranches import *
from .templates.CodeTemplatesTuples import *
from .templates.CodeTemplatesLists import *
from .templates.CodeTemplatesDicts import *
from .templates.CodeTemplatesCalls import *
from .templates.CodeTemplatesLoops import *
from .templates.CodeTemplatesSets import *
from .templates.CodeTemplatesExecEval import *
def enableDebug():
templates = dict(globals())
class TemplateWrapper:
""" Wrapper around templates.
To better trace and control template usage.
"""
def __init__(self, name, value):
self.name = name
self.value = value
def __str__(self):
return self.value
def __mod__(self, other):
assert type( other ) is dict, self.name
for key in other.keys():
if "%%(%s)" % key not in self.value:
from logging import warning
warning(
"Extra value '%s' provided to template '%s'.",
key,
self.name
)
return self.value % other
def split(self, sep):
return self.value.split( sep )
from nuitka.__past__ import iterItems
for template_name, template_value in iterItems(templates):
# Ignore internal attribute like "__name__" that the module will also
# have of course.
if template_name.startswith("_"):
continue
if type(template_value) is str:
globals()[template_name] = TemplateWrapper(
template_name,
template_value
)
from nuitka.Options import isDebug
if isDebug():
enableDebug()
Nuitka-0.5.0.1/nuitka/codegen/Indentation.py 0000644 0001750 0001750 00000002717 12265264105 021077 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Indentation of code.
Language independent, the amount of the spaces is not configurable, as it needs
to be the same as in templates.
"""
def _indentedCode(codes, count):
return "\n".join(
" " * count + line if (line and not line.startswith( "#" )) else line for line in codes
)
def indented(codes, level = 1, vert_block = False):
if type( codes ) is str:
codes = codes.split( "\n" )
if vert_block and codes != [ "" ]:
codes.insert( 0, "" )
codes.append( "" )
return _indentedCode( codes, level * 4 )
def getBlockCode(codes):
if type( codes ) is str:
assert codes == codes.rstrip(), codes
return "{\n%s\n}" % indented( codes )
Nuitka-0.5.0.1/nuitka/codegen/BlobCodes.py 0000644 0001750 0001750 00000003033 12265264105 020447 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Blob codes for storing binary data semi-efficiently.
This module offers means to store and encode binary blobs in C++ semi
efficiently. The "StreamData" class is used in two places, for constants
and for freezing of bytecode.
"""
class StreamData:
def __init__(self):
self.stream_data = bytes()
def getStreamDataCode(self, value, fixed_size = False):
offset = self.stream_data.find(value)
if offset == -1:
offset = len(self.stream_data)
self.stream_data += value
if fixed_size:
return "&stream_data[ %d ]" % offset
else:
return "&stream_data[ %d ], %d" % (
offset,
len(value)
)
def getBytes(self):
return self.stream_data
Nuitka-0.5.0.1/nuitka/codegen/CallCodes.py 0000644 0001750 0001750 00000012024 12265270763 020453 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code generation for calls.
The different kinds of calls get dedicated code. Most notable, calls with
only positional arguments, are attempted through helpers that might be
able to execute them without creating the argument dictionary at all.
"""
from . import CodeTemplates
from .Identifiers import Identifier
def getCallCodeNoArgs(called_identifier):
return Identifier(
"CALL_FUNCTION_NO_ARGS( %(function)s )" % {
"function" : called_identifier.getCodeTemporaryRef(),
},
1
)
# Outside helper code relies on some quick call to be present.
quick_calls_used = set( [ 1, 2, 3 ] )
def getCallCodePosArgsQuick( context, order_relevance, called_identifier,
arguments ):
arg_size = len( arguments )
quick_calls_used.add( arg_size )
from .TupleCodes import addMakeTupleUse
addMakeTupleUse( arg_size )
from .OrderedEvaluation import getOrderRelevanceEnforcedArgsCode
return getOrderRelevanceEnforcedArgsCode(
helper = "CALL_FUNCTION_WITH_ARGS%d" % arg_size,
export_ref = 0,
ref_count = 1,
tmp_scope = "call",
order_relevance = order_relevance,
args = [ called_identifier ] + arguments,
context = context
)
def getCallCodePosArgs( context, order_relevance, called_identifier,
argument_tuple ):
from .OrderedEvaluation import getOrderRelevanceEnforcedArgsCode
return getOrderRelevanceEnforcedArgsCode(
helper = "CALL_FUNCTION_WITH_POSARGS",
export_ref = 0,
ref_count = 1,
tmp_scope = "call",
order_relevance = order_relevance,
args = ( called_identifier, argument_tuple ),
context = context
)
def getCallCodeKeywordArgs( context, order_relevance, called_identifier,
argument_dictionary ):
from .OrderedEvaluation import getOrderRelevanceEnforcedArgsCode
return getOrderRelevanceEnforcedArgsCode(
helper = "CALL_FUNCTION_WITH_KEYARGS",
export_ref = 0,
ref_count = 1,
tmp_scope = "call",
order_relevance = order_relevance,
args = ( called_identifier, argument_dictionary ),
context = context
)
def getCallCodePosKeywordArgs( context, order_relevance, called_identifier,
argument_tuple, argument_dictionary ):
from .OrderedEvaluation import getOrderRelevanceEnforcedArgsCode
return getOrderRelevanceEnforcedArgsCode(
helper = "CALL_FUNCTION",
export_ref = 0,
ref_count = 1,
tmp_scope = "call",
order_relevance = order_relevance,
args = ( called_identifier, argument_tuple,
argument_dictionary ),
context = context
)
def getCallsDecls():
result = []
for quick_call_used in sorted( quick_calls_used ):
args_decl = [
"PyObject *arg%d" % d
for d in range( quick_call_used )
]
result.append(
CodeTemplates.template_call_function_with_args_decl % {
"args_decl" : ", ".join( args_decl ),
"args_count" : quick_call_used
}
)
return CodeTemplates.template_header_guard % {
"header_guard_name" : "__NUITKA_CALLS_H__",
"header_body" : "\n".join( result )
}
def getCallsCode():
result = []
result.append(
CodeTemplates.template_helper_impl_decl % {}
)
result.append(
CodeTemplates.template_call_cpython_function_fast_impl % {}
)
for quick_call_used in sorted( quick_calls_used ):
args_decl = [
"PyObject *arg%d" % d
for d in range( 1, quick_call_used + 1 )
]
args_list = [
"arg%d" % d
for d in range( 1, quick_call_used + 1 )
]
result.append(
CodeTemplates.template_call_function_with_args_impl % {
"args_decl" : ", ".join( args_decl ),
"args_list" : ", ".join( args_list ),
"args_count" : quick_call_used
}
)
return "\n".join( result )
Nuitka-0.5.0.1/nuitka/codegen/Pickling.py 0000644 0001750 0001750 00000004675 12265264105 020370 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Module to hide the complexity of using pickle.
It should be simple, but it is not yet. Not all the pickle modules are well behaved.
"""
from nuitka import Constants, Utils
# pylint: disable=W0622
from ..__past__ import unicode
# pylint: enable=W0622
# Work around for CPython 3.x removal of cpickle.
try:
import cPickle as cpickle
except ImportError:
# False alarm, no double import at all, pylint: disable=W0404
import pickle as cpickle
import pickletools
from logging import warning
if Utils.python_version >= 300:
# Python3: The protocol 3 adds support for bytes type.
# instead.
pickle_protocol = 3
else:
pickle_protocol = 2
def getStreamedConstant(constant_value):
# Note: The marshal module cannot persist all unicode strings and
# therefore cannot be used. Instead we use pickle.
try:
saved = cpickle.dumps(
constant_value,
protocol = 0 if type( constant_value ) is unicode else pickle_protocol
)
except TypeError:
warning( "Problem with persisting constant '%r'." % constant_value )
raise
saved = pickletools.optimize( saved )
# Check that the constant is restored correctly.
try:
restored = cpickle.loads(
saved
)
except:
warning( "Problem with persisting constant '%r'." % constant_value )
raise
if not Constants.compareConstants( restored, constant_value ):
raise AssertionError(
"Streaming of constant changed value",
constant_value,
"!=",
restored,
"types:",
type( constant_value ),
type( restored )
)
return saved
Nuitka-0.5.0.1/nuitka/codegen/ListCodes.py 0000644 0001750 0001750 00000005103 12265264105 020504 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code generation for lists.
Right now only the creation is done here. But more should be added later on.
"""
from . import CodeTemplates
make_lists_used = set( range( 0, 1 ) )
def addMakeListUse(value):
assert type( value ) is int
make_lists_used.add( value )
def getListCreationCode(context, order_relevance, element_identifiers):
from .OrderedEvaluation import getOrderRelevanceEnforcedArgsCode
args_length = len( element_identifiers )
addMakeListUse( args_length )
return getOrderRelevanceEnforcedArgsCode(
helper = "MAKE_LIST%d" % args_length,
export_ref = 1,
ref_count = 1,
tmp_scope = "make_list",
order_relevance = order_relevance,
args = element_identifiers,
context = context
)
def getMakeListsCode():
make_lists_codes = []
for arg_count in sorted( make_lists_used ):
add_elements_code = []
for arg_index in range( arg_count ):
add_elements_code.append(
CodeTemplates.template_add_list_element_code % {
"list_index" : arg_index,
"list_value" : "element%d" % arg_index
}
)
make_lists_codes.append(
CodeTemplates.template_make_list_function % {
"argument_count" : arg_count,
"argument_decl" : ", ".join(
"PyObject *element%d" % arg_index
for arg_index in
range( arg_count )
),
"add_elements_code" : "\n".join( add_elements_code ),
}
)
return CodeTemplates.template_header_guard % {
"header_guard_name" : "__NUITKA_LISTS_H__",
"header_body" : "\n".join( make_lists_codes )
}
Nuitka-0.5.0.1/nuitka/codegen/templates/ 0000755 0001750 0001750 00000000000 12265271051 020236 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesMain.py 0000644 0001750 0001750 00000034657 12265264105 024167 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Main module code templates
This for the main program in case of executables, the module templates and
stuff related to importing, and of course the generated code license.
"""
global_copyright = """\
// Generated code for Python source for module '%(name)s'
// created by Nuitka version %(version)s
// This code is in part copyright 2013 Kay Hayen.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
"""
template_metapath_loader_compiled_module_entry = """\
{ (char *)"%(module_name)s", MOD_INIT_NAME( %(module_identifier)s ), NUITKA_COMPILED_MODULE },"""
template_metapath_loader_shlib_module_entry = """\
{ (char *)"%(module_name)s", NULL, NUITKA_SHLIB_MODULE },"""
main_program = """\
// The main program for C++. It needs to prepare the interpreter and then
// calls the initialization code of the __main__ module.
#include "structseq.h"
#ifdef _NUITKA_WINMAIN_ENTRY_POINT
int __stdcall WinMain( HINSTANCE hInstance, HINSTANCE hPrevInstance, char* lpCmdLine, int nCmdShow )
{
int argc = __argc;
char** argv = __argv;
#else
int main( int argc, char *argv[] )
{
#endif
#ifdef _NUITKA_STANDALONE
prepareStandaloneEnvironment();
#endif
// Initialize Python environment.
Py_DebugFlag = %(python_sysflag_debug)d;
#if %(python_sysflag_py3k_warning)d
Py_Py3kWarningFlag = %(python_sysflag_py3k_warning)d;
#endif
#if %(python_sysflag_division_warning)d
Py_DivisionWarningFlag =
#if %(python_sysflag_py3k_warning)d
Py_Py3kWarningFlag ||
#endif
%(python_sysflag_division_warning)d;
#endif
Py_InspectFlag = %(python_sysflag_inspect)d;
Py_InteractiveFlag = %(python_sysflag_interactive)d;
Py_OptimizeFlag = %(python_sysflag_optimize)d;
Py_DontWriteBytecodeFlag = %(python_sysflag_dont_write_bytecode)d;
Py_NoUserSiteDirectory = %(python_sysflag_no_user_site)d;
Py_IgnoreEnvironmentFlag = %(python_sysflag_ignore_environment)d;
#if %(python_sysflag_tabcheck)d
Py_TabcheckFlag = %(python_sysflag_tabcheck)d;
#endif
Py_VerboseFlag = %(python_sysflag_verbose)d;
#if %(python_sysflag_unicode)d
Py_UnicodeFlag = %(python_sysflag_unicode)d;
#endif
Py_BytesWarningFlag = %(python_sysflag_bytes_warning)d;
#if %(python_sysflag_hash_randomization)d
Py_HashRandomizationFlag = %(python_sysflag_hash_randomization)d;
#endif
// We want to import the site module, but only after we finished our own
// setup. The site module import will be the first thing, the main module
// does.
Py_NoSiteFlag = 1;
// Initialize the embedded CPython interpreter.
setCommandLineParameters( argc, argv, true );
Py_Initialize();
// Lie about it, believe it or not, there are "site" files, that check
// against later imports, see below.
Py_NoSiteFlag = %(python_sysflag_no_site)d;
// Set the command line parameters for run time usage.
setCommandLineParameters( argc, argv, false );
// Initialize the constant values used.
_initBuiltinModule();
_initConstants();
_initBuiltinOriginalValues();
// Revert the wrong sys.flags value, it's used by "site" on at least Debian
// for Python3.3, more uses may exist.
#if %(python_sysflag_no_site)d == 0
#if PYTHON_VERSION >= 330
PyStructSequence_SetItem( PySys_GetObject( "flags" ), 6, const_int_0 );
#elif PYTHON_VERSION >= 320
PyStructSequence_SetItem( PySys_GetObject( "flags" ), 7, const_int_0 );
#elif PYTHON_VERSION >= 260
PyStructSequence_SET_ITEM( PySys_GetObject( (char *)"flags" ), 9, const_int_0 );
#endif
#endif
// Initialize the compiled types of Nuitka.
PyType_Ready( &Nuitka_Generator_Type );
PyType_Ready( &Nuitka_Function_Type );
PyType_Ready( &Nuitka_Method_Type );
PyType_Ready( &Nuitka_Frame_Type );
#if PYTHON_VERSION < 300
initSlotCompare();
#endif
enhancePythonTypes();
// Set the sys.executable path to the original Python executable on Linux
// or to python.exe on Windows.
PySys_SetObject(
(char *)"executable",
%(sys_executable)s
);
patchBuiltinModule();
patchTypeComparison();
// Allow to override the ticker value, to remove checks for threads in
// CPython core from impact on benchmarks.
char const *ticker_value = getenv( "NUITKA_TICKER" );
if ( ticker_value != NULL )
{
_Py_Ticker = atoi( ticker_value );
assert ( _Py_Ticker >= 20 );
}
#if _NUITKA_STANDALONE
setEarlyFrozenModulesFileAttribute();
#endif
// Execute the "__main__" module init function.
MOD_INIT_NAME( __main__ )();
if ( ERROR_OCCURED() )
{
// Cleanup code may need a frame, so put one back.
PyThreadState_GET()->frame = MAKE_FRAME( %(code_identifier)s, module___main__ );
PyErr_PrintEx( 0 );
Py_Exit( 1 );
}
else
{
Py_Exit( 0 );
}
}
"""
module_header_template = """\
#include
MOD_INIT_DECL( %(module_identifier)s );
extern PyObject *module_%(module_identifier)s;
extern PyDictObject *moduledict_%(module_identifier)s;
// Declarations from this module to other modules if any.
%(extra_declarations)s
"""
module_body_template = """
#include "nuitka/prelude.hpp"
#include "__modules.hpp"
#include "__constants.hpp"
#include "__helpers.hpp"
// The _module_%(module_identifier)s is a Python object pointer of module type.
// Note: For full compatability with CPython, every module variable access
// needs to go through it except for cases where the module cannot possibly
// have changed in the mean time.
PyObject *module_%(module_identifier)s;
PyDictObject *moduledict_%(module_identifier)s;
NUITKA_MAY_BE_UNUSED static PyObject *GET_MODULE_VALUE0( PyObject *var_name )
{
// For module variable values, need to lookup in module dictionary or in
// built-in dictionary.
PyObject *result = GET_STRING_DICT_VALUE( moduledict_%(module_identifier)s, (Nuitka_StringObject *)var_name );
if (likely( result != NULL ))
{
assertObject( result );
return result;
}
result = GET_STRING_DICT_VALUE( dict_builtin, (Nuitka_StringObject *)var_name );
if (likely( result != NULL ))
{
assertObject( result );
return result;
}
PyErr_Format( PyExc_NameError, "global name '%%s' is not defined", Nuitka_String_AsString(var_name ));
throw PythonException();
}
NUITKA_MAY_BE_UNUSED static PyObject *GET_MODULE_VALUE1( PyObject *var_name )
{
return INCREASE_REFCOUNT( GET_MODULE_VALUE0( var_name ) );
}
NUITKA_MAY_BE_UNUSED void static DEL_MODULE_VALUE( PyObject *var_name, bool tolerant )
{
int status = PyDict_DelItem( (PyObject *)moduledict_%(module_identifier)s, var_name );
if (unlikely( status == -1 && tolerant == false ))
{
PyErr_Format(
PyExc_NameError,
"global name '%%s' is not defined",
Nuitka_String_AsString( var_name )
);
throw PythonException();
}
}
NUITKA_MAY_BE_UNUSED static PyObject *GET_LOCALS_OR_MODULE_VALUE0( PyObject *locals_dict, PyObject *var_name )
{
PyObject *result = PyDict_GetItem( locals_dict, var_name );
if ( result != NULL )
{
return result;
}
else
{
return GET_MODULE_VALUE0( var_name );
}
}
NUITKA_MAY_BE_UNUSED static PyObject *GET_LOCALS_OR_MODULE_VALUE1( PyObject *locals_dict, PyObject *var_name )
{
PyObject *result = PyDict_GetItem( locals_dict, var_name );
if ( result != NULL )
{
return INCREASE_REFCOUNT( result );
}
else
{
return GET_MODULE_VALUE1( var_name );
}
}
// The module function declarations.
%(module_functions_decl)s
// The module function definitions.
%(module_functions_code)s
#if PYTHON_VERSION >= 300
static struct PyModuleDef mdef_%(module_identifier)s =
{
PyModuleDef_HEAD_INIT,
"%(module_name)s", /* m_name */
NULL, /* m_doc */
-1, /* m_size */
NULL, /* m_methods */
NULL, /* m_reload */
NULL, /* m_traverse */
NULL, /* m_clear */
NULL, /* m_free */
};
#endif
#define _MODULE_UNFREEZER %(use_unfreezer)d
#if _MODULE_UNFREEZER
#include "nuitka/unfreezing.hpp"
// Table for lookup to find "frozen" modules or DLLs, i.e. the ones included in
// or along this binary.
static struct Nuitka_MetaPathBasedLoaderEntry meta_path_loader_entries[] =
{
%(metapath_loader_inittab)s
{ NULL, NULL, 0 }
};
#endif
// The exported interface to CPython. On import of the module, this function
// gets called. It has to have an exact function name, in cases it's a shared
// library export. This is hidden behind the MOD_INIT_DECL.
MOD_INIT_DECL( %(module_identifier)s )
{
#if defined(_NUITKA_EXE) || PYTHON_VERSION >= 300
static bool _init_done = false;
// Packages can be imported recursively in deep executables.
if ( _init_done )
{
return MOD_RETURN_VALUE( module_%(module_identifier)s );
}
else
{
_init_done = true;
}
#endif
#ifdef _NUITKA_MODULE
// In case of a stand alone extension module, need to call initialization
// the init here because that's the first and only time we are going to get
// called here.
// Initialize the constant values used.
_initBuiltinModule();
_initConstants();
// Initialize the compiled types of Nuitka.
PyType_Ready( &Nuitka_Generator_Type );
PyType_Ready( &Nuitka_Function_Type );
PyType_Ready( &Nuitka_Method_Type );
PyType_Ready( &Nuitka_Frame_Type );
#if PYTHON_VERSION < 300
initSlotCompare();
#endif
patchBuiltinModule();
patchTypeComparison();
#endif
#if _MODULE_UNFREEZER
registerMetaPathBasedUnfreezer( meta_path_loader_entries );
#endif
// puts( "in init%(module_identifier)s" );
// Create the module object first. There are no methods initially, all are
// added dynamically in actual code only. Also no "__doc__" is initially
// set at this time, as it could not contain NUL characters this way, they
// are instead set in early module code. No "self" for modules, we have no
// use for it.
#if PYTHON_VERSION < 300
module_%(module_identifier)s = Py_InitModule4(
"%(module_name)s", // Module Name
NULL, // No methods initially, all are added
// dynamically in actual module code only.
NULL, // No __doc__ is initially set, as it could
// not contain NUL this way, added early in
// actual code.
NULL, // No self for modules, we don't use it.
PYTHON_API_VERSION
);
#else
module_%(module_identifier)s = PyModule_Create( &mdef_%(module_identifier)s );
#endif
moduledict_%(module_identifier)s = (PyDictObject *)((PyModuleObject *)module_%(module_identifier)s)->md_dict;
assertObject( module_%(module_identifier)s );
// Seems to work for Python2.7 out of the box, but for Python3, the module
// doesn't automatically enter "sys.modules", so do it manually.
#if PYTHON_VERSION >= 300
{
int r = PyObject_SetItem( PySys_GetObject( (char *)"modules" ), %(module_name_obj)s, module_%(module_identifier)s );
assert( r != -1 );
}
#endif
// For deep importing of a module we need to have "__builtins__", so we set
// it ourselves in the same way than CPython does. Note: This must be done
// before the frame object is allocated, or else it may fail.
PyObject *module_dict = PyModule_GetDict( module_%(module_identifier)s );
if ( PyDict_GetItem( module_dict, const_str_plain___builtins__ ) == NULL )
{
PyObject *value = ( PyObject *)module_builtin;
#ifdef _NUITKA_EXE
if ( module_%(module_identifier)s != module___main__ )
{
#endif
value = PyModule_GetDict( value );
#ifdef _NUITKA_EXE
}
#endif
#ifndef __NUITKA_NO_ASSERT__
int res =
#endif
PyDict_SetItem( module_dict, const_str_plain___builtins__, value );
assert( res == 0 );
}
#if PYTHON_VERSION >= 330
#if _MODULE_UNFREEZER
PyDict_SetItem( module_dict, const_str_plain___loader__, metapath_based_loader );
#else
PyDict_SetItem( module_dict, const_str_plain___loader__, Py_None );
#endif
#endif
// Temp variables if any
%(temps_decl)s
// Module code
%(module_code)s
return MOD_RETURN_VALUE( module_%(module_identifier)s );
}
"""
template_helper_impl_decl = """\
// This file contains helper functions that are automatically created from
// templates.
#include "nuitka/prelude.hpp"
"""
template_header_guard = """\
#ifndef %(header_guard_name)s
#define %(header_guard_name)s
%(header_body)s
#endif
"""
template_frozen_modules = """\
// This provides the frozen (precompiled bytecode) files that are included if
// any.
#include
// Blob from which modules are unstreamed.
extern "C" const unsigned char constant_bin[];
#define stream_data constant_bin
// These modules should be loaded as bytecode. They must e.g. be loadable
// during "Py_Initialize" already, or for irrelevance, they are only included
// in this un-optimized form. These are not compiled by Nuitka, and therefore
// are not accelerated at all, merely bundled with the binary or module, so
// that Python library can start out.
struct _frozen Embedded_FrozenModules[] =
{
%(frozen_modules)s
{ NULL, NULL, 0 }
};
"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesExceptions.py 0000644 0001750 0001750 00000012136 12265264105 025410 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Templates for raising exceptions, making assertions, and try/finally
construct.
"""
try_except_template = """\
try
{
%(tried_code)s
}
catch ( PythonException &_exception )
{
if ( !_exception.hasTraceback() )
{
_exception.setTraceback( %(tb_making)s );
}
else
{
_exception.addTraceback( frame_guard.getFrame0() );
}
frame_guard.preserveExistingException();
#if PYTHON_VERSION >= 300
ExceptionRestorer%(guard_class)s restorer( &frame_guard );
#endif
_exception.toExceptionHandler();
%(exception_code)s
}"""
template_setup_except_handler_detaching = """\
frame_guard.detachFrame();"""
try_except_reraise_template = """\
{
PyTracebackObject *tb = _exception.getTraceback();
frame_guard.setLineNumber( tb->tb_lineno );
PyTracebackObject *tb_next = tb->tb_next;
tb->tb_next = NULL;
_exception.setTraceback( tb_next );
throw;
}"""
try_except_reraise_finally_template = """\
if ( _caught_%(try_count)d.isEmpty() )
{
%(thrower_code)s
}
else
{
_caught_%(try_count)d.rethrow();
}"""
try_except_reraise_unmatched_template = """\
else
{
PyTracebackObject *tb = _exception.getTraceback();
frame_guard.setLineNumber( tb->tb_lineno );
_exception.setTraceback( tb->tb_next );
tb->tb_next = NULL;
throw;
}"""
try_finally_template = """\
PythonExceptionKeeper _caught_%(try_count)d;
#if PYTHON_VERSION < 300
int _at_lineno_%(try_count)d = 0;
#endif
%(rethrow_setups)s
try
{
// Tried block:
%(tried_code)s
}
catch ( PythonException &_exception )
{
#if PYTHON_VERSION >= 300
if ( !_exception.hasTraceback() )
{
_exception.setTraceback( %(tb_making)s );
}
else
{
_exception.addTraceback( frame_guard.getFrame0() );
}
#else
_at_lineno_%(try_count)d = frame_guard.getLineNumber();
#endif
_caught_%(try_count)d.save( _exception );
#if PYTHON_VERSION >= 300
frame_guard.preserveExistingException();
_exception.toExceptionHandler();
#endif
}
%(rethrow_catchers)s
// Final block:
%(final_code)s
#if PYTHON_VERSION < 300
if ( _at_lineno_%(try_count)d != 0 )
{
frame_guard.setLineNumber( _at_lineno_%(try_count)d );
}
#endif
_caught_%(try_count)d.rethrow();
// Final end
%(rethrow_raisers)s"""
try_finally_template_setup_continue = """\
bool _continue_%(try_count)d = false;
"""
try_finally_template_setup_break = """\
bool _break_%(try_count)d = false;
"""
try_finally_template_setup_generator_return = """\
bool _return_%(try_count)d = false;
"""
try_finally_template_setup_return_value = """\
PyObjectTempKeeper1 _return_value_%(try_count)d;
"""
try_finally_template_catch_continue = """\
catch ( ContinueException const & )
{
_continue_%(try_count)d = true;
}
"""
try_finally_template_catch_break = """\
catch ( BreakException const & )
{
_break_%(try_count)d = true;
}
"""
try_finally_template_catch_return_value = """\
catch ( ReturnValueException const &e )
{
_return_value_%(try_count)d.assign( e.getValue1() );
}
"""
try_finally_template_reraise_continue = """\
if ( _continue_%(try_count)d )
{
throw ContinueException();
}"""
try_finally_template_reraise_break = """\
if ( _break_%(try_count)d )
{
throw BreakException();
}"""
try_finally_template_reraise_return_value = """\
if ( _return_value_%(try_count)d.isKeeping() )
{
throw ReturnValueException( _return_value_%(try_count)d.asObject1() );
}"""
try_finally_template_direct_return_value = """\
assert( _return_value_%(try_count)d.isKeeping() ); // Must be true as this is last.
return _return_value_%(try_count)d.asObject1();"""
try_finally_template_direct_generator_return_value = """\
assert( _return_value_%(try_count)d.isKeeping() ); // Must be true as this is last.
throw ReturnValueException( _return_value_%(try_count)d.asObject1() );"""
try_finally_template_indirect_return_value = """\
if ( _return_value_%(try_count)d.isKeeping() )
{
return _return_value_%(try_count)d.asObject1();
}"""
try_finally_template_indirect_generator_return_value = """\
if ( _return_value_%(try_count)d.isKeeping() )
{
throw ReturnValueException( _return_value_%(try_count)d.asObject1() );
}"""
# Very special template for:
# try:
# x = next(iter)
# except StopIteration:
# handler_code
template_try_next_except_stop_iteration = """\
PyObject *%(temp_var)s = ITERATOR_NEXT( %(source_identifier)s );
if ( %(temp_var)s == NULL )
{
%(handler_code)s
}
%(assignment_code)s"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesDicts.py 0000644 0001750 0001750 00000002672 12265264105 024341 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Dict related templates.
"""
template_make_dict_function = """\
NUITKA_MAY_BE_UNUSED static PyObject *MAKE_DICT%(pair_count)d( %(argument_decl)s )
{
PyObject *result = _PyDict_NewPresized( %(pair_count)d );
if (unlikely( result == NULL ))
{
throw PythonException();
}
%(add_elements_code)s
assert( Py_REFCNT( result ) == 1 );
return result;
}
"""
template_add_dict_element_code = """\
assertObject( %(dict_key)s );
assertObject( %(dict_value)s );
{
int status = PyDict_SetItem( result, %(dict_key)s, %(dict_value)s );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesFunction.py 0000644 0001750 0001750 00000014051 12265264105 025052 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Normal function (no yield) related templates.
"""
template_function_make_declaration = """\
static PyObject *MAKE_FUNCTION_%(function_identifier)s( %(function_creation_arg_spec)s );
"""
template_function_direct_declaration = """\
%(file_scope)s PyObject *impl_%(function_identifier)s( %(direct_call_arg_spec)s );
"""
function_context_body_template = """
// This structure is for attachment as self of %(function_identifier)s.
// It is allocated at the time the function object is created.
struct _context_%(function_identifier)s_t
{
// The function can access a read-only closure of the creator.
%(context_decl)s
};
static void _context_%(function_identifier)s_destructor( void *context_voidptr )
{
_context_%(function_identifier)s_t *_python_context = (_context_%(function_identifier)s_t *)context_voidptr;
%(context_free)s
delete _python_context;
}
"""
make_function_with_context_template = """
static PyObject *MAKE_FUNCTION_%(function_identifier)s( %(function_creation_args)s )
{
struct _context_%(function_identifier)s_t *_python_context = new _context_%(function_identifier)s_t;
// Copy the parameter default values and closure values over.
%(context_copy)s
PyObject *result = Nuitka_Function_New(
%(fparse_function_identifier)s,
%(dparse_function_identifier)s,
%(function_name_obj)s,
#if PYTHON_VERSION >= 330
%(function_qualname_obj)s,
#endif
%(code_identifier)s,
%(defaults)s,
#if PYTHON_VERSION >= 300
%(kwdefaults)s,
%(annotations)s,
#endif
%(module_identifier)s,
%(function_doc)s,
_python_context,
_context_%(function_identifier)s_destructor
);
return result;
}
"""
make_function_without_context_template = """
static PyObject *MAKE_FUNCTION_%(function_identifier)s( %(function_creation_args)s )
{
PyObject *result = Nuitka_Function_New(
%(fparse_function_identifier)s,
%(dparse_function_identifier)s,
%(function_name_obj)s,
#if PYTHON_VERSION >= 330
%(function_qualname_obj)s,
#endif
%(code_identifier)s,
%(defaults)s,
#if PYTHON_VERSION >= 300
%(kwdefaults)s,
%(annotations)s,
#endif
%(module_identifier)s,
%(function_doc)s
);
return result;
}
"""
function_body_template = """\
static PyObject *impl_%(function_identifier)s( %(parameter_objects_decl)s )
{
%(context_access_function_impl)s
// Local variable declarations.
%(function_locals)s
// Actual function code.
%(function_body)s
}
"""
function_direct_body_template = """\
%(file_scope)s PyObject *impl_%(function_identifier)s( %(direct_call_arg_spec)s )
{
%(context_access_function_impl)s
// Local variable declarations.
%(function_locals)s
// Actual function code.
%(function_body)s
}
"""
function_dict_setup = """\
// Locals dictionary setup.
PyObjectTemporary locals_dict( PyDict_New() );
"""
frame_guard_full_template = """\
static PyFrameObject *frame_%(frame_identifier)s = NULL;
if ( isFrameUnusable( frame_%(frame_identifier)s ) )
{
if ( frame_%(frame_identifier)s )
{
#if _DEBUG_REFRAME
puts( "reframe for %(frame_identifier)s" );
#endif
Py_DECREF( frame_%(frame_identifier)s );
}
frame_%(frame_identifier)s = MAKE_FRAME( %(code_identifier)s, %(module_identifier)s );
}
%(frame_class_name)s frame_guard( frame_%(frame_identifier)s );
try
{
assert( Py_REFCNT( frame_%(frame_identifier)s ) == 2 ); // Frame stack
%(codes)s
}
catch ( PythonException &_exception )
{
if ( !_exception.hasTraceback() )
{
_exception.setTraceback( %(tb_making)s );
}
else
{
_exception.addTraceback( frame_guard.getFrame0() );
}
%(frame_locals)s
if ( frame_guard.getFrame0() == frame_%(frame_identifier)s )
{
Py_DECREF( frame_%(frame_identifier)s );
frame_%(frame_identifier)s = NULL;
}
%(return_code)s
}
"""
frame_guard_python_return = """
_exception.toPython();
return NULL;"""
frame_guard_cpp_return = """
throw;"""
frame_guard_listcontr_template = """\
FrameGuardVeryLight frame_guard;
%(codes)s"""
frame_guard_once_template = """\
PyFrameObject *frame_%(frame_identifier)s = MAKE_FRAME( %(code_identifier)s, %(module_identifier)s );
%(frame_class_name)s frame_guard( frame_%(frame_identifier)s );
try
{
assert( Py_REFCNT( frame_%(frame_identifier)s ) == 2 ); // Frame stack
%(codes)s
}
catch ( PythonException &_exception )
{
if ( !_exception.hasTraceback() )
{
_exception.setTraceback( %(tb_making)s );
}
else
{
_exception.addTraceback( frame_guard.getFrame0() );
}
#if 0
// TODO: Recognize the need for it
Py_XDECREF( frame_guard.getFrame0()->f_locals );
frame_guard.getFrame0()->f_locals = %(frame_locals)s;
#endif
// Return the error.
_exception.toPython();
%(return_code)s
}"""
template_frame_locals_update = """\
Py_XDECREF( frame_guard.getFrame0()->f_locals );
frame_guard.getFrame0()->f_locals = %(locals_identifier)s;"""
# Bad to read, but the context declaration should be on one line.
# pylint: disable=C0301
function_context_access_template = """\
// The context of the function.
struct _context_%(function_identifier)s_t *_python_context = (struct _context_%(function_identifier)s_t *)self->m_context;"""
function_context_unused_template = """\
// No context is used."""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesLoops.py 0000644 0001750 0001750 00000003075 12265264105 024365 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" For and while loop related templates.
"""
template_loop_break_continue_catching = """\
while( true )
{
try
{
%(loop_body_codes)s
}
catch( ContinueException & )
{ /* Nothing to do */
}
catch ( BreakException & )
{ /* Break the loop */
break;
}
CONSIDER_THREADING();
}"""
template_loop_break_catching = """\
while( true )
{
try
{
%(loop_body_codes)s
}
catch ( BreakException & )
{ /* Break the loop */
break;
}
CONSIDER_THREADING();
}"""
template_loop_continue_catching = """\
while( true )
{
try
{
%(loop_body_codes)s
}
catch( ContinueException & )
{ /* Nothing to do */
}
CONSIDER_THREADING();
}"""
template_loop_simple = """\
while( true )
{
%(loop_body_codes)s
CONSIDER_THREADING();
}"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesPrinting.py 0000644 0001750 0001750 00000002344 12265264105 025061 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code Templates for printing. """
template_print_statement = """\
{
PyObject *target_file = %(target_file)s;
if ( target_file == Py_None )
{
target_file = GET_STDOUT();
Py_INCREF( target_file );
}
PyObjectTemporary file_reference( target_file );
%(print_elements_code)s}"""
template_print_value = """\
PRINT_ITEM_TO( %(target_file)s, %(print_value)s );"""
template_print_newline = """\
PRINT_NEW_LINE_TO( %(target_file)s );"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesImporting.py 0000644 0001750 0001750 00000002064 12265264105 025236 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Import related templates.
"""
import_from_template = """\
{
PyObjectTemporary module_temp(
%(module_lookup)s
);
try
{
%(lookup_code)s
}
catch( PythonException &_exception )
{
_exception.setType( PyExc_ImportError );
throw _exception;
}
}"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesCalls.py 0000644 0001750 0001750 00000014241 12265264105 024324 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Templates for calling functions with positional args only very quickly.
"""
template_call_cpython_function_fast_impl = """\
NUITKA_MAY_BE_UNUSED static PyObject *_fast_function_args( PyObject *func, PyObject **args, int count )
{
PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE( func );
PyObject *globals = PyFunction_GET_GLOBALS( func );
PyObject *argdefs = PyFunction_GET_DEFAULTS(func);
#if PYTHON_VERSION >= 300
PyObject *kwdefs = PyFunction_GET_KW_DEFAULTS(func);
if ( kwdefs == NULL && argdefs == NULL && co->co_argcount == count &&
co->co_flags == ( CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE ))
#else
if ( argdefs == NULL && co->co_argcount == count &&
co->co_flags == ( CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE ))
#endif
{
PyThreadState *tstate = PyThreadState_GET();
assertObject( globals );
PyFrameObject *frame = PyFrame_New( tstate, co, globals, NULL );
if (unlikely( frame == NULL ))
{
throw PythonException();
};
for ( int i = 0; i < count; i++ )
{
frame->f_localsplus[i] = INCREASE_REFCOUNT( args[i] );
}
PyObject *result = PyEval_EvalFrameEx( frame, 0 );
// Frame release protects against recursion as it may lead to variable
// destruction.
++tstate->recursion_depth;
Py_DECREF( frame );
--tstate->recursion_depth;
if ( result == NULL )
{
throw PythonException();
}
return result;
}
PyObject **defaults = NULL;
int nd = 0;
if ( argdefs != NULL )
{
defaults = &PyTuple_GET_ITEM(argdefs, 0);
nd = int( Py_SIZE( argdefs ) );
}
PyObject *result = PyEval_EvalCodeEx(
#if PYTHON_VERSION >= 300
(PyObject *)co,
#else
co, // code object
#endif
globals, // globals
NULL, // no locals
args, // args
count, // argcount
NULL, // kwds
0, // kwcount
defaults, // defaults
nd, // defcount
#if PYTHON_VERSION >= 300
kwdefs,
#endif
PyFunction_GET_CLOSURE( func )
);
if ( result == 0 )
{
throw PythonException();
}
return result;
}
"""
template_call_function_with_args_decl = """\
extern PyObject *CALL_FUNCTION_WITH_ARGS%(args_count)d( PyObject *called, %(args_decl)s );"""
template_call_function_with_args_impl = """\
PyObject *CALL_FUNCTION_WITH_ARGS%(args_count)d( PyObject *called, %(args_decl)s )
{
assertObject( called );
// Check if arguments are valid objects in debug mode.
#ifndef __NUITKA_NO_ASSERT__
PyObject *args_for_test[] = { %(args_list)s };
for( size_t i = 0; i < sizeof( args_for_test ) / sizeof( PyObject * ); i++ )
{
assertObject( args_for_test[ i ] );
}
#endif
if ( Nuitka_Function_Check( called ) )
{
if (unlikely( Py_EnterRecursiveCall( (char *)" while calling a Python object" ) ))
{
throw PythonException();
}
Nuitka_FunctionObject *function = (Nuitka_FunctionObject *)called;
PyObject *result;
PyObject *args[] = { %(args_list)s };
if ( function->m_direct_arg_parser )
{
result = function->m_direct_arg_parser(
function,
args,
sizeof( args ) / sizeof( PyObject * )
);
}
else
{
result = function->m_code(
function,
args,
sizeof( args ) / sizeof( PyObject * ),
NULL
);
}
Py_LeaveRecursiveCall();
if ( result == NULL )
{
throw PythonException();
}
return result;
}
else if ( Nuitka_Method_Check( called ) )
{
Nuitka_MethodObject *method = (Nuitka_MethodObject *)called;
// Unbound method without arguments, let the error path be slow.
if ( method->m_object != NULL )
{
if (unlikely( Py_EnterRecursiveCall( (char *)" while calling a Python object" ) ))
{
throw PythonException();
}
PyObject *args[] = {
method->m_object,
%(args_list)s
};
PyObject *result;
if ( method->m_function->m_direct_arg_parser )
{
result = method->m_function->m_direct_arg_parser(
method->m_function,
args,
sizeof( args ) / sizeof( PyObject * )
);
}
else
{
result = method->m_function->m_code(
method->m_function,
args,
sizeof( args ) / sizeof( PyObject * ),
NULL
);
}
Py_LeaveRecursiveCall();
if ( result == NULL )
{
throw PythonException();
}
return result;
}
}
else if ( PyFunction_Check( called ) )
{
PyObject *args[] = { %(args_list)s };
return _fast_function_args(
called,
args,
sizeof( args ) / sizeof( PyObject * )
);
}
return CALL_FUNCTION(
called,
PyObjectTemporary( MAKE_TUPLE%(args_count)d( %(args_list)s ) ).asObject0(),
NULL
);
}
"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesConstants.py 0000644 0001750 0001750 00000003475 12265264105 025251 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Templates for the constants handling.
"""
template_constants_reading = """
#include "nuitka/prelude.hpp"
// Sentinel PyObject to be used for all our call iterator endings. It will
// become a PyCObject pointing to NULL. It's address is unique, and that's
// enough for us to use it as sentinel value.
PyObject *_sentinel_value = NULL;
%(constant_declarations)s
extern "C" const unsigned char constant_bin[];
#define stream_data constant_bin
static void __initConstants( void )
{
%(constant_locals)s
%(constant_inits)s
}
void _initConstants( void )
{
if ( _sentinel_value == NULL )
{
#if PYTHON_VERSION < 300
_sentinel_value = PyCObject_FromVoidPtr( NULL, NULL );
#else
// The NULL value is not allowed for a capsule, so use something else.
_sentinel_value = PyCapsule_New( (void *)27, "sentinel", NULL );
#endif
assert( _sentinel_value );
__initConstants();
}
}
"""
template_constants_declaration = """\
// Call this to initialize all of the below
void _initConstants( void );
%(constant_declarations)s
"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesTuples.py 0000644 0001750 0001750 00000002472 12265264105 024545 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Tuple related templates.
"""
template_make_tuple_function = """\
NUITKA_MAY_BE_UNUSED static PyObject *MAKE_TUPLE%(argument_count)d( %(argument_decl)s )
{
PyObject *result = PyTuple_New( %(argument_count)d );
if (unlikely( result == NULL ))
{
throw PythonException();
}
%(add_elements_code)s
assert( Py_REFCNT( result ) == 1 );
return result;
}
"""
template_add_tuple_element_code = """\
assertObject( %(tuple_value)s );
PyTuple_SET_ITEM( result, %(tuple_index)d, INCREASE_REFCOUNT( %(tuple_value)s ) );"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesExecEval.py 0000644 0001750 0001750 00000003023 12265264105 024756 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code templates for local and module level (global) uses of exec/eval.
"""
exec_template = """\
PyObjectTemporary globals( %(globals_identifier)s );
PyObjectTemporary locals( %(locals_identifier)s );
PyObjectTemporary code( COMPILE_CODE( %(source_identifier)s, %(filename_identifier)s, %(mode_identifier)s, %(future_flags)s ) );
PyObject *result = EVAL_CODE( code.asObject0(), globals.asObject0(), locals.asObject0() );
Py_DECREF( result );"""
exec_copy_back_template = """
PyObject *locals_source = NULL;
if ( locals.asObject0() == locals_dict.asObject0() )
{
locals_source = locals.asObject0();
}
else if ( globals.asObject0() == locals_dict.asObject0() )
{
locals_source = globals.asObject0();
}
if ( locals_source != NULL )
{
%(store_locals_code)s
}"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesLists.py 0000644 0001750 0001750 00000002434 12265264105 024365 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" List related templates.
"""
template_make_list_function = """\
NUITKA_MAY_BE_UNUSED static PyObject *MAKE_LIST%(argument_count)d( %(argument_decl)s )
{
PyObject *result = PyList_New( %(argument_count)d );
if (unlikely( result == NULL ))
{
throw PythonException();
}
%(add_elements_code)s
assert( Py_REFCNT( result ) == 1 );
return result;
}
"""
template_add_list_element_code = """\
assertObject( %(list_value)s );
PyList_SET_ITEM( result, %(list_index)d, %(list_value)s );"""
Nuitka-0.5.0.1/nuitka/codegen/templates/__init__.py 0000644 0001750 0001750 00000001501 12265264105 022346 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Dummy file to make this directory a package. """
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesGeneratorFunction.py 0000644 0001750 0001750 00000021667 12265264105 026734 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Generator function (with yield) related templates.
"""
genfunc_context_body_template = """
// This structure is for attachment as self of the generator function %(function_identifier)s and
// contains the common closure. It is allocated at the time the genexpr object is created.
struct _context_common_%(function_identifier)s_t
{
// Ref count to keep track of common context usage and release only when it's the last one
int ref_count;
// The generator function can access a read-only closure of the creator.
%(function_common_context_decl)s
};
struct _context_generator_%(function_identifier)s_t
{
_context_common_%(function_identifier)s_t *common_context;
// The generator function instance can access its parameters from creation time.
%(function_instance_context_decl)s
};
static void _context_common_%(function_identifier)s_destructor( void *context_voidptr )
{
_context_common_%(function_identifier)s_t *_python_context = (struct _context_common_%(function_identifier)s_t *)context_voidptr;
assert( _python_context->ref_count > 0 );
_python_context->ref_count -= 1;
%(context_free)s
if ( _python_context->ref_count == 0 )
{
delete _python_context;
}
}
static void _context_generator_%(function_identifier)s_destructor( void *context_voidptr )
{
_context_generator_%(function_identifier)s_t *_python_context = (struct _context_generator_%(function_identifier)s_t *)context_voidptr;
_context_common_%(function_identifier)s_destructor( _python_context->common_context );
delete _python_context;
}
"""
genfunc_context_local_only_template = """
struct _context_generator_%(function_identifier)s_t
{
// The generator function instance can access its parameters from creation time.
%(function_instance_context_decl)s
};
static void _context_generator_%(function_identifier)s_destructor( void *context_voidptr )
{
_context_generator_%(function_identifier)s_t *_python_context = (struct _context_generator_%(function_identifier)s_t *)context_voidptr;
delete _python_context;
}
"""
make_genfunc_with_context_template = """
static PyObject *MAKE_FUNCTION_%(function_identifier)s( %(function_creation_args)s )
{
struct _context_common_%(function_identifier)s_t *_python_context = new _context_common_%(function_identifier)s_t;
_python_context->ref_count = 1;
// Copy the parameter default values and closure values over.
%(context_copy)s
return Nuitka_Function_New(
%(fparse_function_identifier)s,
%(dparse_function_identifier)s,
%(function_name_obj)s,
#if PYTHON_VERSION >= 330
%(function_qualname_obj)s,
#endif
%(code_identifier)s,
%(defaults)s,
#if PYTHON_VERSION >= 300
%(kwdefaults)s,
%(annotations)s,
#endif
%(module_identifier)s,
%(function_doc)s,
_python_context,
_context_common_%(function_identifier)s_destructor
);
}
"""
make_genfunc_without_context_template = """
static PyObject *MAKE_FUNCTION_%(function_identifier)s( %(function_creation_args)s )
{
return Nuitka_Function_New(
%(fparse_function_identifier)s,
%(dparse_function_identifier)s,
%(function_name_obj)s,
#if PYTHON_VERSION >= 330
%(function_qualname_obj)s,
#endif
%(code_identifier)s,
%(defaults)s,
#if PYTHON_VERSION >= 300
%(kwdefaults)s,
%(annotations)s,
#endif
%(module_identifier)s,
%(function_doc)s
);
}
"""
# TODO: Make the try/catch below unnecessary by detecting the presence
# or return statements in generators.
genfunc_yielder_template = """
static void %(function_identifier)s_context( Nuitka_GeneratorObject *generator )
{
try
{
// Make context accessible if one is used.
%(context_access)s
// Local variable inits
%(function_var_inits)s
// Actual function code.
%(function_body)s
}
catch( ReturnValueException &e )
{
PyErr_SetObject( PyExc_StopIteration, e.getValue0() );
}
assert( ERROR_OCCURED() );
// TODO: Won't return, we should tell the compiler about that.
generator->m_yielded = NULL;
swapFiber( &generator->m_yielder_context, &generator->m_caller_context );
}
"""
frame_guard_genfunc_template = """\
static PyFrameObject *frame_%(frame_identifier)s = NULL;
// Must be inside block, or else its d-tor will not be run.
if ( isFrameUnusable( frame_%(frame_identifier)s ) )
{
if ( frame_%(frame_identifier)s )
{
#if _DEBUG_REFRAME
puts( "reframe for %(frame_identifier)s" );
#endif
Py_DECREF( frame_%(frame_identifier)s );
}
frame_%(frame_identifier)s = MAKE_FRAME( %(code_identifier)s, %(module_identifier)s );
}
Py_INCREF( frame_%(frame_identifier)s );
generator->m_frame = frame_%(frame_identifier)s;
Py_CLEAR( generator->m_frame->f_back );
generator->m_frame->f_back = PyThreadState_GET()->frame;
Py_INCREF( generator->m_frame->f_back );
PyThreadState_GET()->frame = generator->m_frame;
FrameGuardLight frame_guard( &generator->m_frame );
// TODO: The inject of the exception through C++ is very non-optimal, this flag
// now indicates only if the exception occurs initially as supposed, or during
// life, this could and should be shortcut.
bool traceback;
try
{
// TODO: In case we don't raise exceptions ourselves, we would still have to do this, so
// beware to not optimize this away for generators without a replacement.
traceback = true;
CHECK_EXCEPTION( generator );
traceback = false;
%(codes)s
PyErr_SetObject( PyExc_StopIteration, (PyObject *)NULL );
}
catch ( PythonException &_exception )
{
if ( !_exception.hasTraceback() )
{
_exception.setTraceback( %(tb_making)s );
}
else if ( traceback == false )
{
_exception.addTraceback( generator->m_frame );
}
_exception.toPython();
// TODO: Moving this code is not allowed yet.
generator->m_yielded = NULL;
}"""
genfunc_common_context_use_template = """\
struct _context_common_%(function_identifier)s_t *_python_common_context = (struct _context_common_%(function_identifier)s_t *)self->m_context;
struct _context_generator_%(function_identifier)s_t *_python_context = new _context_generator_%(function_identifier)s_t;
_python_context->common_context = _python_common_context;
_python_common_context->ref_count += 1;"""
genfunc_local_context_use_template = """\
struct _context_generator_%(function_identifier)s_t *_python_context = \
new _context_generator_%(function_identifier)s_t;"""
genfunc_generator_without_context_making = """\
PyObject *result = Nuitka_Generator_New(
%(function_identifier)s_context,
%(function_name_obj)s,
%(code_identifier)s
);"""
genfunc_generator_with_context_making = """\
PyObject *result = Nuitka_Generator_New(
%(function_identifier)s_context,
%(function_name_obj)s,
%(code_identifier)s,
_python_context,
_context_generator_%(function_identifier)s_destructor
);"""
genfunc_function_maker_template = """
static PyObject *impl_%(function_identifier)s( %(parameter_objects_decl)s )
{
// Create context if any
%(context_making)s
try
{
%(generator_making)s
if (unlikely( result == NULL ))
{
PyErr_Format( PyExc_RuntimeError, "cannot create function %(function_name)s" );
return NULL;
}
// Copy to context parameter values and closured variables if any.
%(context_copy)s
return result;
}
catch ( PythonException &_exception )
{
_exception.toPython();
return NULL;
}
}
"""
generator_context_access_template = """
// The context of the generator.
struct _context_common_%(function_identifier)s_t *_python_context = (struct _context_common_%(function_identifier)s_t *)self->m_context;
"""
generator_context_unused_template = """\
// No context is used.
"""
# TODO: The NUITKA_MAY_BE_UNUSED is because Nuitka doesn't yet detect the case of unused
# parameters (which are stored in the context for generators to share) reliably.
generator_context_access_template2 = """
NUITKA_MAY_BE_UNUSED struct _context_generator_%(function_identifier)s_t *_python_context = (_context_generator_%(function_identifier)s_t *)generator->m_context;
"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesParameterParsing.py 0000644 0001750 0001750 00000033160 12265264105 026533 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Parameter parsing related templates.
"""
template_parameter_function_entry_point = """\
static PyObject *%(parse_function_identifier)s( Nuitka_FunctionObject *self, PyObject **args, Py_ssize_t args_size, PyObject *kw )
{
assert( kw == NULL || PyDict_Check( kw ) );
NUITKA_MAY_BE_UNUSED Py_ssize_t kw_size = kw ? PyDict_Size( kw ) : 0;
NUITKA_MAY_BE_UNUSED Py_ssize_t kw_found = 0;
NUITKA_MAY_BE_UNUSED Py_ssize_t kw_only_found = 0;
Py_ssize_t args_given = args_size;
%(parameter_parsing_code)s
return %(impl_function_identifier)s( %(parameter_objects_list)s );
error_exit:;
%(parameter_release_code)s
return NULL;
}
"""
template_parameter_function_refuses = r"""
if (unlikely( args_given + kw_size > 0 ))
{
#if PYTHON_VERSION < 330
ERROR_NO_ARGUMENTS_ALLOWED(
self,
args_given + kw_size
);
#else
ERROR_NO_ARGUMENTS_ALLOWED(
self,
kw_size > 0 ? kw : NULL,
args_given
);
#endif
goto error_exit;
}
"""
parse_argument_template_check_counts_without_list_star_arg = r"""
// Check if too many arguments were given in case of non star args
if (unlikely( args_given > %(top_level_parameter_count)d ))
{
#if PYTHON_VERSION < 270
ERROR_TOO_MANY_ARGUMENTS( self, args_given, kw_size );
#elif PYTHON_VERSION < 330
ERROR_TOO_MANY_ARGUMENTS( self, args_given + kw_found );
#else
ERROR_TOO_MANY_ARGUMENTS( self, args_given, kw_only_found );
#endif
goto error_exit;
}
"""
parse_argument_usable_count = r"""
// Copy normal parameter values given as part of the args list to the respective variables:
"""
argparse_template_plain_argument = """\
if (likely( %(parameter_position)d < args_given ))
{
if (unlikely( _python_par_%(parameter_name)s != NULL ))
{
ERROR_MULTIPLE_VALUES( self, %(parameter_position)d );
goto error_exit;
}
_python_par_%(parameter_name)s = INCREASE_REFCOUNT( args[ %(parameter_position)d ] );
}
else if ( _python_par_%(parameter_name)s == NULL )
{
if ( %(parameter_position)d + self->m_defaults_given >= %(top_level_parameter_count)d )
{
_python_par_%(parameter_name)s = INCREASE_REFCOUNT( PyTuple_GET_ITEM( self->m_defaults, self->m_defaults_given + %(parameter_position)d - %(top_level_parameter_count)d ) );
}
#if PYTHON_VERSION < 330
else
{
#if PYTHON_VERSION < 270
ERROR_TOO_FEW_ARGUMENTS( self, kw_size, args_given + kw_found );
#elif PYTHON_VERSION < 300
ERROR_TOO_FEW_ARGUMENTS( self, args_given + kw_found );
#else
ERROR_TOO_FEW_ARGUMENTS( self, args_given + kw_found - kw_only_found );
#endif
goto error_exit;
}
#endif
}
"""
template_arguments_check = """
#if PYTHON_VERSION >= 330
if (unlikely( %(parameter_test)s ))
{
PyObject *values[] = { %(parameter_list)s };
ERROR_TOO_FEW_ARGUMENTS( self, values );
goto error_exit;
}
#endif
"""
argparse_template_nested_argument = """\
if (likely( %(parameter_position)d < args_given ))
{
_python_par_%(parameter_name)s = args[ %(parameter_position)d ];
}
else if ( _python_par_%(parameter_name)s == NULL )
{
if ( %(parameter_position)d + self->m_defaults_given >= %(top_level_parameter_count)d )
{
_python_par_%(parameter_name)s = INCREASE_REFCOUNT( PyTuple_GET_ITEM( self->m_defaults, self->m_defaults_given + %(parameter_position)d - %(top_level_parameter_count)d ) );
}
else
{
#if PYTHON_VERSION < 270
ERROR_TOO_FEW_ARGUMENTS( self, kw_size, args_given + kw_found );
#else
ERROR_TOO_FEW_ARGUMENTS( self, args_given + kw_found );
#endif
goto error_exit;
}
}
"""
parse_argument_template_copy_list_star_args = """
// Copy left over argument values to the star list parameter given.
if ( args_given > %(top_level_parameter_count)d )
{
_python_par_%(list_star_parameter_name)s = PyTuple_New( args_size - %(top_level_parameter_count)d );
for( Py_ssize_t i = 0; i < args_size - %(top_level_parameter_count)d; i++ )
{
PyTuple_SET_ITEM( _python_par_%(list_star_parameter_name)s, i, INCREASE_REFCOUNT( args[%(top_level_parameter_count)d+i] ) );
}
}
else
{
_python_par_%(list_star_parameter_name)s = INCREASE_REFCOUNT( const_tuple_empty );
}
"""
parse_argument_template_dict_star_copy = """
if ( kw == NULL )
{
_python_par_%(dict_star_parameter_name)s = PyDict_New();
}
else
{
if ( ((PyDictObject *)kw)->ma_used > 0 )
{
#if PYTHON_VERSION < 330
_python_par_%(dict_star_parameter_name)s = _PyDict_NewPresized( ((PyDictObject *)kw)->ma_used );
for ( int i = 0; i <= ((PyDictObject *)kw)->ma_mask; i++ )
{
PyDictEntry *entry = &((PyDictObject *)kw)->ma_table[ i ];
if ( entry->me_value != NULL )
{
#if PYTHON_VERSION < 300
if (unlikely( !PyString_Check( entry->me_key ) && !PyUnicode_Check( entry->me_key ) ))
#else
if (unlikely( !PyUnicode_Check( entry->me_key ) ))
#endif
{
PyErr_Format( PyExc_TypeError, "%(function_name)s() keywords must be strings" );
goto error_exit;
}
int res = PyDict_SetItem( _python_par_%(dict_star_parameter_name)s, entry->me_key, entry->me_value );
if (unlikely( res == -1 ))
{
goto error_exit;
}
}
}
#else
if ( _PyDict_HasSplitTable( (PyDictObject *)kw) )
{
PyDictObject *mp = (PyDictObject *)kw;
PyObject **newvalues = PyMem_NEW( PyObject *, mp->ma_keys->dk_size );
assert (newvalues != NULL);
PyDictObject *split_copy = PyObject_GC_New( PyDictObject, &PyDict_Type );
assert( split_copy != NULL );
split_copy->ma_values = newvalues;
split_copy->ma_keys = mp->ma_keys;
split_copy->ma_used = mp->ma_used;
mp->ma_keys->dk_refcnt += 1;
Nuitka_GC_Track( split_copy );
int size = mp->ma_keys->dk_size;
for ( int i = 0; i < size; i++ )
{
PyDictKeyEntry *entry = &split_copy->ma_keys->dk_entries[ i ];
if (unlikely( !PyUnicode_Check( entry->me_key ) ))
{
PyErr_Format( PyExc_TypeError, "%(function_name)s() keywords must be strings" );
goto error_exit;
}
split_copy->ma_values[ i ] = INCREASE_REFCOUNT_X( mp->ma_values[ i ] );
}
_python_par_%(dict_star_parameter_name)s = (PyObject *)split_copy;
}
else
{
_python_par_%(dict_star_parameter_name)s = PyDict_New();
PyDictObject *mp = (PyDictObject *)kw;
int size = mp->ma_keys->dk_size;
for ( int i = 0; i < size; i++ )
{
PyDictKeyEntry *entry = &mp->ma_keys->dk_entries[i];
// TODO: One of these cases has been dealt with above.
PyObject *value;
if ( mp->ma_values )
{
value = mp->ma_values[ i ];
}
else
{
value = entry->me_value;
}
if ( value != NULL )
{
if (unlikely( !PyUnicode_Check( entry->me_key ) ))
{
PyErr_Format( PyExc_TypeError, "%(function_name)s() keywords must be strings" );
goto error_exit;
}
int res = PyDict_SetItem( _python_par_%(dict_star_parameter_name)s, entry->me_key, value );
if (unlikely( res == -1 ))
{
goto error_exit;
}
}
}
}
#endif
}
else
{
_python_par_%(dict_star_parameter_name)s = PyDict_New();
}
}
"""
parse_argument_template_check_dict_parameter_with_star_dict = """
// Check if argument %(parameter_name)s was given as keyword argument
if ( kw_size > 0 )
{
PyObject *kw_arg_value = PyDict_GetItem( _python_par_%(dict_star_parameter_name)s, %(parameter_name_object)s );
if ( kw_arg_value != NULL )
{
assert( _python_par_%(parameter_name)s == NULL );
_python_par_%(parameter_name)s = INCREASE_REFCOUNT( kw_arg_value );
PyDict_DelItem( _python_par_%(dict_star_parameter_name)s, %(parameter_name_object)s );
kw_found += 1;
}
}
"""
parse_argument_template_check_dict_parameter_without_star_dict = """
// Check if argument %(parameter_name)s was given as keyword argument
if ( kw_size > 0 )
{
PyObject *kw_arg_value = PyDict_GetItem( kw, %(parameter_name_object)s );
if ( kw_arg_value != NULL )
{
assert( _python_par_%(parameter_name)s == NULL );
_python_par_%(parameter_name)s = INCREASE_REFCOUNT( kw_arg_value );
}
}
"""
argparse_template_assign_from_dict_parameters = """\
if ( kw_size > 0 )
{
Py_ssize_t ppos = 0;
PyObject *key, *value;
while( PyDict_Next( kw, &ppos, &key, &value ) )
{
#if PYTHON_VERSION < 300
if (unlikely( !PyString_Check( key ) && !PyUnicode_Check( key ) ))
#else
if (unlikely( !PyUnicode_Check( key ) ))
#endif
{
PyErr_Format( PyExc_TypeError, "%(function_name)s() keywords must be strings" );
goto error_exit;
}
NUITKA_MAY_BE_UNUSED bool found = false;
Py_INCREF( key );
Py_INCREF( value );
// Quick path, could be our value.
%(parameter_quick_path)s
// Slow path, compare against all parameter names.
%(parameter_slow_path)s
Py_DECREF( key );
if ( found == false )
{
Py_DECREF( value );
PyErr_Format(
PyExc_TypeError,
"%(function_name)s() got an unexpected keyword argument '%%s'",
Nuitka_String_Check( key ) ? Nuitka_String_AsString( key ) : ""
);
goto error_exit;
}
}
#if PYTHON_VERSION < 300
assert( kw_found == kw_size );
assert( kw_only_found == 0 );
#endif
}
"""
argparse_template_assign_from_dict_parameter_quick_path = """\
if ( found == false && %(parameter_name_object)s == key )
{
%(parameter_assign_from_kw)s
found = true;
kw_found += 1;
}
"""
argparse_template_assign_from_dict_parameter_quick_path_kw_only = """\
if ( found == false && %(parameter_name_object)s == key )
{
%(parameter_assign_from_kw)s
found = true;
kw_found += 1;
kw_only_found += 1;
}
"""
argparse_template_assign_from_dict_parameter_slow_path = """\
if ( found == false && RICH_COMPARE_BOOL_EQ_PARAMETERS( %(parameter_name_object)s, key ) )
{
%(parameter_assign_from_kw)s
found = true;
kw_found += 1;
}
"""
argparse_template_assign_from_dict_parameter_slow_path_kw_only = """\
if ( found == false && RICH_COMPARE_BOOL_EQ_PARAMETERS( %(parameter_name_object)s, key ) )
{
%(parameter_assign_from_kw)s
found = true;
kw_found += 1;
kw_only_found += 1;
}
"""
argparse_template_assign_from_dict_finding = """\
assert( _python_par_%(parameter_name)s == NULL );
_python_par_%(parameter_name)s = value;
"""
parse_argument_template_nested_argument_unpack = """\
// Unpack from _python_par_%(parameter_name)s
{
PyObject *_python_iter_%(parameter_name)s = PyObject_GetIter( %(unpack_source_identifier)s );
if (unlikely( _python_iter_%(parameter_name)s == NULL ))
{
goto error_exit;
}
%(unpack_code)s
// Check that the unpack was complete.
if (unlikely( UNPACK_PARAMETER_ITERATOR_CHECK( _python_iter_%(parameter_name)s ) == false ))
{
Py_DECREF( _python_iter_%(parameter_name)s );
goto error_exit;
}
Py_DECREF( _python_iter_%(parameter_name)s );
}"""
parse_argument_template_nested_argument_assign = """
// Unpack to _python_par_%(parameter_name)s
_python_par_%(parameter_name)s = UNPACK_PARAMETER_NEXT( _python_iter_%(iter_name)s, %(unpack_count)d );
if (unlikely (_python_par_%(parameter_name)s == NULL ))
{
Py_DECREF( _python_iter_%(iter_name)s );
goto error_exit;
}
"""
template_kwonly_argument_default = """\
if ( _python_par_%(parameter_name)s == NULL )
{
_python_par_%(parameter_name)s = PyDict_GetItem( self->m_kwdefaults, %(parameter_name_object)s );
#if PYTHON_VERSION < 330
if (unlikely (_python_par_%(parameter_name)s == NULL ))
{
PyErr_Format( PyExc_TypeError, "%(function_name)s() needs keyword-only argument %(parameter_name)s" );
goto error_exit;
}
Py_INCREF( _python_par_%(parameter_name)s );
#else
Py_XINCREF( _python_par_%(parameter_name)s );
#endif
}"""
template_kwonly_arguments_check = """
#if PYTHON_VERSION >= 330
if (unlikely( %(parameter_test)s ))
{
PyObject *values[] = { %(parameter_list)s };
ERROR_TOO_FEW_KWONLY( self, values );
goto error_exit;
}
#endif
"""
template_dparser = """
static PyObject *dparse_%(function_identifier)s( Nuitka_FunctionObject *self, PyObject **args, int size )
{
if ( size == %(arg_count)d )
{
return impl_%(function_identifier)s( self%(args_forward)s );
}
else
{
PyObject *result = fparse_%(function_identifier)s( self, args, size, NULL );
return result;
}
}
"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesSets.py 0000644 0001750 0001750 00000002361 12265264105 024204 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Set related templates.
"""
template_make_set_function = """\
NUITKA_MAY_BE_UNUSED static PyObject *MAKE_SET%(argument_count)d( %(argument_decl)s )
{
PyObject *result = PySet_New( NULL );
if (unlikely( result == NULL ))
{
throw PythonException();
}
%(add_elements_code)s
assert( Py_REFCNT( result ) == 1 );
return result;
}
"""
template_add_set_element_code = """\
assertObject( %(set_value)s );
PySet_Add( result, %(set_value)s );"""
Nuitka-0.5.0.1/nuitka/codegen/templates/CodeTemplatesBranches.py 0000644 0001750 0001750 00000002114 12265264105 025007 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Templates for conditional statements and expressions.
"""
template_branch_one = """\
if ( %(condition)s )
{
%(branch_code)s
}"""
template_branch_two = """\
if ( %(condition)s )
{
%(branch_yes_code)s
}
else
{
%(branch_no_code)s
}"""
template_conditional_expression = """\
( %(condition)s ? %(yes)s : %(no)s )"""
Nuitka-0.5.0.1/nuitka/codegen/DictCodes.py 0000644 0001750 0001750 00000005367 12265264105 020470 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code generation for dicts.
Right now only the creation is done here. But more should be added later on.
"""
from . import CodeTemplates
make_dicts_used = set( range( 0, 3 ) )
def addMakeDictUse(args_length):
assert type( args_length ) is int
make_dicts_used.add( args_length )
def getDictionaryCreationCode(context, order_relevance, args_identifiers):
from .OrderedEvaluation import getOrderRelevanceEnforcedArgsCode
assert len( args_identifiers ) % 2 == 0
args_length = len( args_identifiers ) // 2
addMakeDictUse( args_length )
return getOrderRelevanceEnforcedArgsCode(
helper = "MAKE_DICT%d" % args_length,
export_ref = 0,
ref_count = 1,
tmp_scope = "make_dict",
order_relevance = order_relevance,
args = args_identifiers,
context = context
)
def getMakeDictsCode():
make_dicts_codes = []
for arg_count in sorted( make_dicts_used ):
add_elements_code = []
for arg_index in range( arg_count ):
add_elements_code.append(
CodeTemplates.template_add_dict_element_code % {
"dict_key" : "key%d" % ( arg_index + 1 ),
"dict_value" : "value%d" % ( arg_index + 1 )
}
)
make_dicts_codes.append(
CodeTemplates.template_make_dict_function % {
"pair_count" : arg_count,
"argument_decl" : ", ".join(
"PyObject *value%(index)d, PyObject *key%(index)d" % {
"index" : (arg_index+1)
}
for arg_index in
range( arg_count )
),
"add_elements_code" : "\n".join( add_elements_code ),
}
)
return CodeTemplates.template_header_guard % {
"header_guard_name" : "__NUITKA_DICTS_H__",
"header_body" : "\n".join( make_dicts_codes )
}
Nuitka-0.5.0.1/nuitka/codegen/Identifiers.py 0000644 0001750 0001750 00000017604 12265264105 021071 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Identifiers hold code references.
These are generally the means to effectively hide the reference count. The best
part is where getCheapRefCount allows to not allocate references not needed.
"""
# The method signatures do not always require usage of self, sometimes can be
# decided based on class. pylint: disable=R0201
class Identifier:
def __init__(self, code, ref_count):
self.code = code
self.ref_count = ref_count
def getRefCount(self):
return self.ref_count
def getCheapRefCount(self):
return self.ref_count
def getCode(self):
return self.code
def getCodeObject(self):
return self.code
def getCodeExportRef(self):
if self.getRefCount():
return self.getCodeObject()
else:
return "INCREASE_REFCOUNT( %s )" % self.getCodeObject()
def getCodeTemporaryRef(self):
if self.getRefCount():
return "PyObjectTemporary( %s ).asObject0()" % self.getCodeObject()
else:
return self.getCodeObject()
def getCodeDropRef(self):
if self.ref_count == 0:
return self.getCodeTemporaryRef()
else:
return "DECREASE_REFCOUNT( %s )" % self.getCodeObject()
def __repr__(self):
return "" % ( self.code, self.ref_count )
def isConstantIdentifier(self):
return False
class NameIdentifier(Identifier):
def __init__(self, name, hint, code, ref_count):
self.name = name
self.hint = hint
Identifier.__init__(
self,
code = code,
ref_count = ref_count
)
def __repr__(self):
return "" % (
self.hint,
self.name,
self.code
)
def getCodeObject(self):
return "%s.asObject0()" % self.getCode()
def getCodeTemporaryRef(self):
return "%s.asObject0()" % self.getCode()
def getCodeExportRef(self):
return "%s.asObject1()" % self.getCode()
def getCodeDropRef(self):
return self.getCodeTemporaryRef()
class ConstantIdentifier(Identifier):
def __init__(self, constant_code, constant_value):
Identifier.__init__( self, constant_code, 0 )
self.constant_value = constant_value
def __repr__(self):
return "" % self.code
def isConstantIdentifier(self):
return True
def getCheapRefCount(self):
return 0
def getConstant(self):
return self.constant_value
class SpecialConstantIdentifier(ConstantIdentifier):
def __init__(self, constant_value):
if constant_value is None:
ConstantIdentifier.__init__( self, "Py_None", None )
elif constant_value is True:
ConstantIdentifier.__init__( self, "Py_True", True )
elif constant_value is False:
ConstantIdentifier.__init__( self, "Py_False", False )
elif constant_value is Ellipsis:
ConstantIdentifier.__init__( self, "Py_Ellipsis", Ellipsis )
else:
assert False, constant_value
class EmptyDictIdentifier(Identifier):
def __init__(self):
Identifier.__init__( self, "PyDict_New()", 1 )
def getCheapRefCount(self):
return 1
def isConstantIdentifier(self):
return True
def getConstant(self):
return {}
class ModuleVariableIdentifier:
def __init__(self, var_name, var_code_name):
self.var_name = var_name
self.var_code_name = var_code_name
def isConstantIdentifier(self):
return False
def __repr__(self):
return "" % self.var_name
def getRefCount(self):
return 0
def getCheapRefCount(self):
# The asObject0 is the fastest way, stealing a reference directly from
# the module dictionary if possible.
return 0
def getCodeTemporaryRef(self):
return "GET_MODULE_VALUE0( %s )" % (
self.var_code_name
)
def getCodeExportRef(self):
return "GET_MODULE_VALUE1( %s )" % (
self.var_code_name
)
def getCodeDropRef(self):
return self.getCodeTemporaryRef()
class MaybeModuleVariableIdentifier(ModuleVariableIdentifier):
def __init__(self, var_name, var_code_name):
ModuleVariableIdentifier.__init__(
self,
var_name = var_name,
var_code_name = var_code_name
)
def __repr__(self):
return "" % self.var_name
def getCodeTemporaryRef(self):
return "GET_LOCALS_OR_MODULE_VALUE0( locals_dict.asObject0(), %s )" % (
self.var_code_name
)
def getCodeExportRef(self):
return "GET_LOCALS_OR_MODULE_VALUE1( locals_dict.asObject0(), %s )" % (
self.var_code_name
)
class TempObjectIdentifier(Identifier):
def __init__(self, var_name, code):
self.var_name = var_name
Identifier.__init__( self, code, 0 )
def __repr__(self):
return "" % self.var_name
class KeeperAccessIdentifier(Identifier):
def __init__(self, var_name):
Identifier.__init__( self, var_name, 1 )
def getCheapRefCount(self):
return 0
class NullIdentifier(Identifier):
def __init__(self):
Identifier.__init__(
self,
code = "NULL",
ref_count = 0
)
def getCodeExportRef(self):
return "NULL"
class ThrowingIdentifier(Identifier):
def __init__(self, code):
Identifier.__init__(
self,
code = code,
ref_count = 0
)
def getCodeExportRef(self):
return self.getCodeObject()
def getCodeTemporaryRef(self):
return self.getCodeObject()
def getCheapRefCount(self):
return 0
class CallIdentifier(Identifier):
def __init__(self, called, args):
Identifier.__init__(
self,
code = "%s( %s )" % (
called,
", ".join( args )
),
ref_count = 1
)
class HelperCallIdentifier(CallIdentifier):
def __init__(self, helper, *args):
CallIdentifier.__init__(
self,
called = helper,
args = [
arg.getCodeTemporaryRef() if arg is not None else "NULL"
for arg in
args
]
)
def getCodeTemporaryRefs(identifiers):
""" Helper to create temporary reference code of many identifiers at once.
"""
return [ identifier.getCodeTemporaryRef() for identifier in identifiers ]
def getCodeExportRefs(identifiers):
""" Helper to create export reference code of many identifiers at once.
"""
return [ identifier.getCodeExportRef() for identifier in identifiers ]
def defaultToNullIdentifier(identifier):
if identifier is not None:
return identifier
else:
return NullIdentifier()
def defaultToNoneIdentifier(identifier):
if identifier is not None:
return identifier
else:
return SpecialConstantIdentifier( constant_value = None )
Nuitka-0.5.0.1/nuitka/codegen/LineNumberCodes.py 0000644 0001750 0001750 00000003223 12265264105 021632 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Generate code that updates the source code line.
"""
# Stack of source code references, to push and pop from, for loops and branches
# to not rely on their last line number.
source_ref_stack = [ None ]
def resetLineNumber():
source_ref_stack[-1] = None
def pushLineNumberBranch():
source_ref_stack.append( source_ref_stack[-1] )
def popLineNumberBranch():
del source_ref_stack[-1]
def mergeLineNumberBranches():
source_ref_stack[-1] = None
def _getLineNumberCode(line_number):
return "frame_guard.setLineNumber( %d )" % line_number
def getLineNumberCode(source_ref):
if source_ref.shallSetCurrentLine():
line_number = source_ref.getLineNumber()
if line_number != source_ref_stack[-1]:
source_ref_stack[-1] = line_number
return _getLineNumberCode( line_number )
else:
return ""
else:
return ""
Nuitka-0.5.0.1/nuitka/codegen/Namify.py 0000644 0001750 0001750 00000015427 12265264105 020050 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Namify constants.
This determines the identifier names of constants in the generated code. We
try to have readable names where possible, and resort to hash codes only when
it is really necessary.
"""
# pylint: disable=W0622
from nuitka.__past__ import long, unicode
# pylint: enable=W0622
from logging import warning
import hashlib, re, math
# False alarms about "hashlib.md5" due to its strange way of defining what is
# exported, pylint won't understand it. pylint: disable=E1101
class ExceptionCannotNamify(Exception):
pass
def namifyConstant(constant):
# Many branches, statements and every case has a return, this is a huge case
# statement, that encodes the naming policy of constants, with often complex
# decisions to make, pylint: disable=R0911,R0912,R0915
if type( constant ) is int:
if constant == 0:
result = "int_0"
elif constant > 0:
result = "int_pos_%d" % constant
else:
result = "int_neg_%d" % abs( constant )
if len( result ) > 32:
result = _digest( result )
return result
elif type( constant ) is long:
if constant == 0:
result = "long_0"
elif constant > 0:
result = "long_pos_%d" % constant
else:
result = "long_neg_%d" % abs( constant )
if len( result ) > 32:
result = _digest( result )
return result
elif constant is None:
return "none"
elif constant is True:
return "true"
elif constant is False:
return "false"
elif constant is Ellipsis:
return "ellipsis"
elif type( constant ) is str:
return "str_" + _namifyString( constant )
elif type( constant ) is bytes:
return "bytes_" + _namifyString( constant )
elif type( constant ) is unicode:
if _isAscii( constant ):
return "unicode_" + _namifyString( str( constant ) )
else:
# Others are better digested to not cause compiler trouble
return "unicode_digest_" + _digest( repr( constant ) )
elif type( constant ) is float:
if math.isnan( constant ):
return "float_%s_nan" % (
"minus" if math.copysign( 1, constant ) < 0 else "plus"
)
return "float_%s" % repr( constant ).replace( ".", "_" ).\
replace( "-", "_minus_" ).replace( "+", "" )
elif type( constant ) is complex:
value = str( constant ).replace( "+", "p" ).replace( "-", "m" ).\
replace( ".", "_" )
if value.startswith( "(" ) and value.endswith( ")" ):
value = value[1:-1]
return "complex_%s" % value
elif type( constant ) is dict:
if constant == {}:
return "dict_empty"
else:
return "dict_" + _digest( repr( constant ) )
elif type( constant ) is set:
if constant == set():
return "set_empty"
else:
return "set_" + _digest( repr( constant ) )
elif type( constant ) is frozenset:
if constant == frozenset():
return "frozenset_empty"
else:
return "frozenset_" + _digest( repr( constant ) )
elif type( constant ) is tuple:
if constant == ():
return "tuple_empty"
else:
try:
result = "_".join(
namifyConstant( value )
for value in
constant
)
if len( result ) > 60:
result = _digest( repr( constant ) )
return "tuple_" + result + "_tuple"
except ExceptionCannotNamify:
warning( "Couldn't namify '%r'" % value )
return "tuple_" + _digest( repr( constant ) )
elif type( constant ) is list:
if constant == []:
return "list_empty"
else:
try:
result = "_".join(
namifyConstant( value )
for value in
constant
)
if len( result ) > 60:
result = _digest( repr( constant ) )
return "list_" + result + "_list"
except ExceptionCannotNamify:
warning( "Couldn't namify '%r'" % value )
return "list_" + _digest( repr( constant ) )
elif type( constant ) is range:
# Python3 type only.
return "range_%s" % (
str( constant )[6:-1].replace( " ", "" ).replace( ",", "_" )
)
raise ExceptionCannotNamify( "%r" % constant )
_re_str_needs_no_digest = re.compile( r"^([a-z]|[A-Z]|[0-9]|_){1,40}$", re.S )
def _namifyString(string):
# Many branches case has a return, encodes the naming policy of strings
# constants, with often complex decisions to make, pylint: disable=R0911
if string in ( "", b"" ):
return "empty"
elif string == " ":
return "space"
elif string == ".":
return "dot"
elif type( string ) is str and \
_re_str_needs_no_digest.match( string ) and \
"\n" not in string:
# Some strings can be left intact for source code readability.
return "plain_" + string
elif len( string ) == 1:
return "chr_%d" % ord( string )
elif len( string ) > 2 and string[0] == "<" and string[-1] == ">" and \
_re_str_needs_no_digest.match( string[1:-1] ) and \
"\n" not in string:
return "angle_" + string[1:-1]
else:
# Others are better digested to not cause compiler trouble
return "digest_" + _digest( string )
def _isAscii(string):
try:
_unused = str( string )
return True
except UnicodeEncodeError:
return False
def _digest(value):
if str is not unicode:
return hashlib.md5( value ).hexdigest()
else:
if type( value ) is bytes:
return hashlib.md5( value ).hexdigest()
else:
# Do the hash not in UTF-8 as that won't allow "surrogates".
return hashlib.md5( value.encode( "utf-16" ) ).hexdigest()
Nuitka-0.5.0.1/nuitka/codegen/ModuleCodes.py 0000644 0001750 0001750 00000007445 12265270513 021031 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code to generate and interact with compiled module objects.
"""
from .ConstantCodes import (
getConstantCode,
)
from .VariableCodes import (
getLocalVariableInitCode,
)
from .Indentation import indented
from . import CodeTemplates
from nuitka import Options
import re
def getModuleAccessCode(context):
return "module_%s" % context.getModuleCodeName()
def getModuleIdentifier(module_name):
# TODO: This is duplication with ModuleNode.getCodeName, remove it.
def r(match):
c = match.group()
if c == '.':
return "$"
else:
return "$$%d$" % ord(c)
return "".join(re.sub("[^a-zA-Z0-9_]", r ,c) for c in module_name)
def getModuleDeclarationCode(module_name, extra_declarations):
module_header_code = CodeTemplates.module_header_template % {
"module_identifier" : getModuleIdentifier( module_name ),
"extra_declarations" : extra_declarations
}
return CodeTemplates.template_header_guard % {
"header_guard_name" : "__%s_H__" % getModuleIdentifier(module_name),
"header_body" : module_header_code
}
def getModuleMetapathLoaderEntryCode(module_name, is_shlib):
if is_shlib:
return CodeTemplates.template_metapath_loader_shlib_module_entry % {
"module_name" : module_name
}
else:
return CodeTemplates.template_metapath_loader_compiled_module_entry % {
"module_name" : module_name,
"module_identifier" : getModuleIdentifier( module_name ),
}
def getModuleCode( context, module_name, codes, metapath_loader_inittab,
function_decl_codes, function_body_codes, temp_variables ):
# For the module code, lots of attributes come together.
# pylint: disable=R0914
module_identifier = getModuleIdentifier( module_name )
header = CodeTemplates.global_copyright % {
"name" : module_name,
"version" : Options.getVersion()
}
# Temp local variable initializations
local_var_inits = [
getLocalVariableInitCode(
context = context,
variable = variable
)
for variable in
temp_variables
# TODO: Should become uncessary to filter.
if variable.getNeedsFree() is not None
if not variable.needsLateDeclaration()
]
module_code = CodeTemplates.module_body_template % {
"module_name" : module_name,
"module_name_obj" : getConstantCode(
context = context,
constant = module_name
),
"module_identifier" : module_identifier,
"module_functions_decl" : function_decl_codes,
"module_functions_code" : function_body_codes,
"temps_decl" : indented( local_var_inits ),
"module_code" : indented( codes ),
"metapath_loader_inittab" : indented(
sorted( metapath_loader_inittab )
),
"use_unfreezer" : 1 if metapath_loader_inittab else 0
}
return header + module_code
Nuitka-0.5.0.1/nuitka/codegen/VariableCodes.py 0000644 0001750 0001750 00000025570 12265264105 021330 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Low level variable code generation.
"""
from nuitka import Variables
from .Identifiers import (
MaybeModuleVariableIdentifier,
ModuleVariableIdentifier,
TempObjectIdentifier,
NameIdentifier
)
from .ConstantCodes import getConstantCode
from .Indentation import (
getBlockCode,
)
def _getContextAccess(context, force_closure = False):
# Context access is variant depending on if that's a created function or
# not. For generators, they even share closure variables in the common
# context.
if context.isPythonModule():
return ""
else:
function = context.getFunction()
if function.needsCreation():
if function.isGenerator():
if force_closure:
return "_python_context->common_context->"
else:
return "_python_context->"
else:
if force_closure:
return "_python_context->"
else:
return ""
else:
if function.isGenerator():
return "_python_context->"
else:
return ""
def getVariableHandle(context, variable):
assert isinstance( variable, Variables.Variable ), variable
var_name = variable.getName()
if variable.isParameterVariable() or \
variable.isLocalVariable() or \
variable.isClassVariable() or \
( variable.isClosureReference() and not variable.isTempVariableReference() ):
from_context = _getContextAccess(
context = context,
force_closure = variable.isClosureReference()
)
code = from_context + variable.getCodeName()
if variable.isParameterVariable():
hint = "parameter"
elif variable.isClassVariable():
hint = "classvar"
elif variable.isClosureReference():
hint = "closure"
else:
hint = "localvar"
return NameIdentifier(
hint = hint,
name = var_name,
code = code,
ref_count = 0
)
elif variable.isTempVariableReference() and \
variable.isClosureReference() and \
variable.isShared( True ):
return TempVariableIdentifier(
var_name = var_name,
from_context = _getContextAccess(
context = context,
force_closure = True
)
)
elif variable.isTempVariableReference():
variable = variable.getReferenced()
if variable.isTempVariableReference():
variable = variable.getReferenced()
if variable.needsLateDeclaration():
from_context = ""
else:
from_context = _getContextAccess(
context,
variable.isShared( True )
)
if variable.isShared( True ):
return NameIdentifier(
hint = "tempvar",
name = var_name,
code = from_context + variable.getCodeName(),
ref_count = 0
)
elif not variable.getNeedsFree():
return TempObjectIdentifier(
var_name = var_name,
code = from_context + variable.getCodeName()
)
else:
return NameIdentifier(
hint = "tempvar",
name = var_name,
code = from_context + variable.getCodeName(),
ref_count = 0
)
elif variable.isMaybeLocalVariable():
assert context.getFunction().hasLocalsDict(), context
return MaybeModuleVariableIdentifier(
var_name = var_name,
var_code_name = getConstantCode(
context = context,
constant = var_name
)
)
elif variable.isModuleVariable():
return ModuleVariableIdentifier(
var_name = var_name,
var_code_name = getConstantCode(
context = context,
constant = var_name
)
)
else:
assert False, variable
def getVariableCode(context, variable):
var_identifier = getVariableHandle(
context = context,
variable = variable
)
return var_identifier.getCode()
def getLocalVariableInitCode( context, variable, init_from = None,
in_context = False ):
# This has many cases to deal with, so there need to be a lot of branches.
# pylint: disable=R0912
assert not variable.isModuleVariable()
assert init_from is None or hasattr( init_from, "getCodeTemporaryRef" )
result = variable.getDeclarationTypeCode( in_context )
# For pointer types, we don't have to separate with spaces.
if not result.endswith( "*" ):
result += " "
store_name = variable.getMangledName()
result += variable.getCodeName()
if not in_context:
if variable.isTempVariable():
assert init_from is None
if variable.isShared( True ):
result += "( NULL )"
elif not variable.getNeedsFree():
result += " = " + variable.getDeclarationInitValueCode()
else:
result += "( "
result += "%s" % getConstantCode(
context = context,
constant = store_name
)
if init_from is not None:
result += ", %s" % init_from.getCodeExportRef()
result += " )"
result += ";"
return result
def getVariableAssignmentCode(context, variable, identifier):
assert isinstance( variable, Variables.Variable ), variable
# This ought to be impossible to happen, as an assignment to an overflow
# variable would have made it a local one.
assert not variable.isMaybeLocalVariable()
if variable.isTempVariableReference():
referenced = variable.getReferenced()
if referenced.needsLateDeclaration() and not referenced.isDeclared():
referenced.markAsDeclared()
variable_code = getVariableCode(
variable = variable,
context = context
)
local_inits = context.getTempKeeperDecl()
if referenced.getNeedsFree():
prefix = "PyObject *_%s;" % variable_code
if local_inits:
result = "_%s = %s;" % (
variable_code,
identifier.getCodeExportRef()
)
result = getBlockCode(
local_inits + result.split( "\n" )
)
postfix = "%s %s( _%s );" % (
referenced.getDeclarationTypeCode( False ),
variable_code,
variable_code
)
return prefix + "\n" + result + "\n" + postfix
else:
return "%s %s( %s );" % (
referenced.getDeclarationTypeCode( False ),
variable_code,
identifier.getCodeExportRef()
)
else:
if local_inits:
prefix = "PyObject *%s;" % variable_code
result = "%s = %s;" % (
variable_code,
identifier.getCodeTemporaryRef()
)
result = getBlockCode(
local_inits + result.split( "\n" )
)
return prefix + "\n" + result
else:
return "PyObject *%s = %s;" % (
variable_code,
identifier.getCodeTemporaryRef()
)
if not referenced.isShared( True ) and not referenced.getNeedsFree():
# So won't get a reference, and take none, or else it may get lost,
# which we don't want to happen.
# This must be true, otherwise the needs no free statement was made
# in error.
assert identifier.getCheapRefCount() == 0
left_side = getVariableCode(
variable = variable,
context = context
)
assert "(" not in left_side
return "%s = %s;" % (
left_side,
identifier.getCodeTemporaryRef()
)
if identifier.getCheapRefCount() == 0:
identifier_code = identifier.getCodeTemporaryRef()
assign_code = "0"
else:
identifier_code = identifier.getCodeExportRef()
assign_code = "1"
if variable.isModuleVariable():
return "UPDATE_STRING_DICT%s( moduledict_%s, (Nuitka_StringObject *)%s, %s );" % (
assign_code,
context.getModuleCodeName(),
getConstantCode(
constant = variable.getName(),
context = context
),
identifier_code
)
return "%s.assign%s( %s );" % (
getVariableCode(
variable = variable,
context = context
),
assign_code,
identifier_code
)
def getVariableDelCode(context, tolerant, variable):
assert isinstance( variable, Variables.Variable ), variable
if variable.isModuleVariable():
return "DEL_MODULE_VALUE( %s, %s );" % (
getConstantCode(
context = context,
constant = variable.getName()
),
"true" if tolerant else "false"
)
else:
if variable.isTempVariableReference():
if not variable.isShared( True ) and \
not variable.getReferenced().getNeedsFree():
return "%s = NULL;" % (
getVariableCode(
variable = variable,
context = context
)
)
return "%s.del( %s );" % (
getVariableCode(
variable = variable,
context = context
),
"true" if tolerant else "false"
)
Nuitka-0.5.0.1/nuitka/codegen/FunctionCodes.py 0000644 0001750 0001750 00000057040 12265264105 021365 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code to generate and interact with compiled function objects.
"""
from .VariableCodes import (
getLocalVariableInitCode,
getVariableCode
)
from .ParameterParsing import (
getDirectFunctionEntryPointIdentifier,
getParameterEntryPointIdentifier,
getQuickEntryPointIdentifier,
getParameterParsingCode,
)
from .Identifiers import defaultToNullIdentifier
from .OrderedEvaluation import getOrderRelevanceEnforcedArgsCode
from .ConstantCodes import (
getConstantCode,
)
from .CodeObjectCodes import (
getCodeObjectHandle,
)
from .Indentation import indented
from .Identifiers import (
SpecialConstantIdentifier,
NullIdentifier,
Identifier
)
from .ModuleCodes import (
getModuleAccessCode,
)
from . import CodeTemplates
from nuitka import Utils
def getClosureVariableProvisionCode(context, closure_variables):
result = []
for variable in closure_variables:
assert variable.isClosureReference()
variable = variable.getProviderVariable()
result.append(
getVariableCode(
context = context,
variable = variable
)
)
return result
def _getFunctionCreationArgs( defaults_identifier, kw_defaults_identifier,
annotations_identifier, closure_variables ):
result = []
if not kw_defaults_identifier.isConstantIdentifier():
result.append( "PyObject *kwdefaults" )
if not defaults_identifier.isConstantIdentifier():
result.append( "PyObject *defaults" )
if annotations_identifier is not None and \
not annotations_identifier.isConstantIdentifier():
result.append( "PyObject *annotations" )
for closure_variable in closure_variables:
result.append(
"%s &%s" % (
( "PyObjectSharedTempVariable" if
closure_variable.isTempVariableReference() else
"PyObjectSharedLocalVariable" ),
closure_variable.getCodeName()
)
)
return result
def _getFuncDefaultValue(defaults_identifier):
if defaults_identifier.isConstantIdentifier():
return defaults_identifier
else:
return Identifier( "defaults", 1 )
def _getFuncKwDefaultValue(kw_defaults_identifier):
if kw_defaults_identifier.isConstantIdentifier():
if kw_defaults_identifier.getConstant():
return kw_defaults_identifier
else:
return SpecialConstantIdentifier( constant_value = None )
else:
return Identifier( "kwdefaults", 1 )
def _getFuncAnnotationsValue(annotations_identifier):
if annotations_identifier is None:
return NullIdentifier()
elif annotations_identifier.isConstantIdentifier():
return annotations_identifier
else:
return Identifier( "annotations", 1 )
def getFunctionMakerDecl( function_identifier, defaults_identifier,
kw_defaults_identifier, annotations_identifier,
closure_variables ):
function_creation_arg_spec = _getFunctionCreationArgs(
defaults_identifier = defaults_identifier,
kw_defaults_identifier = kw_defaults_identifier,
annotations_identifier = annotations_identifier,
closure_variables = closure_variables
)
return CodeTemplates.template_function_make_declaration % {
"function_identifier" : function_identifier,
"function_creation_arg_spec" : ", ".join(
function_creation_arg_spec
)
}
def getFunctionMakerCode( context, function_name, function_qualname,
function_identifier, parameters, local_variables,
closure_variables, defaults_identifier,
kw_defaults_identifier, annotations_identifier,
source_ref, function_doc, is_generator ):
# We really need this many parameters here. pylint: disable=R0913
# Functions have many details, that we express as variables
# pylint: disable=R0914
function_name_obj = getConstantCode(
context = context,
constant = function_name
)
var_names = parameters.getCoArgNames()
# Apply mangled names of local variables too.
var_names += [
local_variable.getMangledName()
for local_variable in
local_variables
if not local_variable.isParameterVariable()
]
code_identifier = getCodeObjectHandle(
context = context,
filename = source_ref.getFilename(),
var_names = var_names,
arg_count = parameters.getArgumentCount(),
kw_only_count = parameters.getKwOnlyParameterCount(),
line_number = source_ref.getLineNumber(),
code_name = function_name,
is_generator = is_generator,
is_optimized = not context.hasLocalsDict(),
has_starlist = parameters.getStarListArgumentName() is not None,
has_stardict = parameters.getStarDictArgumentName() is not None,
)
function_creation_args = _getFunctionCreationArgs(
defaults_identifier = defaults_identifier,
kw_defaults_identifier = kw_defaults_identifier,
annotations_identifier = annotations_identifier,
closure_variables = closure_variables,
)
func_defaults = _getFuncDefaultValue(
defaults_identifier = defaults_identifier
)
func_kwdefaults = _getFuncKwDefaultValue(
kw_defaults_identifier = kw_defaults_identifier
)
func_annotations = _getFuncAnnotationsValue(
annotations_identifier = annotations_identifier
)
if Utils.python_version < 330 or function_qualname == function_name:
function_qualname_obj = "NULL"
else:
function_qualname_obj = getConstantCode(
constant = function_qualname,
context = context
)
if closure_variables:
context_copy = []
for closure_variable in closure_variables:
context_copy.append(
"_python_context->%s.shareWith( %s );" % (
closure_variable.getCodeName(),
closure_variable.getCodeName()
)
)
if is_generator:
template = CodeTemplates.make_genfunc_with_context_template
else:
template = CodeTemplates.make_function_with_context_template
result = template % {
"function_name_obj" : function_name_obj,
"function_qualname_obj" : function_qualname_obj,
"function_identifier" : function_identifier,
"fparse_function_identifier" : getParameterEntryPointIdentifier(
function_identifier = function_identifier,
),
"dparse_function_identifier" : getQuickEntryPointIdentifier(
function_identifier = function_identifier,
parameters = parameters
),
"function_creation_args" : ", ".join(
function_creation_args
),
"code_identifier" : code_identifier.getCodeTemporaryRef(),
"context_copy" : indented( context_copy ),
"function_doc" : getConstantCode(
constant = function_doc,
context = context
),
"defaults" : func_defaults.getCodeExportRef(),
"kwdefaults" : func_kwdefaults.getCodeExportRef(),
"annotations" : func_annotations.getCodeExportRef(),
"module_identifier" : getModuleAccessCode(
context = context
),
}
else:
if is_generator:
template = CodeTemplates.make_genfunc_without_context_template
else:
template = CodeTemplates.make_function_without_context_template
result = template % {
"function_name_obj" : function_name_obj,
"function_qualname_obj" : function_qualname_obj,
"function_identifier" : function_identifier,
"fparse_function_identifier" : getParameterEntryPointIdentifier(
function_identifier = function_identifier,
),
"dparse_function_identifier" : getQuickEntryPointIdentifier(
function_identifier = function_identifier,
parameters = parameters
),
"function_creation_args" : ", ".join(
function_creation_args
),
"code_identifier" : code_identifier.getCodeTemporaryRef(),
"function_doc" : getConstantCode(
constant = function_doc,
context = context
),
"defaults" : func_defaults.getCodeExportRef(),
"kwdefaults" : func_kwdefaults.getCodeExportRef(),
"annotations" : func_annotations.getCodeExportRef(),
"module_identifier" : getModuleAccessCode(
context = context
),
}
return result
def getFunctionCreationCode( context, function_identifier, order_relevance,
default_args, closure_variables ):
return getOrderRelevanceEnforcedArgsCode(
helper = "MAKE_FUNCTION_%s" % function_identifier,
export_ref = 1,
ref_count = 1,
tmp_scope = "make_func",
suffix_args = getClosureVariableProvisionCode(
context = context,
closure_variables = closure_variables
),
order_relevance = order_relevance,
args = default_args,
context = context
)
def getDirectionFunctionCallCode( function_identifier, arguments,
order_relevance, closure_variables,
extra_arguments, context ):
function_identifier = getDirectFunctionEntryPointIdentifier(
function_identifier = function_identifier
)
return getOrderRelevanceEnforcedArgsCode(
helper = function_identifier,
export_ref = 1,
ref_count = 1,
tmp_scope = "call_tmp",
prefix_args = [
defaultToNullIdentifier(extra_argument).getCodeTemporaryRef()
for extra_argument in
extra_arguments
],
suffix_args = getClosureVariableProvisionCode(
context = context,
closure_variables = closure_variables
),
order_relevance = order_relevance,
args = arguments,
context = context
)
def getFunctionDirectDecl( function_identifier, closure_variables,
parameter_variables, file_scope ):
parameter_objects_decl = [
"PyObject *_python_par_" + variable.getName()
for variable in
parameter_variables
]
for closure_variable in closure_variables:
parameter_objects_decl.append(
closure_variable.getDeclarationCode()
)
result = CodeTemplates.template_function_direct_declaration % {
"file_scope" : file_scope,
"function_identifier" : function_identifier,
"direct_call_arg_spec" : ", ".join( parameter_objects_decl ),
}
return result
def getFunctionContextDefinitionCode( context, function_identifier,
closure_variables ):
context_decl = []
# Always empty now, but we may not use C++ destructors for everything in the
# future, so leave it.
context_free = []
for closure_variable in closure_variables:
context_decl.append(
getLocalVariableInitCode(
context = context,
variable = closure_variable,
in_context = True
)
)
return CodeTemplates.function_context_body_template % {
"function_identifier" : function_identifier,
"context_decl" : indented( context_decl ),
"context_free" : indented( context_free ),
}
def getFunctionCode( context, function_name, function_identifier, parameters,
closure_variables, user_variables, temp_variables,
function_codes, function_doc, file_scope ):
# Functions have many details, that we express as variables, with many
# branches to decide, pylint: disable=R0912,R0914
parameter_variables, entry_point_code, parameter_objects_decl = \
getParameterParsingCode(
function_identifier = function_identifier,
function_name = function_name,
parameters = parameters,
needs_creation = context.isForCreatedFunction(),
context = context,
)
function_parameter_decl = [
getLocalVariableInitCode(
context = context,
variable = variable,
init_from = Identifier( "_python_par_" + variable.getName(), 1 )
)
for variable in
parameter_variables
]
# User local variable initializations
local_var_inits = [
getLocalVariableInitCode(
context = context,
variable = variable
)
for variable in
user_variables + tuple(
variable
for variable in
temp_variables
if not variable.needsLateDeclaration()
# TODO: This filter should not be possible.
if variable.getNeedsFree() is not None
)
]
function_doc = getConstantCode(
context = context,
constant = function_doc
)
function_locals = []
if context.hasLocalsDict():
function_locals += CodeTemplates.function_dict_setup.split("\n")
function_locals += function_parameter_decl + local_var_inits
result = ""
if closure_variables and context.isForCreatedFunction():
context_access_function_impl = CodeTemplates.function_context_access_template % {
"function_identifier" : function_identifier,
}
else:
context_access_function_impl = str( CodeTemplates.function_context_unused_template )
if context.isForDirectCall():
for closure_variable in closure_variables:
parameter_objects_decl.append(
closure_variable.getDeclarationCode()
)
result += CodeTemplates.function_direct_body_template % {
"file_scope" : file_scope,
"function_identifier" : function_identifier,
"context_access_function_impl" : context_access_function_impl,
"direct_call_arg_spec" : ", ".join(
parameter_objects_decl
),
"function_locals" : indented( function_locals ),
"function_body" : indented( function_codes ),
}
else:
result += CodeTemplates.function_body_template % {
"function_identifier" : function_identifier,
"context_access_function_impl" : context_access_function_impl,
"parameter_objects_decl" : ", ".join( parameter_objects_decl ),
"function_locals" : indented( function_locals ),
"function_body" : indented( function_codes ),
}
if context.isForCreatedFunction():
result += entry_point_code
return result
def getGeneratorFunctionCode( context, function_name, function_identifier,
parameters, closure_variables, user_variables,
temp_variables, function_codes, source_ref,
function_doc ):
# We really need this many parameters here. pylint: disable=R0913
# Functions have many details, that we express as variables, with many
# branches to decide, pylint: disable=R0912,R0914,R0915
parameter_variables, entry_point_code, parameter_objects_decl = \
getParameterParsingCode(
function_identifier = function_identifier,
function_name = function_name,
parameters = parameters,
needs_creation = context.isForCreatedFunction(),
context = context,
)
context_decl = []
context_copy = []
context_free = []
function_parameter_decl = [
getLocalVariableInitCode(
context = context,
variable = variable,
in_context = True
)
for variable in
parameter_variables
]
parameter_context_assign = []
for variable in parameter_variables:
parameter_context_assign.append(
"_python_context->%s.setVariableNameAndValue( %s, _python_par_%s );" % (
variable.getCodeName(),
getConstantCode(
constant = variable.getName(),
context = context
),
variable.getName()
)
)
del variable
function_var_inits = []
local_var_decl = []
for user_variable in user_variables:
local_var_decl.append(
getLocalVariableInitCode(
context = context,
variable = user_variable,
in_context = True
)
)
function_var_inits.append(
"_python_context->%s.setVariableName( %s );" % (
user_variable.getCodeName(),
getConstantCode(
constant = user_variable.getName(),
context = context
)
)
)
for temp_variable in temp_variables:
assert temp_variable.isTempVariable(), variable
if temp_variable.needsLateDeclaration():
continue
# TODO: This filter should not be possible.
if temp_variable.getNeedsFree() is None:
continue
local_var_decl.append(
getLocalVariableInitCode(
context = context,
variable = temp_variable,
in_context = True
)
)
for closure_variable in closure_variables:
assert closure_variable.isShared()
context_decl.append(
getLocalVariableInitCode(
context = context,
variable = closure_variable,
in_context = True
)
)
context_copy.append(
"_python_context->%s.shareWith( %s );" % (
closure_variable.getCodeName(),
closure_variable.getCodeName()
)
)
function_doc = getConstantCode(
context = context,
constant = function_doc
)
function_name_obj = getConstantCode(
constant = function_name,
context = context,
)
instance_context_decl = function_parameter_decl + local_var_decl
if context.isForDirectCall():
instance_context_decl = context_decl + instance_context_decl
context_decl = []
if context_decl:
result = CodeTemplates.genfunc_context_body_template % {
"function_identifier" : function_identifier,
"function_common_context_decl" : indented( context_decl ),
"function_instance_context_decl" : indented( instance_context_decl ),
"context_free" : indented( context_free, 2 ),
}
elif instance_context_decl:
result = CodeTemplates.genfunc_context_local_only_template % {
"function_identifier" : function_identifier,
"function_instance_context_decl" : indented( instance_context_decl )
}
else:
result = ""
if instance_context_decl or context_decl:
context_access_instance = CodeTemplates.generator_context_access_template2 % {
"function_identifier" : function_identifier
}
else:
context_access_instance = ""
function_locals = []
if context.hasLocalsDict():
function_locals += CodeTemplates.function_dict_setup.split( "\n" )
function_locals += function_var_inits
result += CodeTemplates.genfunc_yielder_template % {
"function_identifier" : function_identifier,
"function_body" : indented( function_codes, 2 ),
"function_var_inits" : indented( function_locals, 2 ),
"context_access" : indented( context_access_instance, 2 ),
}
code_identifier = getCodeObjectHandle(
context = context,
filename = source_ref.getFilename(),
var_names = parameters.getCoArgNames(),
arg_count = parameters.getArgumentCount(),
kw_only_count = parameters.getKwOnlyParameterCount(),
line_number = source_ref.getLineNumber(),
code_name = function_name,
is_generator = True,
is_optimized = not context.hasLocalsDict(),
has_starlist = parameters.getStarListArgumentName() is not None,
has_stardict = parameters.getStarDictArgumentName() is not None,
)
if context_decl or instance_context_decl:
if context_decl:
context_making = CodeTemplates.genfunc_common_context_use_template % {
"function_identifier" : function_identifier,
}
else:
context_making = CodeTemplates.genfunc_local_context_use_template % {
"function_identifier" : function_identifier,
}
context_making = context_making.split( "\n" )
if context.isForDirectCall():
context_making += context_copy
generator_making = CodeTemplates.genfunc_generator_with_context_making % {
"function_name_obj" : function_name_obj,
"function_identifier" : function_identifier,
"code_identifier" : code_identifier.getCodeTemporaryRef()
}
else:
generator_making = CodeTemplates.genfunc_generator_without_context_making % {
"function_name_obj" : function_name_obj,
"function_identifier" : function_identifier,
"code_identifier" : code_identifier.getCodeTemporaryRef()
}
context_making = []
if context.isForDirectCall():
for closure_variable in closure_variables:
parameter_objects_decl.append(
closure_variable.getDeclarationCode()
)
result += CodeTemplates.genfunc_function_maker_template % {
"function_name" : function_name,
"function_identifier" : function_identifier,
"context_making" : indented( context_making, 1 ),
"context_copy" : indented( parameter_context_assign, 2 ),
"generator_making" : generator_making,
"parameter_objects_decl" : ", ".join( parameter_objects_decl ),
}
if context.isForCreatedFunction():
result += entry_point_code
return result
Nuitka-0.5.0.1/nuitka/codegen/SetCodes.py 0000644 0001750 0001750 00000005374 12265264105 020336 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code generation for sets.
Right now only the creation is done here. But more should be added later on.
"""
from .Identifiers import ConstantIdentifier
from . import CodeTemplates
# Take note of the MAKE_SET element count variants actually used.
make_sets_used = set()
def addMakeSetUse(value):
assert type( value ) is int
make_sets_used.add( value )
def getSetCreationCode(context, order_relevance, element_identifiers):
if len( element_identifiers ) == 0:
return ConstantIdentifier( "_python_set_empty", set() )
from .OrderedEvaluation import getOrderRelevanceEnforcedArgsCode
args_length = len( element_identifiers )
addMakeSetUse( args_length )
return getOrderRelevanceEnforcedArgsCode(
helper = "MAKE_SET%d" % args_length,
export_ref = 0,
ref_count = 1,
tmp_scope = "make_set",
order_relevance = order_relevance,
args = element_identifiers,
context = context
)
def getMakeSetsCode():
make_sets_codes = []
for arg_count in sorted( make_sets_used ):
add_elements_code = []
for arg_index in range( arg_count ):
add_elements_code.append(
CodeTemplates.template_add_set_element_code % {
"set_value" : "element%d" % arg_index
}
)
make_sets_codes.append(
CodeTemplates.template_make_set_function % {
"argument_count" : arg_count,
"argument_decl" : ", ".join(
"PyObject *element%d" % arg_index
for arg_index in
range( arg_count )
),
"add_elements_code" : "\n".join( add_elements_code ),
}
)
# TODO: Why is this not a helper function.
return CodeTemplates.template_header_guard % {
"header_guard_name" : "__NUITKA_SETS_H__",
"header_body" : "\n".join( make_sets_codes )
}
Nuitka-0.5.0.1/nuitka/codegen/MainCodes.py 0000644 0001750 0001750 00000007170 12265270763 020472 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Main program code generation.
This is for the actual entry point code, which is mostly fed by a template,
but also customized through a lot of runtime configuration values.
Examples of these are sys.executable, and sys.flags, but of course also the
frame object data (filename, etc).
"""
from .ConstantCodes import getConstantCode
from .CodeObjectCodes import getCodeObjectHandle
from .Identifiers import NullIdentifier
from . import CodeTemplates
from nuitka import Options, Utils
import sys
def getMainCode(codes, context):
python_flags = Options.getPythonFlags()
if context.isEmptyModule():
code_identifier = NullIdentifier()
else:
code_identifier = getCodeObjectHandle(
context = context,
filename = context.getFilename(),
var_names = (),
arg_count = 0,
kw_only_count = 0,
line_number = 0,
code_name = "",
is_generator = False,
is_optimized = False,
has_starlist = False,
has_stardict = False
)
main_code = CodeTemplates.main_program % {
"sys_executable" : getConstantCode(
constant = "python.exe"
if Utils.getOS() == "Windows" and \
Options.isStandaloneMode() else
sys.executable,
context = context
),
"python_sysflag_debug" : sys.flags.debug,
"python_sysflag_py3k_warning" : ( sys.flags.py3k_warning
if hasattr( sys.flags, "py3k_warning" ) else 0 ),
"python_sysflag_division_warning" : ( sys.flags.division_warning
if hasattr( sys.flags, "division_warning" ) else 0 ),
#"python_sysflag_division_new" : sys.flags.division_new, #not supported
"python_sysflag_inspect" : sys.flags.inspect,
"python_sysflag_interactive" : sys.flags.interactive,
"python_sysflag_optimize" : sys.flags.optimize,
"python_sysflag_dont_write_bytecode" : sys.flags.dont_write_bytecode,
"python_sysflag_no_site" : sys.flags.no_site,
"python_sysflag_no_user_site" : sys.flags.no_user_site,
"python_sysflag_ignore_environment" : sys.flags.ignore_environment,
"python_sysflag_tabcheck" : ( sys.flags.tabcheck
if hasattr( sys.flags, "tabcheck" ) else 0 ),
"python_sysflag_verbose" : 1 if "trace_imports" in python_flags else 0,
"python_sysflag_unicode" : ( sys.flags.unicode
if hasattr( sys.flags, "unicode" ) else 0 ),
"python_sysflag_bytes_warning" : sys.flags.bytes_warning,
"python_sysflag_hash_randomization" : ( sys.flags.hash_randomization
if hasattr( sys.flags, "hash_randomization" ) else 0 ),
"code_identifier" : code_identifier.getCodeTemporaryRef()
}
return codes + main_code
Nuitka-0.5.0.1/nuitka/codegen/OrderedEvaluation.py 0000644 0001750 0001750 00000016423 12265264105 022236 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Generate code that enforces ordered evaluation.
"""
from .Identifiers import (
getCodeTemporaryRefs,
getCodeExportRefs,
Identifier
)
from .Indentation import getBlockCode
from .LineNumberCodes import getLineNumberCode
def pickFirst(order_relevance):
for value in order_relevance:
if value:
return value
else:
return None
def _getAssignmentTempKeeperCode(source_identifier, variable_name, context):
ref_count = source_identifier.getCheapRefCount()
context.addTempKeeperUsage( variable_name, ref_count )
return Identifier(
"%s.assign( %s )" % (
variable_name,
source_identifier.getCodeExportRef()
if ref_count else
source_identifier.getCodeTemporaryRef()
),
0
)
def getOrderRelevanceEnforcedArgsCode( helper, tmp_scope, order_relevance, args,
export_ref, context, ref_count,
prefix_args = None, suffix_args = None ):
# Preserving order with line numbers, and special purpose arguments needs
# many variables, with many branches to decide, pylint: disable=R0912,R0914
if prefix_args is None:
prefix_args = []
if suffix_args is None:
suffix_args = []
assert len( args ) == len( order_relevance )
if order_relevance.count( None ) <= len( args ) - 2:
order_relevance, source_refs = _getMinimalOrderRelevance(
order_relevance
)
assert len( args ) == len( source_refs )
order_codes = []
arg_codes = []
for argument, order_relevant, source_ref in zip(
args, order_relevance, source_refs ):
if source_ref is not None:
line_code = getLineNumberCode( source_ref )
if line_code:
order_codes.append(
line_code
)
if order_relevant:
variable_name = "%s%d" % (
tmp_scope,
context.allocateTempNumber( tmp_scope )
)
order_codes.append(
_getAssignmentTempKeeperCode(
source_identifier = argument,
variable_name = variable_name,
context = context
).getCode()
)
arg_codes.append(
"%s.asObject%d()" % (
variable_name,
1 if export_ref else 0
)
)
else:
# TODO: Should delete the reference immediately after call, if
# ref_count = 1
if export_ref:
arg_codes.append( argument.getCodeExportRef() )
else:
arg_codes.append( argument.getCodeTemporaryRef() )
order_codes.append(
"%s( %s )" % (
helper,
", ".join( list(prefix_args) + arg_codes + list( suffix_args ) )
)
)
code = "( %s )" % ", ".join( order_codes )
if ref_count is None:
return code
else:
return Identifier(
code,
ref_count = ref_count
)
else:
if export_ref:
arg_codes = getCodeExportRefs( args )
else:
arg_codes = getCodeTemporaryRefs( args )
code = "%s( %s )" % (
helper,
", ".join( list(prefix_args) + arg_codes + list(suffix_args) )
)
if ref_count is None:
return code
else:
return Identifier(
code,
ref_count
)
def _getMinimalOrderRelevance(order_relevance):
last_true = None
source_refs = []
for count, value in enumerate( order_relevance ):
if value:
last_true = count
if value.shallSetCurrentLine():
source_refs.append( value )
else:
source_refs.append( None )
else:
source_refs.append( None )
if last_true is not None:
new_order_relevance = list( order_relevance )
new_order_relevance[ last_true ] = None
order_relevance = new_order_relevance
return order_relevance, source_refs
def _getTempDeclCode(order_relevance, names, values):
assert len( names ) == len( values )
assert len( names ) == len( order_relevance )
order_relevance, source_refs = _getMinimalOrderRelevance( order_relevance )
usages = []
decls = []
scoped = False
for name, value, order_relevant, source_ref in zip( names,
values,
order_relevance,
source_refs ):
if source_ref is not None:
line_code = getLineNumberCode( source_ref )
if line_code:
decls.append(
line_code + ";"
)
if not order_relevant:
usages.append( value.getCodeTemporaryRef() )
else:
scoped = True
tmp_name = "tmp_" + name
if value.getRefCount() == 1:
decls.append(
"PyObjectTemporary %s( %s );" % (
tmp_name,
value.getCodeExportRef()
)
)
usages.append( tmp_name + ".asObject0()" )
else:
decls.append(
"PyObject *%s = %s;" % (
tmp_name,
value.getCodeTemporaryRef()
)
)
usages.append( tmp_name )
return scoped, decls, usages
def getOrderRelevanceEnforcedCallCode( helper, order_relevance, names, values,
prefix_args = None, suffix_args = None ):
if prefix_args is None:
prefix_args = []
if suffix_args is None:
suffix_args = []
scoped, decls, usages = _getTempDeclCode( order_relevance, names, values )
args = prefix_args + usages + suffix_args
if decls:
code = decls + [ "%s( %s );" % ( helper, ", ".join( args ) ) ]
if scoped:
return getBlockCode(
code
)
else:
return "\n".join( code )
else:
return "%s( %s );" % ( helper, ", ".join( args ) )
Nuitka-0.5.0.1/nuitka/codegen/OperatorCodes.py 0000644 0001750 0001750 00000005305 12265264105 021370 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Operator code tables
These are mostly used to look up the Python C/API from operations or a wrapper used.
"""
binary_operator_codes = {
# Those commented out in this section have fully specialized variants already.
# "Add" : "PyNumber_Add",
# "Sub" : "PyNumber_Subtract",
# "Div" : "PyNumber_Divide",
# "Mult" : "PyNumber_Multiply",
# "Mod" : "PyNumber_Remainder",
# These have their own variants only to make sure the generic code is inlined
# but the CPython code is not inlined.
# "Pow" : "PyNumber_Power",
# "IPow" : "PyNumber_InPlacePower",
# The others are generic code and would be faster if they had a specialized variant too.
"FloorDiv" : "PyNumber_FloorDivide",
"TrueDiv" : "PyNumber_TrueDivide",
"LShift" : "PyNumber_Lshift",
"RShift" : "PyNumber_Rshift",
"BitAnd" : "PyNumber_And",
"BitOr" : "PyNumber_Or",
"BitXor" : "PyNumber_Xor",
"IAdd" : "PyNumber_InPlaceAdd",
"ISub" : "PyNumber_InPlaceSubtract",
"IMult" : "PyNumber_InPlaceMultiply",
"IDiv" : "PyNumber_InPlaceDivide",
"IFloorDiv" : "PyNumber_InPlaceFloorDivide",
"ITrueDiv" : "PyNumber_InPlaceTrueDivide",
"IMod" : "PyNumber_InPlaceRemainder",
"ILShift" : "PyNumber_InPlaceLshift",
"IRShift" : "PyNumber_InPlaceRshift",
"IBitAnd" : "PyNumber_InPlaceAnd",
"IBitOr" : "PyNumber_InPlaceOr",
"IBitXor" : "PyNumber_InPlaceXor",
}
unary_operator_codes = {
"UAdd" : ( "PyNumber_Positive", 1 ),
"USub" : ( "PyNumber_Negative", 1 ),
"Invert" : ( "PyNumber_Invert", 1 ),
"Repr" : ( "PyObject_Repr", 1 ),
"Not" : ( "UNARY_NOT", 0 )
}
rich_comparison_codes = {
"Lt" : "LT",
"LtE" : "LE",
"Eq" : "EQ",
"NotEq" : "NE",
"Gt" : "GT",
"GtE" : "GE"
}
normal_comparison_codes = {
"In" : "SEQUENCE_CONTAINS",
"NotIn" : "SEQUENCE_CONTAINS_NOT"
}
Nuitka-0.5.0.1/nuitka/codegen/__init__.py 0000644 0001750 0001750 00000001501 12265264105 020350 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Dummy file to make this directory a package. """
Nuitka-0.5.0.1/nuitka/codegen/ParameterParsing.py 0000644 0001750 0001750 00000030756 12265264105 022073 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Low level code generation for parameter parsing.
"""
from . import CodeTemplates
from .ConstantCodes import getConstantCode
from .Identifiers import Identifier
from .Indentation import indented
from nuitka.Utils import python_version
def getParameterEntryPointIdentifier(function_identifier):
return "fparse_" + function_identifier
def getQuickEntryPointIdentifier(function_identifier, parameters):
if parameters.hasNestedParameterVariables() or \
parameters.getKwOnlyParameterCount() > 0:
return "NULL"
else:
return "dparse_" + function_identifier
def getDirectFunctionEntryPointIdentifier(function_identifier):
return "impl_" + function_identifier
def _getParameterParsingCode(context, parameters, function_name):
# There is really no way this could be any less complex.
# pylint: disable=R0912,R0914
# First, declare all parameter objects as variables.
parameter_parsing_code = "".join(
[
"PyObject *_python_par_" + variable.getName() + " = NULL;\n"
for variable in
parameters.getAllVariables()
]
)
top_level_parameters = parameters.getTopLevelVariables()
# Max allowed number of positional arguments, all except keyword only
# arguments.
plain_possible_count = len( top_level_parameters ) - \
parameters.getKwOnlyParameterCount()
if top_level_parameters:
parameter_parsing_code += "// Copy given dictionary values to the the respective variables:\n"
if parameters.getDictStarArgVariable() is not None:
# In the case of star dict arguments, we need to check what is for it
# and which arguments with names we have.
parameter_parsing_code += CodeTemplates.parse_argument_template_dict_star_copy % {
"dict_star_parameter_name" : parameters.getStarDictArgumentName(),
"function_name" : function_name,
}
# Check for each variable.
for variable in top_level_parameters:
if not variable.isNestedParameterVariable():
parameter_parsing_code += CodeTemplates.parse_argument_template_check_dict_parameter_with_star_dict % {
"parameter_name" : variable.getName(),
"parameter_name_object" : getConstantCode(
constant = variable.getName(),
context = context
),
"dict_star_parameter_name" : parameters.getStarDictArgumentName(),
}
elif not parameters.isEmpty():
quick_path_code = ""
slow_path_code = ""
for variable in top_level_parameters:
# Only named ones can be assigned from the dict.
if variable.isNestedParameterVariable():
continue
parameter_name_object = getConstantCode(
constant = variable.getName(),
context = context
)
parameter_assign_from_kw = CodeTemplates.argparse_template_assign_from_dict_finding % {
"parameter_name" : variable.getName(),
}
if variable.isParameterVariableKwOnly():
assign_quick = CodeTemplates.argparse_template_assign_from_dict_parameter_quick_path_kw_only
assign_slow = CodeTemplates.argparse_template_assign_from_dict_parameter_slow_path_kw_only
else:
assign_quick = CodeTemplates.argparse_template_assign_from_dict_parameter_quick_path
assign_slow = CodeTemplates.argparse_template_assign_from_dict_parameter_slow_path
quick_path_code += assign_quick % {
"parameter_name_object" : parameter_name_object,
"parameter_assign_from_kw" : indented( parameter_assign_from_kw )
}
slow_path_code += assign_slow % {
"parameter_name_object" : parameter_name_object,
"parameter_assign_from_kw" : indented( parameter_assign_from_kw )
}
parameter_parsing_code += CodeTemplates.argparse_template_assign_from_dict_parameters % {
"function_name" : function_name,
"parameter_quick_path" : indented( quick_path_code, 2 ),
"parameter_slow_path" : indented( slow_path_code, 2 )
}
if parameters.isEmpty():
parameter_parsing_code += CodeTemplates.template_parameter_function_refuses % {}
elif python_version < 330:
if parameters.getListStarArgVariable() is None:
parameter_parsing_code += CodeTemplates.parse_argument_template_check_counts_without_list_star_arg % {
"top_level_parameter_count" : plain_possible_count,
}
if plain_possible_count > 0:
plain_var_names = []
parameter_parsing_code += CodeTemplates.parse_argument_usable_count % {}
for count, variable in enumerate( top_level_parameters ):
if variable.isNestedParameterVariable():
parameter_parsing_code += CodeTemplates.argparse_template_nested_argument % {
"parameter_name" : variable.getName(),
"parameter_position" : count,
"top_level_parameter_count" : plain_possible_count,
}
elif not variable.isParameterVariableKwOnly():
parameter_parsing_code += CodeTemplates.argparse_template_plain_argument % {
"parameter_name" : variable.getName(),
"parameter_position" : count,
"top_level_parameter_count" : plain_possible_count,
}
plain_var_names.append( "_python_par_" + variable.getName() )
parameter_parsing_code += CodeTemplates.template_arguments_check % {
"parameter_test" : " || ".join(
"%s == NULL" % plain_var_name
for plain_var_name in
plain_var_names
),
"parameter_list" : ", ".join( plain_var_names )
}
if parameters.getListStarArgVariable() is not None:
parameter_parsing_code += CodeTemplates.parse_argument_template_copy_list_star_args % {
"list_star_parameter_name" : parameters.getStarListArgumentName(),
"top_level_parameter_count" : plain_possible_count
}
elif python_version >= 330:
parameter_parsing_code += CodeTemplates.parse_argument_template_check_counts_without_list_star_arg % {
"top_level_parameter_count" : plain_possible_count,
}
def unPackNestedParameterVariables(variables):
result = ""
for count, variable in enumerate( variables ):
if variable.isNestedParameterVariable():
assign_source = Identifier(
"_python_par_%s" % variable.getName(),
0
)
unpack_code = ""
child_variables = variable.getTopLevelVariables()
for count, child_variable in enumerate( child_variables ):
unpack_code += CodeTemplates.parse_argument_template_nested_argument_assign % {
"parameter_name" : child_variable.getName(),
"iter_name" : variable.getName(),
"unpack_count" : count
}
result += CodeTemplates.parse_argument_template_nested_argument_unpack % {
"unpack_source_identifier" : assign_source.getCode(),
"parameter_name" : variable.getName(),
"unpack_code" : unpack_code
}
for variable in variables:
if variable.isNestedParameterVariable():
result += unPackNestedParameterVariables(
variables = variable.getTopLevelVariables()
)
return result
parameter_parsing_code += unPackNestedParameterVariables(
variables = top_level_parameters
)
kw_only_var_names = []
for variable in parameters.getKwOnlyVariables():
parameter_parsing_code += CodeTemplates.template_kwonly_argument_default % {
"function_name" : function_name,
"parameter_name" : variable.getName(),
"parameter_name_object" : getConstantCode(
constant = variable.getName(),
context = context
)
}
kw_only_var_names.append( "_python_par_" + variable.getName() )
if kw_only_var_names:
parameter_parsing_code += CodeTemplates.template_kwonly_arguments_check % {
"parameter_test" : " || ".join(
"%s == NULL" % kw_only_var_name
for kw_only_var_name in
kw_only_var_names
),
"parameter_list" : ", ".join( kw_only_var_names )
}
return indented( parameter_parsing_code )
def getParameterParsingCode( context, function_identifier, function_name,
parameters, needs_creation ):
function_parameter_variables = parameters.getVariables()
if function_parameter_variables:
parameter_objects_decl = [
"PyObject *_python_par_" + variable.getName()
for variable in
function_parameter_variables
]
parameter_objects_list = [
"_python_par_" + variable.getName()
for variable in
function_parameter_variables
]
else:
parameter_objects_decl = []
parameter_objects_list = []
if needs_creation:
parameter_objects_decl.insert( 0, "Nuitka_FunctionObject *self" )
parameter_objects_list.insert( 0, "self" )
parameter_release_code = "".join(
[
" Py_XDECREF( _python_par_" + variable.getName() + " );\n"
for variable in
parameters.getAllVariables()
if not variable.isNestedParameterVariable()
]
)
parameter_entry_point_code = CodeTemplates.template_parameter_function_entry_point % {
"parameter_parsing_code" : _getParameterParsingCode(
context = context,
function_name = function_name,
parameters = parameters,
),
"parse_function_identifier" : getParameterEntryPointIdentifier(
function_identifier = function_identifier,
),
"impl_function_identifier" : getDirectFunctionEntryPointIdentifier(
function_identifier = function_identifier
),
"parameter_objects_list" : ", ".join( parameter_objects_list ),
"parameter_release_code" : parameter_release_code,
}
if not parameters.hasNestedParameterVariables() and \
not parameters.getKwOnlyParameterCount() > 0:
args_forward = []
count = -1
for count, variable in enumerate( parameters.getTopLevelVariables() ):
args_forward.append(
", INCREASE_REFCOUNT( args[ %d ] )" % count
)
if parameters.getListStarArgVariable() is not None:
count += 1
args_forward.append(
", MAKE_TUPLE( &args[ %d ], size > %d ? size-%d : 0 )" % (
count, count, count
)
)
if parameters.getDictStarArgVariable() is not None:
args_forward.append(
", PyDict_New()"
)
# print args_forward
parameter_entry_point_code += CodeTemplates.template_dparser % {
"function_identifier" : function_identifier,
"arg_count" : len( function_parameter_variables ),
"args_forward" : "".join( args_forward )
}
return (
function_parameter_variables,
parameter_entry_point_code,
parameter_objects_decl
)
Nuitka-0.5.0.1/nuitka/codegen/TupleCodes.py 0000644 0001750 0001750 00000005661 12265264105 020673 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code generation for tuples.
Right now only the creation is done here. But more should be added later on.
"""
from .Identifiers import ConstantIdentifier
from . import CodeTemplates
# Take note of the MAKE_TUPLE variants actually used, but have EVAL_ORDER for
# 1..6 in any case, so we can use it in the C++ code freely without concern.
make_tuples_used = set( range( 1, 6 ) )
def addMakeTupleUse(value):
assert type( value ) is int
make_tuples_used.add( value )
def getTupleCreationCode(context, order_relevance, element_identifiers):
if len( element_identifiers ) == 0:
return ConstantIdentifier( "const_tuple_empty", () )
from .OrderedEvaluation import getOrderRelevanceEnforcedArgsCode
args_length = len( element_identifiers )
addMakeTupleUse( args_length )
return getOrderRelevanceEnforcedArgsCode(
helper = "MAKE_TUPLE%d" % args_length,
export_ref = 0,
ref_count = 1,
tmp_scope = "make_tuple",
order_relevance = order_relevance,
args = element_identifiers,
context = context
)
def getMakeTuplesCode():
make_tuples_codes = []
for arg_count in sorted( make_tuples_used ):
add_elements_code = []
for arg_index in range( arg_count ):
add_elements_code.append(
CodeTemplates.template_add_tuple_element_code % {
"tuple_index" : arg_index,
"tuple_value" : "element%d" % arg_index
}
)
make_tuples_codes.append(
CodeTemplates.template_make_tuple_function % {
"argument_count" : arg_count,
"argument_decl" : ", ".join(
"PyObject *element%d" % arg_index
for arg_index in
range( arg_count )
),
"add_elements_code" : "\n".join( add_elements_code ),
}
)
# TODO: Why is this not a helper function.
return CodeTemplates.template_header_guard % {
"header_guard_name" : "__NUITKA_TUPLES_H__",
"header_body" : "\n".join( make_tuples_codes )
}
Nuitka-0.5.0.1/nuitka/codegen/Contexts.py 0000644 0001750 0001750 00000030142 12265264105 020423 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code generation contexts.
"""
from .Identifiers import (
SpecialConstantIdentifier,
ConstantIdentifier,
Identifier
)
from .Namify import namifyConstant
from nuitka.Constants import HashableConstant, constant_builtin_types
from nuitka.Utils import python_version
from nuitka import Options
from ..__past__ import iterItems
# Many methods won't use self, but it's the interface. pylint: disable=R0201
class PythonContextBase:
def __init__(self):
self.try_count = 0
self.try_finally_counts = []
self.temp_counts = {}
def isPythonModule(self):
return False
def hasLocalsDict(self):
return False
def allocateTryNumber(self):
self.try_count += 1
return self.try_count
def setTryFinallyCount(self, value):
self.try_finally_counts.append( value )
def removeFinallyCount(self):
del self.try_finally_counts[-1]
def getTryFinallyCount(self):
if self.try_finally_counts:
return self.try_finally_counts[-1]
else:
return None
def allocateTempNumber(self, tmp_scope):
result = self.temp_counts.get( tmp_scope, 0 ) + 1
self.temp_counts[ tmp_scope ] = result
return result
class PythonChildContextBase(PythonContextBase):
def __init__(self, parent):
PythonContextBase.__init__( self )
self.parent = parent
def getConstantHandle(self, constant):
return self.parent.getConstantHandle( constant )
def getModuleCodeName(self):
return self.parent.getModuleCodeName()
def getModuleName(self):
return self.parent.getModuleName()
def addHelperCode(self, key, code):
self.parent.addHelperCode( key, code )
def addDeclaration(self, key, code):
self.parent.addDeclaration( key, code )
def _getConstantDefaultPopulation():
result = (
# Basic values that the helper code uses all the times.
(),
{},
"",
True,
False,
0,
1,
# For Python3 empty bytes, no effect for Python2, same as "", used for
# code objects.
b"",
# Python mechanics, used in various helpers.
"__module__",
"__class__",
"__name__",
"__metaclass__",
"__dict__",
"__doc__",
"__file__",
"__enter__",
"__exit__",
"__builtins__",
"__all__",
"__cmp__",
# Patched module name.
"inspect",
# Names of builtins used in helper code.
"compile",
"range",
"open",
"__import__",
# COMPILE_CODE uses read/strip method lookups.
"read",
"strip",
)
# For Python3 modules
if python_version >= 300:
result += ( "__cached__", )
# For Python3 print
if python_version >= 300:
result += ( "print", "end", "file" )
if python_version >= 330:
result += ( "__loader__", )
# For patching Python2 internal class type
if python_version < 300:
result += (
"__getattr__",
"__setattr__",
"__delattr__"
)
# For patching Python2 sys attributes for current exception
if python_version < 300:
result += (
"exc_type",
"exc_value",
"exc_traceback"
)
# The xrange built-in is Python2 only.
if python_version < 300:
result += (
"xrange",
)
# Executables only
if not Options.shallMakeModule():
result += ( "__main__", )
# The "site" module is referenced in inspect patching.
result += ( "site", )
# Builtin original values
if not Options.shallMakeModule():
result += (
"type",
"len",
"range",
"repr",
"int",
"iter",
)
if python_version < 300:
result += (
"long",
)
return result
class PythonGlobalContext:
def __init__(self):
self.constants = {}
for value in _getConstantDefaultPopulation():
self.getConstantHandle( value )
def getConstantHandle(self, constant, real_use = True):
# There are many branches, each supposed to return.
# pylint: disable=R0911
if constant is None:
return SpecialConstantIdentifier( None )
elif constant is True:
return SpecialConstantIdentifier( True )
elif constant is False:
return SpecialConstantIdentifier( False )
elif constant is Ellipsis:
return SpecialConstantIdentifier( Ellipsis )
elif constant in constant_builtin_types:
type_name = constant.__name__
if constant is int and python_version >= 300:
type_name = "long"
if constant is str and python_version < 300:
type_name = "string"
if constant is str and python_version > 300:
type_name = "unicode"
return Identifier(
"(PyObject *)&Py%s_Type" % type_name.title(),
0
)
else:
if real_use:
key = ( type( constant ), HashableConstant( constant ) )
if key not in self.constants:
self.constants[ key ] = "const_" + namifyConstant(
constant
)
return ConstantIdentifier( self.constants[ key ], constant )
else:
return Identifier( "const_" + namifyConstant( constant ), 0 )
def getConstants(self):
return self.constants
class PythonModuleContext(PythonContextBase):
# Plent of attributes, because it's storing so many different things.
# pylint: disable=R0902
def __init__( self, module_name, code_name, filename, is_empty,
global_context ):
PythonContextBase.__init__( self )
self.name = module_name
self.code_name = code_name
self.filename = filename
self.is_empty = is_empty
self.global_context = global_context
self.declaration_codes = {}
self.helper_codes = {}
def __repr__(self):
return "" % self.filename
def isPythonModule(self):
return True
def getFrameHandle(self):
return Identifier( "frame_guard.getFrame()", 1 )
def getFrameGuardClass(self):
return "FrameGuard"
def getConstantHandle(self, constant):
return self.global_context.getConstantHandle(constant)
def getName(self):
return self.name
def getFilename(self):
return self.filename
def isEmptyModule(self):
return self.is_empty
getModuleName = getName
def getModuleCodeName(self):
return self.code_name
# There cannot be local variable in modules no need to consider the name.
# pylint: disable=W0613
def hasLocalVariable(self, var_name):
return False
def hasClosureVariable(self, var_name):
return False
# pylint: enable=W0613
def setFrameGuardMode(self, guard_mode):
assert guard_mode == "once"
def getReturnErrorCode(self):
return "return MOD_RETURN_VALUE( NULL );"
def addHelperCode(self, key, code):
assert key not in self.helper_codes, key
self.helper_codes[ key ] = code
def getHelperCodes(self):
return self.helper_codes
def addDeclaration(self, key, code):
assert key not in self.declaration_codes
self.declaration_codes[ key ] = code
def getDeclarations(self):
return self.declaration_codes
class PythonFunctionContext(PythonChildContextBase):
def __init__(self, parent, function):
PythonChildContextBase.__init__(
self,
parent = parent
)
self.function = function
# Make sure the local names are available as constants
for local_name in function.getLocalVariableNames():
self.getConstantHandle(
constant = local_name
)
self.guard_mode = None
def __repr__(self):
return "" % (
"function" if not self.function.isClassDictCreation() else "class",
self.function.getName()
)
def getFunction(self):
return self.function
def hasLocalsDict(self):
return self.function.hasLocalsDict()
def hasLocalVariable(self, var_name):
return var_name in self.function.getLocalVariableNames()
def hasClosureVariable(self, var_name):
return var_name in self.function.getClosureVariableNames()
def getFrameHandle(self):
if self.function.isGenerator():
return Identifier(
"generator->m_frame",
0
)
else:
return Identifier(
"frame_guard.getFrame()",
1
)
def getFrameGuardMode(self):
return self.guard_mode
def setFrameGuardMode(self, guard_mode):
self.guard_mode = guard_mode
def getFrameGuardClass(self):
if self.guard_mode == "generator":
return "FrameGuardLight"
elif self.guard_mode == "full":
return "FrameGuard"
elif self.guard_mode == "pass_through":
return "FrameGuardVeryLight"
else:
assert False, (self, self.guard_mode)
class PythonFunctionDirectContext(PythonFunctionContext):
def isForDirectCall(self):
return True
def isForCrossModuleUsage(self):
return self.function.isCrossModuleUsed()
def isForCreatedFunction(self):
return False
class PythonFunctionCreatedContext(PythonFunctionContext):
def isForDirectCall(self):
return False
def isForCreatedFunction(self):
return True
class PythonStatementContext(PythonChildContextBase):
def __init__(self, parent):
PythonChildContextBase.__init__( self, parent = parent )
self.temp_keepers = {}
def getFrameHandle(self):
return self.parent.getFrameHandle()
def getFrameGuardClass(self):
return self.parent.getFrameGuardClass()
def hasLocalsDict(self):
return self.parent.hasLocalsDict()
def isPythonModule(self):
return self.parent.isPythonModule()
def getFunction(self):
return self.parent.getFunction()
def allocateCallTempNumber(self):
return self.parent.allocateCallTempNumber()
def addTempKeeperUsage(self, variable_name, ref_count):
self.temp_keepers[ variable_name ] = ref_count
def getTempKeeperRefCount(self, variable_name):
return self.temp_keepers[ variable_name ]
def getTempKeeperUsages(self):
result = self.temp_keepers
self.temp_keepers = {}
return result
def getTempKeeperDecl(self):
tmp_keepers = self.getTempKeeperUsages()
return [
"PyObjectTempKeeper%s %s;" % ( ref_count, tmp_variable )
for tmp_variable, ref_count in sorted( iterItems( tmp_keepers ) )
]
def allocateTryNumber(self):
return self.parent.allocateTryNumber()
def setTryFinallyCount(self, value):
self.parent.setTryFinallyCount( value )
def removeFinallyCount(self):
self.parent.removeFinallyCount()
def getTryFinallyCount(self):
return self.parent.getTryFinallyCount()
Nuitka-0.5.0.1/nuitka/codegen/CodeGeneration.py 0000644 0001750 0001750 00000275100 12265270763 021516 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" The code generation.
No language specifics at all are supposed to be present here. Instead it is
using primitives from the given generator to build either Identifiers
(referenced counted expressions) or code sequences (list of strings).
As such this is the place that knows how to take a condition and two code
branches and make a code block out of it. But it doesn't contain any target
language syntax.
"""
from . import (
Generator,
Contexts,
)
from nuitka import (
PythonOperators,
Constants,
Tracing,
Options,
Utils
)
from nuitka.__past__ import iterItems
def generateTupleCreationCode(elements, context):
if _areConstants( elements ):
return Generator.getConstantHandle(
context = context,
constant = tuple( element.getConstant() for element in elements )
)
else:
identifiers = generateExpressionsCode(
expressions = elements,
context = context
)
return Generator.getTupleCreationCode(
element_identifiers = identifiers,
order_relevance = getOrderRelevance( elements ),
context = context
)
def generateListCreationCode(elements, context):
if _areConstants( elements ):
return Generator.getConstantHandle(
context = context,
constant = list( element.getConstant() for element in elements )
)
else:
identifiers = generateExpressionsCode(
expressions = elements,
context = context
)
return Generator.getListCreationCode(
element_identifiers = identifiers,
order_relevance = getOrderRelevance( elements ),
context = context
)
def generateConditionCode( condition, context, inverted = False,
allow_none = False ):
# The complexity is needed to avoid unnecessary complex generated C++, so
# e.g. inverted is typically a branch inside every optimizable case.
# pylint: disable=R0912,R0915
if condition is None and allow_none:
assert not inverted
result = Generator.getTrueExpressionCode()
elif condition.isExpressionConstantRef():
value = condition.getConstant()
if inverted:
value = not value
inverted = False
if value:
result = Generator.getTrueExpressionCode()
else:
result = Generator.getFalseExpressionCode()
elif condition.isExpressionComparison():
left = generateExpressionCode(
expression = condition.getLeft(),
context = context
)
right = generateExpressionCode(
expression = condition.getRight(),
context = context
)
comparator = condition.getComparator()
# Do not allow this, expected to be optimized away.
assert not inverted or \
comparator not in PythonOperators.comparison_inversions, \
condition
result = Generator.getComparisonExpressionBoolCode(
order_relevance = getOrderRelevance(
condition.getVisitableNodes(),
),
comparator = comparator,
left = left,
right = right,
context = context
)
if inverted:
result = Generator.getConditionNotBoolCode(
condition = result
)
inverted = False
elif condition.isExpressionOperationNOT():
if not inverted:
result = Generator.getConditionNotBoolCode(
condition = generateConditionCode(
condition = condition.getOperand(),
context = context
)
)
inverted = False
else:
result = generateConditionCode(
condition = condition.getOperand(),
context = context
)
elif condition.isExpressionConditional():
expression_yes = condition.getExpressionYes()
expression_no = condition.getExpressionNo()
condition = condition.getCondition()
if condition.isExpressionAssignmentTempKeeper():
if expression_yes.isExpressionTempKeeperRef() and \
expression_yes.getVariableName() == condition.getVariableName():
result = Generator.getConditionOrCode(
operands = (
generateConditionCode(
condition = condition.getAssignSource(),
context = context,
),
generateConditionCode(
condition = expression_no,
context = context,
)
)
)
elif expression_no.isExpressionTempKeeperRef() and \
expression_no.getVariableName() == condition.getVariableName():
result = Generator.getConditionAndCode(
operands = (
generateConditionCode(
condition = condition.getAssignSource(),
context = context,
),
generateConditionCode(
condition = expression_yes,
context = context,
)
)
)
else:
assert False, condition
else:
result = Generator.getConditionSelectionCode(
condition_code = generateConditionCode(
condition = condition,
context = context
),
yes_code = generateConditionCode(
condition = expression_yes,
context = context
),
no_code = generateConditionCode(
condition = expression_no,
context = context
)
)
elif condition.isExpressionBuiltinHasattr():
result = Generator.getAttributeCheckBoolCode(
order_relevance = getOrderRelevance(
condition.getVisitableNodes()
),
attribute = generateExpressionCode(
expression = condition.getAttribute(),
context = context
),
source = generateExpressionCode(
expression = condition.getLookupSource(),
context = context
),
context = context
)
elif condition.isExpressionBuiltinIsinstance():
result = Generator.getBuiltinIsinstanceBoolCode(
order_relevance = getOrderRelevance(
condition.getVisitableNodes(),
),
inst_identifier = generateExpressionCode(
expression = condition.getInstance(),
context = context
),
cls_identifier = generateExpressionCode(
expression = condition.getCls(),
context = context
),
context = context
)
else:
condition_identifier = generateExpressionCode(
expression = condition,
context = context
)
if inverted:
result = Generator.getConditionCheckFalseCode(
condition = condition_identifier
)
inverted = False
else:
result = Generator.getConditionCheckTrueCode(
condition = condition_identifier
)
if inverted:
result = Generator.getConditionNotBoolCode(
condition = result
)
assert type( result ) is str, result
return result
def generateFunctionCallCode(call_node, context):
assert call_node.getFunction().isExpressionFunctionCreation()
function_body = call_node.getFunction().getFunctionRef().getFunctionBody()
function_identifier = function_body.getCodeName()
extra_arguments = []
argument_values = call_node.getArgumentValues()
return Generator.getDirectionFunctionCallCode(
function_identifier = function_identifier,
arguments = generateExpressionsCode(
expressions = argument_values,
context = context
),
order_relevance = getOrderRelevance( argument_values ),
closure_variables = function_body.getClosureVariables(),
extra_arguments = extra_arguments,
context = context
)
_generated_functions = {}
def generateFunctionCreationCode( function_body, defaults, kw_defaults,
annotations, context ):
assert function_body.needsCreation(), function_body
parameters = function_body.getParameters()
if defaults:
defaults_identifier = generateTupleCreationCode(
elements = defaults,
context = context
)
else:
defaults_identifier = Generator.getConstantHandle(
constant = None,
context = context
)
if kw_defaults:
kw_defaults_identifier = generateExpressionCode(
expression = kw_defaults,
context = context
)
else:
kw_defaults_identifier = Generator.getConstantHandle(
constant = None,
context = context
)
annotations_identifier = generateExpressionCode(
expression = annotations,
allow_none = True,
context = context,
)
default_args = []
order_relevance = []
if not kw_defaults_identifier.isConstantIdentifier():
default_args.append(kw_defaults_identifier)
order_relevance.extend(
getOrderRelevance( ( kw_defaults, ) )
)
if not defaults_identifier.isConstantIdentifier():
default_args.append(defaults_identifier)
order_relevance.append(
Generator.pickFirst( getOrderRelevance( defaults ) )
)
if annotations_identifier is not None and \
not annotations_identifier.isConstantIdentifier():
default_args.append(annotations_identifier)
order_relevance.extend(
getOrderRelevance( ( annotations, ) )
)
function_identifier = function_body.getCodeName()
maker_code = Generator.getFunctionMakerCode(
context = context,
function_name = function_body.getFunctionName(),
function_qualname = function_body.getFunctionQualname(),
function_identifier = function_identifier,
parameters = parameters,
local_variables = function_body.getLocalVariables(),
closure_variables = function_body.getClosureVariables(),
defaults_identifier = defaults_identifier,
kw_defaults_identifier = kw_defaults_identifier,
annotations_identifier = annotations_identifier,
source_ref = function_body.getSourceReference(),
function_doc = function_body.getDoc(),
is_generator = function_body.isGenerator()
)
context.addHelperCode( function_identifier, maker_code )
function_decl = Generator.getFunctionMakerDecl(
function_identifier = function_body.getCodeName(),
defaults_identifier = defaults_identifier,
kw_defaults_identifier = kw_defaults_identifier,
annotations_identifier = annotations_identifier,
closure_variables = function_body.getClosureVariables()
)
if function_body.getClosureVariables() and not function_body.isGenerator():
function_decl += "\n"
function_decl += Generator.getFunctionContextDefinitionCode(
context = context,
function_identifier = function_body.getCodeName(),
closure_variables = function_body.getClosureVariables(),
)
context.addDeclaration( function_identifier, function_decl )
return Generator.getFunctionCreationCode(
function_identifier = function_body.getCodeName(),
order_relevance = order_relevance,
default_args = default_args,
closure_variables = function_body.getClosureVariables(),
context = context
)
def generateFunctionBodyCode(function_body, context):
function_identifier = function_body.getCodeName()
if function_identifier in _generated_functions:
return _generated_functions[ function_identifier ]
if function_body.needsCreation():
function_context = Contexts.PythonFunctionCreatedContext(
parent = context,
function = function_body
)
else:
function_context = Contexts.PythonFunctionDirectContext(
parent = context,
function = function_body
)
# TODO: Generate both codes, and base direct/etc. decisions on context.
function_codes = generateStatementSequenceCode(
statement_sequence = function_body.getBody(),
allow_none = True,
context = function_context
)
function_codes = function_codes or []
parameters = function_body.getParameters()
if function_body.isGenerator():
function_code = Generator.getGeneratorFunctionCode(
context = function_context,
function_name = function_body.getFunctionName(),
function_identifier = function_identifier,
parameters = parameters,
closure_variables = function_body.getClosureVariables(),
user_variables = function_body.getUserLocalVariables(),
temp_variables = function_body.getTempVariables(),
source_ref = function_body.getSourceReference(),
function_codes = function_codes,
function_doc = function_body.getDoc()
)
else:
function_code = Generator.getFunctionCode(
context = function_context,
function_name = function_body.getFunctionName(),
function_identifier = function_identifier,
parameters = parameters,
closure_variables = function_body.getClosureVariables(),
user_variables = function_body.getUserLocalVariables(),
temp_variables = function_body.getTempVariables(),
function_codes = function_codes,
function_doc = function_body.getDoc(),
file_scope = Generator.getExportScopeCode(
cross_module = function_body.isCrossModuleUsed()
)
)
return function_code
def generateAttributeLookupCode(source, attribute_name, context):
if attribute_name == "__dict__":
return Generator.getAttributeLookupDictSlotCode( source )
elif attribute_name == "__class__":
return Generator.getAttributeLookupClassSlotCode( source )
else:
return Generator.getAttributeLookupCode(
attribute = Generator.getConstantHandle(
context = context,
constant = attribute_name
),
source = source
)
def generateOperationCode(operator, operands, context):
return Generator.getOperationCode(
order_relevance = getOrderRelevance( operands ),
operator = operator,
identifiers = generateExpressionsCode(
expressions = operands,
context = context
),
context = context
)
def generateComparisonExpressionCode(comparison_expression, context):
left = generateExpressionCode(
expression = comparison_expression.getLeft(),
context = context
)
right = generateExpressionCode(
expression = comparison_expression.getRight(),
context = context
)
return Generator.getComparisonExpressionCode(
comparator = comparison_expression.getComparator(),
order_relevance = getOrderRelevance(
comparison_expression.getVisitableNodes()
),
left = left,
right = right,
context = context
)
def generateDictionaryCreationCode(pairs, context):
args = []
# Strange as it is, CPython evalutes the key/value pairs strictly in order,
# but for each pair, the value first.
for pair in pairs:
args.append( pair.getValue() )
args.append( pair.getKey() )
if _areConstants( args ):
constant = {}
for pair in pairs:
key = pair.getKey()
value = pair.getValue()
constant[ key.getConstant() ] = value.getConstant()
return Generator.getConstantHandle(
context = context,
constant = constant
)
else:
args_identifiers = generateExpressionsCode(
expressions = args,
context = context
)
return Generator.getDictionaryCreationCode(
context = context,
order_relevance = getOrderRelevance( args ),
args_identifiers = args_identifiers
)
def generateSetCreationCode(elements, context):
element_identifiers = generateExpressionsCode(
expressions = elements,
context = context
)
return Generator.getSetCreationCode(
order_relevance = getOrderRelevance( elements ),
element_identifiers = element_identifiers,
context = context
)
def _areConstants(expressions):
for expression in expressions:
if not expression.isExpressionConstantRef():
return False
if expression.isMutable():
return False
else:
return True
def generateSliceRangeIdentifier(lower, upper, context):
def isSmallNumberConstant(node):
value = node.getConstant()
if Constants.isNumberConstant( value ):
return abs(int(value)) < 2**63-1
else:
return False
if lower is None:
lower = Generator.getMinIndexCode()
elif lower.isExpressionConstantRef() and isSmallNumberConstant( lower ):
lower = Generator.getIndexValueCode(
int( lower.getConstant() )
)
else:
lower = Generator.getIndexCode(
identifier = generateExpressionCode(
expression = lower,
context = context
)
)
if upper is None:
upper = Generator.getMaxIndexCode()
elif upper.isExpressionConstantRef() and isSmallNumberConstant( upper ):
upper = Generator.getIndexValueCode(
int( upper.getConstant() )
)
else:
upper = Generator.getIndexCode(
identifier = generateExpressionCode(
expression = upper,
context = context
)
)
return lower, upper
def generateSliceAccessIdentifiers(sliced, lower, upper, context):
lower, upper = generateSliceRangeIdentifier( lower, upper, context )
sliced = generateExpressionCode(
expression = sliced,
context = context
)
return sliced, lower, upper
_slicing_available = Utils.python_version < 300
def decideSlicing(lower, upper):
return _slicing_available and \
( lower is None or lower.isIndexable() ) and \
( upper is None or upper.isIndexable() )
def generateSubscriptLookupCode(expression, context):
return Generator.getSubscriptLookupCode(
order_relevance = getOrderRelevance(
( expression.getLookupSource(), expression.getSubscript() )
),
subscript = generateExpressionCode(
expression = expression.getSubscript(),
context = context
),
source = generateExpressionCode(
expression = expression.getLookupSource(),
context = context
),
context = context
)
def generateSliceLookupCode(expression, context):
lower = expression.getLower()
upper = expression.getUpper()
if decideSlicing( lower, upper ):
expression_identifier, lower_identifier, upper_identifier = generateSliceAccessIdentifiers(
sliced = expression.getLookupSource(),
lower = lower,
upper = upper,
context = context
)
return Generator.getSliceLookupIndexesCode(
source = expression_identifier,
lower = lower_identifier,
upper = upper_identifier
)
else:
if _slicing_available:
return Generator.getSliceLookupCode(
order_relevance = getOrderRelevance(
(
expression.getLookupSource(),
expression.getLower(),
expression.getUpper()
),
allow_none = True,
),
source = generateExpressionCode(
expression = expression.getLookupSource(),
context = context
),
lower = generateExpressionCode(
expression = lower,
allow_none = True,
context = context
),
upper = generateExpressionCode(
expression = upper,
allow_none = True,
context = context
),
context = context
)
else:
order_relevance = getOrderRelevance(
( expression.getLookupSource(), lower, upper, None ),
allow_none = True
)
return Generator.getSubscriptLookupCode(
order_relevance = (
order_relevance[0],
Generator.pickFirst( order_relevance[1:] )
),
source = generateExpressionCode(
expression = expression.getLookupSource(),
context = context
),
subscript = Generator.getSliceObjectCode(
order_relevance = order_relevance[1:],
lower = generateExpressionCode(
expression = lower,
allow_none = True,
context = context
),
upper = generateExpressionCode(
expression = upper,
allow_none = True,
context = context
),
step = None,
context = context
),
context = context
)
def generateCallCode(call_node, context):
called_identifier = generateExpressionCode(
expression = call_node.getCalled(),
context = context
)
argument_dictionary = generateExpressionCode(
expression = call_node.getCallKw(),
context = context
)
# Avoid non-empty arguments for positional or keywords, where they are in
# fact empty.
if argument_dictionary is not None and \
argument_dictionary.isConstantIdentifier() and \
argument_dictionary.getConstant() == {}:
argument_dictionary = None
call_args = call_node.getCallArgs()
if argument_dictionary is None:
if call_args.isExpressionConstantRef() and \
call_args.getConstant() == ():
return Generator.getCallCodeNoArgs(
called_identifier = called_identifier
)
elif call_args.isExpressionMakeTuple():
return Generator.getCallCodePosArgsQuick(
order_relevance = getOrderRelevance(
( call_node.getCalled(), ) + call_args.getElements(),
),
called_identifier = called_identifier,
arguments = generateExpressionsCode(
expressions = call_args.getElements(),
context = context
),
context = context
)
elif call_args.isExpressionConstantRef():
return Generator.getCallCodePosArgsQuick(
order_relevance =
( call_node.isOrderRelevant(), ) + \
( None, ) * len( call_args.getConstant() ),
called_identifier = called_identifier,
arguments = [
Generator.getConstantAccess(
constant = element,
context = context
)
for element in
call_args.getConstant()
],
context = context
)
else:
return Generator.getCallCodePosArgs(
order_relevance = getOrderRelevance(
( call_node.getCalled(), call_node.getCallArgs() ),
),
called_identifier = called_identifier,
argument_tuple = generateExpressionCode(
expression = call_args,
context = context
),
context = context
)
else:
argument_tuple = generateExpressionCode(
expression = call_args,
context = context
)
if argument_tuple is None:
return Generator.getCallCodeKeywordArgs(
order_relevance = getOrderRelevance(
( call_node.getCalled(), call_node.getCallKw() ),
),
called_identifier = called_identifier,
argument_dictionary = argument_dictionary,
context = context
)
else:
return Generator.getCallCodePosKeywordArgs(
order_relevance = getOrderRelevance(
( call_node.getCalled(), call_node.getCallArgs(),
call_node.getCallKw() ),
),
called_identifier = called_identifier,
argument_dictionary = argument_dictionary,
argument_tuple = argument_tuple,
context = context
)
def _decideLocalsMode(provider):
# TODO: This information should be in the node instead, and not decided
# during code generation, as optimization needs to know it. It should also
# be sticky and safe against inline this way.
if Utils.python_version >= 300:
mode = "updated"
elif provider.isExpressionFunctionBody() and ( provider.isEarlyClosure() or provider.isUnoptimized() ):
mode = "updated"
else:
mode = "copy"
return mode
def generateBuiltinLocalsCode(locals_node, context):
provider = locals_node.getParentVariableProvider()
return Generator.getLoadLocalsCode(
context = context,
provider = provider,
mode = _decideLocalsMode( provider )
)
def generateBuiltinDir0Code(dir_node, context):
provider = dir_node.getParentVariableProvider()
return Generator.getLoadDirCode(
context = context,
provider = provider
)
def generateBuiltinDir1Code(dir_node, context):
return Generator.getBuiltinDir1Code(
identifier = generateExpressionCode(
expression = dir_node.getValue(),
context = context
)
)
def generateExpressionsCode(expressions, context, allow_none = False):
assert type( expressions ) in ( tuple, list )
return [
generateExpressionCode(
expression = expression,
context = context,
allow_none = allow_none
)
for expression in
expressions
]
def getOrderRelevance(expressions, allow_none = False):
result = []
for expression in expressions:
if expression is None and allow_none:
result.append( None )
elif expression.isOrderRelevant():
result.append( expression.getSourceReference() )
else:
result.append( None )
return result
def _generateExpressionCode(expression, context, allow_none):
# This is a dispatching function with a branch per expression node type, and
# therefore many statements even if every branch is small
# pylint: disable=R0912,R0915
if expression is None and allow_none:
return None
# Make sure we don't generate code twice for any node, this uncovers bugs
# where nodes are shared in the tree, which is not allowed.
assert not hasattr( expression, "code_generated" ), expression
expression.code_generated = True
def makeExpressionCode(expression, allow_none = False):
if allow_none and expression is None:
return None
return generateExpressionCode(
expression = expression,
context = context
)
if not expression.isExpression():
Tracing.printError( "No expression %r" % expression )
expression.dump()
assert False, expression
if expression.isExpressionVariableRef():
if expression.getVariable() is None:
Tracing.printError( "Illegal variable reference, not resolved" )
expression.dump()
assert False, (
expression.getSourceReference(),
expression.getVariableName()
)
identifier = Generator.getVariableHandle(
variable = expression.getVariable(),
context = context
)
elif expression.isExpressionTempVariableRef():
identifier = Generator.getVariableHandle(
variable = expression.getVariable(),
context = context
)
elif expression.isExpressionConstantRef():
identifier = Generator.getConstantAccess(
constant = expression.getConstant(),
context = context
)
elif expression.isOperation():
identifier = generateOperationCode(
operator = expression.getOperator(),
operands = expression.getOperands(),
context = context
)
elif expression.isExpressionMakeTuple():
identifier = generateTupleCreationCode(
elements = expression.getElements(),
context = context
)
elif expression.isExpressionMakeList():
identifier = generateListCreationCode(
elements = expression.getElements(),
context = context
)
elif expression.isExpressionMakeSet():
identifier = generateSetCreationCode(
elements = expression.getElements(),
context = context
)
elif expression.isExpressionMakeDict():
assert expression.getPairs()
identifier = generateDictionaryCreationCode(
pairs = expression.getPairs(),
context = context
)
elif expression.isExpressionCall():
identifier = generateCallCode(
call_node = expression,
context = context
)
elif expression.isExpressionFunctionCall():
identifier = generateFunctionCallCode(
call_node = expression,
context = context
)
elif expression.isExpressionAttributeLookup():
identifier = generateAttributeLookupCode(
source = makeExpressionCode( expression.getLookupSource() ),
attribute_name = expression.getAttributeName(),
context = context
)
elif expression.isExpressionSpecialAttributeLookup():
identifier = Generator.getSpecialAttributeLookupCode(
attribute = Generator.getConstantHandle(
context = context,
constant = expression.getAttributeName()
),
source = makeExpressionCode( expression.getLookupSource() ),
)
elif expression.isExpressionBuiltinHasattr():
identifier = Generator.getAttributeCheckCode(
order_relevance = getOrderRelevance(
expression.getVisitableNodes()
),
attribute = makeExpressionCode(
expression.getAttribute()
),
source = makeExpressionCode(
expression.getLookupSource()
),
context = context
)
elif expression.isExpressionBuiltinGetattr():
identifier = Generator.getAttributeGetCode(
order_relevance = getOrderRelevance(
(
expression.getLookupSource(),
expression.getAttribute(),
expression.getDefault(),
),
allow_none = True
),
source = makeExpressionCode(
expression.getLookupSource()
),
attribute = makeExpressionCode(
expression.getAttribute()
),
default = makeExpressionCode(
expression.getDefault(),
allow_none = True
),
context = context
)
elif expression.isExpressionBuiltinSetattr():
identifier = Generator.getAttributeSetCode(
order_relevance = getOrderRelevance(
expression.getVisitableNodes()
),
source = makeExpressionCode(
expression.getLookupSource()
),
attribute = makeExpressionCode(
expression.getAttribute()
),
value = makeExpressionCode(
expression.getValue()
),
context = context
)
elif expression.isExpressionImportName():
identifier = Generator.getImportNameCode(
import_name = Generator.getConstantHandle(
context = context,
constant = expression.getImportName()
),
module = makeExpressionCode(
expression.getModule()
)
)
elif expression.isExpressionSubscriptLookup():
identifier = generateSubscriptLookupCode(
expression = expression,
context = context
)
elif expression.isExpressionSliceLookup():
identifier = generateSliceLookupCode(
expression = expression,
context = context
)
elif expression.isExpressionSliceObject():
identifier = Generator.getSliceObjectCode(
order_relevance = getOrderRelevance(
(
expression.getLower(),
expression.getUpper(),
expression.getStep()
),
allow_none = True
),
lower = makeExpressionCode(
expression = expression.getLower(),
allow_none = True
),
upper = makeExpressionCode(
expression = expression.getUpper(),
allow_none = True
),
step = makeExpressionCode(
expression = expression.getStep(),
allow_none = True
),
context = context
)
elif expression.isExpressionConditional():
identifier = Generator.getConditionalExpressionCode(
condition_code = generateConditionCode(
condition = expression.getCondition(),
context = context
),
identifier_yes = makeExpressionCode(
expression.getExpressionYes()
),
identifier_no = makeExpressionCode( expression.getExpressionNo() )
)
elif expression.isExpressionBuiltinRange1():
identifier = Generator.getBuiltinRange1Code(
value = makeExpressionCode( expression.getLow() )
)
elif expression.isExpressionBuiltinRange2():
identifier = Generator.getBuiltinRange2Code(
order_relevance = getOrderRelevance(
expression.getVisitableNodes()
),
low = makeExpressionCode( expression.getLow() ),
high = makeExpressionCode( expression.getHigh() ),
context = context
)
elif expression.isExpressionBuiltinRange3():
identifier = Generator.getBuiltinRange3Code(
order_relevance = getOrderRelevance(
expression.getVisitableNodes()
),
low = makeExpressionCode( expression.getLow() ),
high = makeExpressionCode( expression.getHigh() ),
step = makeExpressionCode( expression.getStep() ),
context = context
)
elif expression.isExpressionBuiltinXrange():
identifier = Generator.getBuiltinXrangeCode(
order_relevance = getOrderRelevance(
(
expression.getLow(),
expression.getHigh(),
expression.getStep()
),
allow_none = True
),
low = makeExpressionCode(
expression.getLow(),
allow_none = False
),
high = makeExpressionCode(
expression.getHigh(),
allow_none = True
),
step = makeExpressionCode(
expression.getStep(),
allow_none = True
),
context = context
)
elif expression.isExpressionBuiltinGlobals():
identifier = Generator.getLoadGlobalsCode(
context = context
)
elif expression.isExpressionBuiltinLocals():
identifier = generateBuiltinLocalsCode(
locals_node = expression,
context = context
)
elif expression.isExpressionBuiltinDir0():
identifier = generateBuiltinDir0Code(
dir_node = expression,
context = context
)
elif expression.isExpressionBuiltinDir1():
identifier = generateBuiltinDir1Code(
dir_node = expression,
context = context
)
elif expression.isExpressionBuiltinVars():
identifier = Generator.getLoadVarsCode(
identifier = makeExpressionCode( expression.getSource() )
)
elif expression.isExpressionBuiltinEval():
identifier = generateEvalCode(
context = context,
eval_node = expression
)
elif expression.isExpressionBuiltinOpen():
identifier = Generator.getBuiltinOpenCode(
order_relevance = getOrderRelevance(
(
expression.getFilename(),
expression.getMode(),
expression.getBuffering()
),
allow_none = True
),
filename = makeExpressionCode(
expression = expression.getFilename(),
allow_none = True
),
mode = makeExpressionCode(
expression = expression.getMode(),
allow_none = True
),
buffering = makeExpressionCode(
expression = expression.getBuffering(),
allow_none = True
),
context = context
)
elif expression.isExpressionFunctionCreation():
identifier = generateFunctionCreationCode(
function_body = expression.getFunctionRef().getFunctionBody(),
defaults = expression.getDefaults(),
kw_defaults = expression.getKwDefaults(),
annotations = expression.getAnnotations(),
context = context
)
elif expression.isExpressionComparison():
identifier = generateComparisonExpressionCode(
comparison_expression = expression,
context = context
)
elif expression.isExpressionYield():
identifier = Generator.getYieldCode(
identifier = makeExpressionCode(
expression = expression.getExpression()
),
in_handler = expression.isExceptionPreserving()
)
elif expression.isExpressionYieldFrom():
identifier = Generator.getYieldFromCode(
identifier = makeExpressionCode(
expression = expression.getExpression()
),
in_handler = expression.isExceptionPreserving()
)
elif expression.isExpressionImportModule():
identifier = generateImportModuleCode(
expression = expression,
context = context
)
elif expression.isExpressionBuiltinImport():
identifier = generateBuiltinImportCode(
expression = expression,
context = context
)
elif expression.isExpressionBuiltinChr():
identifier = Generator.getBuiltinChrCode(
value = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionBuiltinOrd():
identifier = Generator.getBuiltinOrdCode(
value = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionBuiltinBin():
identifier = Generator.getBuiltinBinCode(
value = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionBuiltinOct():
identifier = Generator.getBuiltinOctCode(
value = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionBuiltinHex():
identifier = Generator.getBuiltinHexCode(
value = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionBuiltinLen():
identifier = Generator.getBuiltinLenCode(
identifier = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionBuiltinIter1():
identifier = Generator.getBuiltinIter1Code(
value = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionBuiltinIter2():
identifier = Generator.getBuiltinIter2Code(
order_relevance = getOrderRelevance(
expression.getVisitableNodes()
),
callable_identifier = makeExpressionCode(
expression.getCallable()
),
sentinel_identifier = makeExpressionCode(
expression.getSentinel()
),
context = context
)
elif expression.isExpressionBuiltinNext1():
identifier = Generator.getBuiltinNext1Code(
value = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionSpecialUnpack():
identifier = Generator.getUnpackNextCode(
iterator_identifier = makeExpressionCode( expression.getValue() ),
count = expression.getCount()
)
elif expression.isExpressionBuiltinNext2():
identifier = Generator.getBuiltinNext2Code(
order_relevance = getOrderRelevance(
expression.getVisitableNodes()
),
iterator_identifier = makeExpressionCode(
expression.getIterator()
),
default_identifier = makeExpressionCode(
expression.getDefault()
),
context = context
)
elif expression.isExpressionBuiltinType1():
identifier = Generator.getBuiltinType1Code(
value = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionBuiltinType3():
identifier = Generator.getBuiltinType3Code(
order_relevance = getOrderRelevance(
expression.getVisitableNodes()
),
name_identifier = makeExpressionCode( expression.getTypeName() ),
bases_identifier = makeExpressionCode( expression.getBases() ),
dict_identifier = makeExpressionCode( expression.getDict() ),
context = context
)
elif expression.isExpressionBuiltinTuple():
identifier = Generator.getBuiltinTupleCode(
identifier = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionBuiltinList():
identifier = Generator.getBuiltinListCode(
identifier = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionBuiltinDict():
# assert not expression.hasOnlyConstantArguments()
identifier = Generator.getBuiltinDictCode(
seq_identifier = makeExpressionCode(
expression = expression.getPositionalArgument(),
allow_none = True
),
dict_identifier = generateDictionaryCreationCode(
pairs = expression.getNamedArgumentPairs(),
context = context
)
)
elif expression.isExpressionBuiltinSet():
identifier = Generator.getBuiltinSetCode(
identifier = makeExpressionCode( expression.getValue() )
)
elif Utils.python_version < 300 and expression.isExpressionBuiltinStr():
identifier = Generator.getBuiltinStrCode(
identifier = makeExpressionCode( expression.getValue() )
)
elif Utils.python_version < 300 and expression.isExpressionBuiltinUnicode() or \
Utils.python_version >= 300 and expression.isExpressionBuiltinStr():
encoding = expression.getEncoding()
errors = expression.getErrors()
if encoding is None and errors is None:
identifier = Generator.getBuiltinUnicode1Code(
identifier = makeExpressionCode(
expression = expression.getValue()
)
)
else:
identifier = Generator.getBuiltinUnicode3Code(
order_relevance = getOrderRelevance(
(
expression.getValue(),
encoding,
errors
),
allow_none = True
),
identifier = makeExpressionCode(
expression = expression.getValue()
),
encoding = makeExpressionCode(
expression = encoding,
allow_none = True
),
errors = makeExpressionCode(
expression = errors,
allow_none = True
),
context = context
)
elif expression.isExpressionBuiltinFloat():
identifier = Generator.getBuiltinFloatCode(
identifier = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionBuiltinBool():
identifier = Generator.getBuiltinBoolCode(
identifier = makeExpressionCode( expression.getValue() )
)
elif expression.isExpressionRaiseException():
# Missed optimization opportunity, please report.
if Options.isDebug():
parent = expression.parent
assert parent.isExpressionSideEffects() or \
parent.isExpressionConditional(), \
( expression, expression.parent )
identifier = Generator.getRaiseExceptionExpressionCode(
exception_type = makeExpressionCode(
expression = expression.getExceptionType()
),
exception_value = makeExpressionCode(
expression = expression.getExceptionValue(),
allow_none = True
),
context = context
)
elif expression.isExpressionBuiltinMakeException():
identifier = Generator.getMakeBuiltinExceptionCode(
exception_type = expression.getExceptionName(),
exception_args = generateExpressionsCode(
expressions = expression.getArgs(),
context = context
),
order_relevance = getOrderRelevance( expression.getArgs() ),
context = context
)
elif expression.isExpressionBuiltinOriginalRef():
assert not expression.isExpressionBuiltinRef()
identifier = Generator.getBuiltinOriginalRefCode(
builtin_name = expression.getBuiltinName(),
)
elif expression.isExpressionBuiltinRef():
identifier = Generator.getBuiltinRefCode(
builtin_name = expression.getBuiltinName(),
context = context
)
elif expression.isExpressionBuiltinAnonymousRef():
identifier = Generator.getBuiltinAnonymousRefCode(
builtin_name = expression.getBuiltinName(),
)
elif expression.isExpressionBuiltinExceptionRef():
identifier = Generator.getExceptionRefCode(
exception_type = expression.getExceptionName(),
)
elif expression.isExpressionAssignmentTempKeeper():
identifier = Generator.getAssignmentTempKeeperCode(
variable = expression.getVariable(),
source_identifier = makeExpressionCode(
expression.getAssignSource()
),
context = context
)
elif expression.isExpressionBuiltinInt():
value = expression.getValue()
base = expression.getBase()
assert value is not None or base is not None
if base is None:
identifier = Generator.getBuiltinInt1Code(
identifier = makeExpressionCode(
value,
allow_none = True
),
context = context
)
else:
identifier = Generator.getBuiltinInt2Code(
order_relevance = getOrderRelevance(
( value, base ),
allow_none = True
),
identifier = makeExpressionCode(
value,
allow_none = True
),
base = makeExpressionCode(
expression.getBase(),
),
context = context
)
elif Utils.python_version < 300 and expression.isExpressionBuiltinLong():
value = expression.getValue()
base = expression.getBase()
assert value is not None or base is not None
if base is None:
identifier = Generator.getBuiltinLong1Code(
identifier = makeExpressionCode(
value,
allow_none = True
),
context = context
)
else:
identifier = Generator.getBuiltinLong2Code(
order_relevance = getOrderRelevance(
( value, base ),
allow_none = True
),
identifier = makeExpressionCode(
value,
allow_none = True
),
base = makeExpressionCode(
expression.getBase(),
),
context = context
)
elif expression.isExpressionCaughtExceptionTypeRef():
identifier = Generator.getCurrentExceptionTypeCode()
elif expression.isExpressionCaughtExceptionValueRef():
identifier = Generator.getCurrentExceptionValueCode()
elif expression.isExpressionCaughtExceptionTracebackRef():
identifier = Generator.getCurrentExceptionTracebackCode()
elif expression.isExpressionListOperationAppend():
identifier = Generator.getListOperationAppendCode(
list_identifier = makeExpressionCode( expression.getList() ),
value_identifier = makeExpressionCode( expression.getValue() ),
)
# TODO: These operations need order enforcement, once they are added by
# optimization and not be re-formulations only.
elif expression.isExpressionSetOperationAdd():
identifier = Generator.getSetOperationAddCode(
set_identifier = makeExpressionCode( expression.getSet() ),
value_identifier = makeExpressionCode( expression.getValue() ),
)
elif expression.isExpressionDictOperationSet():
identifier = Generator.getDictOperationSetCode(
dict_identifier = makeExpressionCode( expression.getDict() ),
key_identifier = makeExpressionCode( expression.getKey() ),
value_identifier = makeExpressionCode( expression.getValue() ),
)
elif expression.isExpressionDictOperationGet():
identifier = Generator.getDictOperationGetCode(
dict_identifier = makeExpressionCode( expression.getDict() ),
key_identifier = makeExpressionCode( expression.getKey() ),
)
elif expression.isExpressionTempKeeperRef():
identifier = Generator.getTempKeeperHandle(
variable = expression.getVariable(),
context = context
)
elif expression.isExpressionSideEffects():
identifier = Generator.getSideEffectsCode(
side_effects = generateExpressionsCode(
expressions = expression.getSideEffects(),
context = context
),
identifier = makeExpressionCode( expression.getExpression() )
)
elif expression.isExpressionBuiltinSuper():
identifier = Generator.getBuiltinSuperCode(
order_relevance = getOrderRelevance(
( expression.getType(), expression.getObject() ),
allow_none = True
),
type_identifier = makeExpressionCode(
expression.getType(),
allow_none = True
),
object_identifier = makeExpressionCode(
expression.getObject(),
allow_none = True
),
context = context
)
elif expression.isExpressionBuiltinIsinstance():
identifier = Generator.getBuiltinIsinstanceCode(
order_relevance = getOrderRelevance(
expression.getVisitableNodes()
),
inst_identifier = makeExpressionCode( expression.getInstance() ),
cls_identifier = makeExpressionCode( expression.getCls() ),
context = context
)
elif expression.isExpressionSelectMetaclass():
identifier = Generator.getSelectMetaclassCode(
metaclass_identifier = makeExpressionCode(
expression.getMetaclass(),
allow_none = True
),
bases_identifier = makeExpressionCode( expression.getBases() ),
context = context
)
elif Utils.python_version < 300 and \
expression.isExpressionBuiltinExecfile():
identifier = generateExecfileCode(
context = context,
execfile_node = expression
)
elif Utils.python_version >= 300 and expression.isExpressionBuiltinExec():
# exec builtin of Python3, as opposed to Python2 statement
identifier = generateEvalCode(
context = context,
eval_node = expression
)
else:
assert False, expression
if not hasattr( identifier, "getCodeTemporaryRef" ):
raise AssertionError( "not a code object?", repr( identifier ) )
return identifier
def generateExpressionCode(expression, context, allow_none = False):
try:
return _generateExpressionCode(
expression = expression,
context = context,
allow_none = allow_none
)
except:
Tracing.printError(
"Problem with %r at %s" % (
expression,
expression.getSourceReference()
)
)
raise
def generateAssignmentVariableCode(variable_ref, value, context):
return Generator.getVariableAssignmentCode(
variable = variable_ref.getVariable(),
identifier = value,
context = context
)
def generateAssignmentAttributeCode( lookup_source, attribute_name, value,
context ):
target = generateExpressionCode(
expression = lookup_source,
context = context
)
identifer = generateExpressionCode(
expression = value,
context = context
)
order_relevance = getOrderRelevance( ( value, lookup_source ) )
if attribute_name == "__dict__":
return Generator.getAttributeAssignmentDictSlotCode(
order_relevance = order_relevance,
target = target,
identifier = identifer
)
elif attribute_name == "__class__":
return Generator.getAttributeAssignmentClassSlotCode(
order_relevance = order_relevance,
target = target,
identifier = identifer
)
else:
order_relevance.append( None )
return Generator.getAttributeAssignmentCode(
order_relevance = order_relevance,
target = target,
attribute = Generator.getConstantHandle(
context = context,
constant = attribute_name
),
identifier = identifer
)
def generateAssignmentSliceCode(lookup_source, lower, upper, value, context):
value_identifier = generateExpressionCode(
expression = value,
context = context
)
if decideSlicing( lower, upper ):
expression_identifier, lower_identifier, upper_identifier = generateSliceAccessIdentifiers(
sliced = lookup_source,
lower = lower,
upper = upper,
context = context
)
return Generator.getSliceAssignmentIndexesCode(
target = expression_identifier,
upper = upper_identifier,
lower = lower_identifier,
identifier = value_identifier
)
else:
if _slicing_available:
return Generator.getSliceAssignmentCode(
order_relevance = getOrderRelevance(
( value, lookup_source, lower, upper ),
allow_none = True
),
target = generateExpressionCode(
expression = lookup_source,
context = context
),
lower = generateExpressionCode(
expression = lower,
allow_none = True,
context = context
),
upper = generateExpressionCode(
expression = upper,
allow_none = True,
context = context
),
identifier = value_identifier
)
else:
order_relevance = getOrderRelevance(
( value, lookup_source, lower, upper, None ),
allow_none = True
)
return Generator.getSubscriptAssignmentCode(
order_relevance = (
order_relevance[0],
order_relevance[1],
Generator.pickFirst( order_relevance[2:] )
),
subscribed = generateExpressionCode(
expression = lookup_source,
context = context
),
subscript = Generator.getSliceObjectCode(
order_relevance = order_relevance[2:],
lower = generateExpressionCode(
expression = lower,
allow_none = True,
context = context
),
upper = generateExpressionCode(
expression = upper,
allow_none = True,
context = context
),
step = None,
context = context
),
identifier = value_identifier
)
def generateDelVariableCode(variable_ref, tolerant, context):
return Generator.getVariableDelCode(
variable = variable_ref.getVariable(),
tolerant = tolerant,
context = context
)
def generateDelSubscriptCode(subscribed, subscript, context):
return Generator.getSubscriptDelCode(
order_relevance = getOrderRelevance(
( subscribed, subscript )
),
subscribed = generateExpressionCode(
expression = subscribed,
context = context
),
subscript = generateExpressionCode(
expression = subscript,
context = context
)
)
def generateDelSliceCode(lookup_source, lower, upper, context):
if decideSlicing( lower, upper ):
target_identifier, lower_identifier, upper_identifier = generateSliceAccessIdentifiers(
sliced = lookup_source,
lower = lower,
upper = upper,
context = context
)
return Generator.getSliceDelCode(
target = target_identifier,
lower = lower_identifier,
upper = upper_identifier
)
else:
order_relevance = getOrderRelevance(
( lookup_source, lower, upper, None ),
allow_none = True
)
return Generator.getSubscriptDelCode(
order_relevance = (
order_relevance[0],
Generator.pickFirst( order_relevance[1:] )
),
subscribed = generateExpressionCode(
expression = lookup_source,
context = context
),
subscript = Generator.getSliceObjectCode(
order_relevance = order_relevance[1:],
lower = generateExpressionCode(
expression = lower,
allow_none = True,
context = context
),
upper = generateExpressionCode(
expression = upper,
allow_none = True,
context = context
),
step = None,
context = context
)
)
def generateDelAttributeCode(statement, context):
return Generator.getAttributeDelCode(
target = generateExpressionCode(
expression = statement.getLookupSource(),
context = context
),
attribute = Generator.getConstantHandle(
context = context,
constant = statement.getAttributeName()
)
)
def _generateEvalCode(node, context):
globals_value = node.getGlobals()
if globals_value is None:
globals_identifier = Generator.getConstantHandle(
constant = None,
context = context
)
else:
globals_identifier = generateExpressionCode(
expression = globals_value,
context = context
)
locals_value = node.getLocals()
if locals_value is None:
locals_identifier = Generator.getConstantHandle(
constant = None,
context = context
)
else:
locals_identifier = generateExpressionCode(
expression = locals_value,
context = context
)
if node.isExpressionBuiltinEval() or \
( Utils.python_version >= 300 and node.isExpressionBuiltinExec() ):
filename = ""
else:
filename = ""
order_relevance = getOrderRelevance(
( node.getSourceCode(), globals_value, locals_value ),
allow_none = True
)
identifier = Generator.getEvalCode(
order_relevance = order_relevance,
exec_code = generateExpressionCode(
expression = node.getSourceCode(),
context = context
),
globals_identifier = globals_identifier,
locals_identifier = locals_identifier,
filename_identifier = Generator.getConstantHandle(
constant = filename,
context = context
),
mode_identifier = Generator.getConstantHandle(
constant = "eval" if node.isExpressionBuiltinEval() else "exec",
context = context
),
future_flags = Generator.getFutureFlagsCode(
future_spec = node.getSourceReference().getFutureSpec()
),
context = context
)
return identifier
def generateEvalCode(eval_node, context):
return _generateEvalCode(
node = eval_node,
context = context
)
def generateExecfileCode(execfile_node, context):
return _generateEvalCode(
node = execfile_node,
context = context
)
def generateExecCode(exec_def, context):
exec_globals = exec_def.getGlobals()
if exec_globals is None:
globals_identifier = Generator.getConstantHandle(
constant = None,
context = context
)
else:
globals_identifier = generateExpressionCode(
expression = exec_globals,
context = context
)
exec_locals = exec_def.getLocals()
if exec_locals is None:
locals_identifier = Generator.getConstantHandle(
constant = None,
context = context
)
elif exec_locals is not None:
locals_identifier = generateExpressionCode(
expression = exec_locals,
context = context
)
return Generator.getExecCode(
context = context,
provider = exec_def.getParentVariableProvider(),
exec_code = generateExpressionCode(
context = context,
expression = exec_def.getSourceCode()
),
globals_identifier = globals_identifier,
locals_identifier = locals_identifier,
future_flags = Generator.getFutureFlagsCode(
future_spec = exec_def.getSourceReference().getFutureSpec()
),
source_ref = exec_def.getSourceReference()
)
def generateTryFinallyCode(statement, context):
try_count = context.allocateTryNumber()
context.setTryFinallyCount( try_count )
code_final = generateStatementSequenceCode(
statement_sequence = statement.getBlockFinal(),
context = context
)
context.removeFinallyCount()
code_tried = generateStatementSequenceCode(
statement_sequence = statement.getBlockTry(),
context = context
)
needs_return_value_catch = statement.needsExceptionReturnValueCatcher()
needs_return_value_reraise = statement.needsExceptionReturnValueReraiser()
return Generator.getTryFinallyCode(
code_tried = code_tried,
code_final = code_final,
needs_break = statement.needsExceptionBreak(),
needs_continue = statement.needsExceptionContinue(),
needs_return_value_catch = needs_return_value_catch,
needs_return_value_reraise = needs_return_value_reraise,
aborting = statement.isStatementAborting(),
try_count = try_count,
context = context
)
def generateTryExceptCode(statement, context):
tried_block = statement.getBlockTry()
assert tried_block.mayRaiseException( BaseException )
if statement.isStatementTryExceptOptimized():
tried_statements = tried_block.getStatements()
assert len( tried_statements ) == 1
tried_statement = tried_statements[0]
source = tried_statement.getAssignSource()
assert source.isExpressionBuiltinNext1()
assert not source.getValue().mayRaiseException( BaseException )
handlers = statement.getExceptionHandlers()
assert len( handlers ) == 1
temp_identifier = Generator.getTryNextExceptStopIterationIdentifier(
context = context
)
source_identifier = generateExpressionCode(
expression = source.getValue(),
context = context
)
assign_code = generateAssignmentVariableCode(
variable_ref = tried_statement.getTargetVariableRef(),
value = temp_identifier,
context = context
)
handler_code = generateStatementSequenceCode(
statement_sequence = handlers[0].getExceptionBranch(),
allow_none = True,
context = context
)
return Generator.getTryNextExceptStopIterationCode(
temp_identifier = temp_identifier,
assign_code = assign_code,
handler_code = handler_code,
source_identifier = source_identifier
)
handler_codes = []
code_tried = generateStatementSequenceCode(
statement_sequence = tried_block,
context = context,
)
for count, handler in enumerate( statement.getExceptionHandlers() ):
Generator.pushLineNumberBranch()
exception_identifiers = generateExpressionsCode(
expressions = handler.getExceptionTypes(),
allow_none = True,
context = context
)
exception_branch = handler.getExceptionBranch()
handler_code = generateStatementSequenceCode(
statement_sequence = exception_branch,
allow_none = True,
context = context
)
handler_codes += Generator.getTryExceptHandlerCode(
exception_identifiers = exception_identifiers,
handler_code = handler_code,
first_handler = count == 0,
# TODO: Check if the code may access traceback or not, we can create
# more efficient code if not, which ought to be the common case.
needs_frame_detach = exception_branch is not None
)
Generator.popLineNumberBranch()
Generator.mergeLineNumberBranches()
return Generator.getTryExceptCode(
context = context,
code_tried = code_tried,
handler_codes = handler_codes
)
def generateRaiseCode(statement, context):
exception_type = statement.getExceptionType()
exception_value = statement.getExceptionValue()
exception_tb = statement.getExceptionTrace()
exception_cause = statement.getExceptionCause()
# Exception cause is only possible with simple raise form.
if exception_cause is not None:
assert exception_type is not None
assert exception_value is None
assert exception_tb is None
return Generator.getRaiseExceptionWithCauseCode(
order_relevance = getOrderRelevance(
( statement.getExceptionType(), statement.getExceptionCause() )
),
exception_type = generateExpressionCode(
expression = statement.getExceptionType(),
context = context
),
exception_cause = generateExpressionCode(
expression = statement.getExceptionCause(),
context = context
),
context = context
)
elif exception_type is None:
assert exception_cause is None
return Generator.getReRaiseExceptionCode(
local = statement.isReraiseExceptionLocal(),
final = context.getTryFinallyCount()
if statement.isReraiseExceptionFinally()
else None,
)
elif exception_value is None and exception_tb is None:
return Generator.getRaiseExceptionWithTypeCode(
order_relevance = getOrderRelevance(
( exception_type, ),
),
exception_type = generateExpressionCode(
expression = exception_type,
context = context
),
context = context
)
elif exception_tb is None:
return Generator.getRaiseExceptionWithValueCode(
order_relevance = getOrderRelevance(
( exception_type, exception_value ),
),
exception_type = generateExpressionCode(
expression = exception_type,
context = context
),
exception_value = generateExpressionCode(
expression = exception_value,
context = context
),
implicit = statement.isImplicit(),
context = context
)
else:
return Generator.getRaiseExceptionWithTracebackCode(
order_relevance = getOrderRelevance(
( exception_type, exception_value, exception_tb ),
),
exception_type = generateExpressionCode(
expression = exception_type,
context = context
),
exception_value = generateExpressionCode(
expression = exception_value,
context = context
),
exception_tb = generateExpressionCode(
expression = exception_tb,
context = context
)
)
def generateImportModuleCode(expression, context):
provider = expression.getParentVariableProvider()
globals_dict = Generator.getLoadGlobalsCode(
context = context
)
if provider.isPythonModule():
locals_dict = globals_dict
else:
locals_dict = generateBuiltinLocalsCode(
locals_node = expression,
context = context
)
order_relevance = [ None ] * 5
return Generator.getBuiltinImportCode(
order_relevance = order_relevance,
module_identifier = Generator.getConstantHandle(
constant = expression.getModuleName(),
context = context
),
globals_dict = globals_dict,
locals_dict = locals_dict,
import_list = Generator.getConstantHandle(
constant = expression.getImportList(),
context = context
),
level = Generator.getConstantHandle(
constant = expression.getLevel(),
context = context
),
context = context
)
def generateBuiltinImportCode(expression, context):
globals_dict = generateExpressionCode(
expression = expression.getGlobals(),
allow_none = True,
context = context
)
locals_dict = generateExpressionCode(
expression = expression.getLocals(),
allow_none = True,
context = context
)
order_relevance = getOrderRelevance(
(
expression.getImportName(),
expression.getGlobals(),
expression.getLocals(),
expression.getFromList(),
expression.getLevel()
),
allow_none = True
)
if globals_dict is None:
globals_dict = Generator.getLoadGlobalsCode(
context = context
)
if locals_dict is None:
provider = expression.getParentVariableProvider()
if provider.isPythonModule():
locals_dict = globals_dict
else:
locals_dict = generateBuiltinLocalsCode(
locals_node = expression,
context = context
)
return Generator.getBuiltinImportCode(
order_relevance = order_relevance,
module_identifier = generateExpressionCode(
expression = expression.getImportName(),
context = context
),
import_list = generateExpressionCode(
expression = expression.getFromList(),
context = context
),
globals_dict = globals_dict,
locals_dict = locals_dict,
level = generateExpressionCode(
expression = expression.getLevel(),
context = context
),
context = context
)
def generateImportStarCode(statement, context):
return Generator.getImportFromStarCode(
module_identifier = generateImportModuleCode(
expression = statement.getModule(),
context = context
),
context = context
)
def generatePrintCode(statement, target_file, context):
expressions = statement.getValues()
values = generateExpressionsCode(
context = context,
expressions = expressions,
)
return Generator.getPrintCode(
target_file = target_file,
identifiers = values,
newline = statement.isNewlinePrint()
)
def generateBranchCode(statement, context):
condition_code = generateConditionCode(
condition = statement.getCondition(),
context = context
)
Generator.pushLineNumberBranch()
yes_codes = generateStatementSequenceCode(
statement_sequence = statement.getBranchYes(),
allow_none = True,
context = context
)
Generator.popLineNumberBranch()
Generator.pushLineNumberBranch()
no_codes = generateStatementSequenceCode(
statement_sequence = statement.getBranchNo(),
allow_none = True,
context = context
)
Generator.popLineNumberBranch()
Generator.mergeLineNumberBranches()
# Do not allow this, optimization must have avoided it.
assert yes_codes is not None, statement
return Generator.getBranchCode(
condition_code = condition_code,
yes_codes = yes_codes,
no_codes = no_codes
)
def generateLoopCode(statement, context):
# The loop is re-entrant, therefore force setting the line number at start
# again, even if the same as before.
Generator.resetLineNumber()
loop_body_codes = generateStatementSequenceCode(
statement_sequence = statement.getLoopBody(),
allow_none = True,
context = context
)
return Generator.getLoopCode(
loop_body_codes = loop_body_codes,
needs_break_exception = statement.needsExceptionBreak(),
needs_continue_exception = statement.needsExceptionContinue()
)
def generateReturnCode(statement, context):
return Generator.getReturnCode(
identifier = generateExpressionCode(
expression = statement.getExpression(),
context = context
),
via_exception = statement.isExceptionDriven(),
context = context
)
def generateGeneratorReturnCode(statement, context):
return Generator.getReturnCode(
identifier = generateExpressionCode(
expression = statement.getExpression(),
context = context
),
# TODO: Use the knowledge about actual need and return immediately if
# possible.
via_exception = True,
context = context
)
def generateStatementCode(statement, context):
try:
statement_context = Contexts.PythonStatementContext( context )
result = _generateStatementCode( statement, statement_context )
local_inits = statement_context.getTempKeeperDecl()
if local_inits:
result = Generator.getBlockCode(
local_inits + result.split( "\n" )
)
return result
except:
Tracing.printError(
"Problem with %r at %s" % (
statement,
statement.getSourceReference()
)
)
raise
def _generateStatementCode(statement, context):
# This is a dispatching function with a branch per statement node type.
# pylint: disable=R0912,R0915
if not statement.isStatement():
statement.dump()
assert False
def makeExpressionCode(expression, allow_none = False):
if allow_none and expression is None:
return None
return generateExpressionCode(
expression = expression,
context = context
)
if statement.isStatementAssignmentVariable():
code = generateAssignmentVariableCode(
variable_ref = statement.getTargetVariableRef(),
value = makeExpressionCode( statement.getAssignSource() ),
context = context
)
elif statement.isStatementAssignmentAttribute():
code = generateAssignmentAttributeCode(
lookup_source = statement.getLookupSource(),
attribute_name = statement.getAttributeName(),
value = statement.getAssignSource(),
context = context
)
elif statement.isStatementAssignmentSubscript():
code = Generator.getSubscriptAssignmentCode(
order_relevance = getOrderRelevance(
statement.getVisitableNodes()
),
subscribed = makeExpressionCode( statement.getSubscribed() ),
subscript = makeExpressionCode( statement.getSubscript() ),
identifier = makeExpressionCode( statement.getAssignSource() ),
)
elif statement.isStatementAssignmentSlice():
code = generateAssignmentSliceCode(
lookup_source = statement.getLookupSource(),
lower = statement.getLower(),
upper = statement.getUpper(),
value = statement.getAssignSource(),
context = context
)
elif statement.isStatementDelVariable():
code = generateDelVariableCode(
variable_ref = statement.getTargetVariableRef(),
tolerant = statement.isTolerant(),
context = context
)
elif statement.isStatementDelSubscript():
code = generateDelSubscriptCode(
subscribed = statement.getSubscribed(),
subscript = statement.getSubscript(),
context = context
)
elif statement.isStatementDelSlice():
code = generateDelSliceCode(
lookup_source = statement.getLookupSource(),
lower = statement.getLower(),
upper = statement.getUpper(),
context = context
)
elif statement.isStatementDelAttribute():
code = generateDelAttributeCode(
statement = statement,
context = context
)
elif statement.isStatementExpressionOnly():
code = Generator.getStatementCode(
identifier = makeExpressionCode(
statement.getExpression()
)
)
elif statement.isStatementPrint():
code = generatePrintCode(
statement = statement,
target_file = makeExpressionCode(
expression = statement.getDestination(),
allow_none = True
),
context = context
)
elif statement.isStatementReturn():
code = generateReturnCode(
statement = statement,
context = context
)
elif statement.isStatementGeneratorReturn():
code = generateGeneratorReturnCode(
statement = statement,
context = context
)
elif statement.isStatementLoop():
code = generateLoopCode(
statement = statement,
context = context
)
elif statement.isStatementConditional():
code = generateBranchCode(
statement = statement,
context = context
)
elif statement.isStatementContinueLoop():
code = Generator.getLoopContinueCode(
needs_exceptions = statement.isExceptionDriven()
)
elif statement.isStatementBreakLoop():
code = Generator.getLoopBreakCode(
needs_exceptions = statement.isExceptionDriven()
)
elif statement.isStatementImportStar():
code = generateImportStarCode(
statement = statement,
context = context
)
elif statement.isStatementTryFinally():
code = generateTryFinallyCode(
statement = statement,
context = context
)
elif statement.isStatementTryExcept():
code = generateTryExceptCode(
statement = statement,
context = context
)
elif statement.isStatementRaiseException():
code = generateRaiseCode(
statement = statement,
context = context
)
elif statement.isStatementExec():
code = generateExecCode(
exec_def = statement,
context = context
)
elif statement.isStatementSpecialUnpackCheck():
code = Generator.getUnpackCheckCode(
iterator_identifier = makeExpressionCode( statement.getIterator() ),
count = statement.getCount()
)
elif statement.isStatementDictOperationRemove():
code = Generator.getDictOperationRemoveCode(
dict_identifier = makeExpressionCode( statement.getDict() ),
key_identifier = makeExpressionCode( statement.getKey() )
)
elif statement.isStatementSetLocals():
code = Generator.getSetLocalsCode(
new_locals_identifier = makeExpressionCode(
statement.getNewLocals()
)
)
else:
assert False, statement.__class__
if code != code.strip():
raise AssertionError( "Code contains leading or trailing whitespace", statement, "'%s'" % code )
return code
def generateStatementSequenceCode( statement_sequence, context,
allow_none = False ):
# The complexity is related to frame guard types, which are also handled in
# here. pylint: disable=R0912
if allow_none and statement_sequence is None:
return None
assert statement_sequence.isStatementsSequence(), statement_sequence
if statement_sequence.isStatementsFrame():
guard_mode = statement_sequence.getGuardMode()
context.setFrameGuardMode( guard_mode )
statements = statement_sequence.getStatements()
codes = []
for statement in statements:
source_ref = statement.getSourceReference()
if Options.shallTraceExecution():
statement_repr = repr( statement )
if Utils.python_version >= 300:
statement_repr = statement_repr.encode( "utf8" )
codes.append(
Generator.getStatementTrace(
source_ref.getAsString(),
statement_repr
)
)
if statement.isStatementsSequence():
code = "\n".join(
generateStatementSequenceCode(
statement_sequence = statement,
context = context
)
)
code = code.strip()
line_code = ""
else:
if statement.needsLineNumber():
line_code = Generator.getLineNumberCode(
source_ref = source_ref
)
else:
line_code = ""
code = generateStatementCode(
statement = statement,
context = context
)
# Cannot happen
assert code != "", statement
if line_code:
code = line_code + ";\n" + code
statement_codes = code.split( "\n" )
assert statement_codes[0].strip() != "", (
"Code '%s'" % code, statement )
assert statement_codes[-1].strip() != "", (
"Code '%s'" % code, statement )
codes += statement_codes
if statement_sequence.isStatementsFrame():
provider = statement_sequence.getParentVariableProvider()
source_ref = statement_sequence.getSourceReference()
needs_preserve = statement_sequence.needsFrameExceptionPreversing()
if guard_mode == "generator":
assert provider.isExpressionFunctionBody() and \
provider.isGenerator()
# TODO: This case should care about "needs_preserve", as for
# Python3 it is actually not a stub of empty code.
code = Generator.getFrameGuardLightCode(
frame_identifier = provider.getCodeName(),
code_identifier = statement_sequence.getCodeObjectHandle(
context = context
),
codes = codes,
context = context
)
elif guard_mode == "pass_through":
assert provider.isExpressionFunctionBody()
# This case does not care about "needs_preserve", as for that kind
# of frame, it is an empty code stub anyway.
code = Generator.getFrameGuardVeryLightCode(
codes = codes,
)
elif guard_mode == "full":
assert provider.isExpressionFunctionBody()
locals_code = Generator.getFrameLocalsUpdateCode(
locals_identifier = Generator.getLoadLocalsCode(
context = context,
provider = provider,
mode = "updated"
)
)
code = Generator.getFrameGuardHeavyCode(
frame_identifier = provider.getCodeName(),
code_identifier = statement_sequence.getCodeObjectHandle(
context
),
locals_code = locals_code,
codes = codes,
needs_preserve = needs_preserve,
context = context
)
elif guard_mode == "once":
code = Generator.getFrameGuardOnceCode(
frame_identifier = provider.getCodeName(),
code_identifier = statement_sequence.getCodeObjectHandle(
context = context
),
locals_identifier = Generator.getLoadLocalsCode(
context = context,
provider = provider,
mode = "updated"
),
codes = codes,
needs_preserve = needs_preserve,
context = context
)
else:
assert False, guard_mode
codes = code.split( "\n" )
assert type( codes ) is list, type( codes )
return codes
def generateModuleCode(global_context, module, module_name, other_modules):
assert module.isPythonModule(), module
context = Contexts.PythonModuleContext(
module_name = module_name,
code_name = Generator.getModuleIdentifier( module_name ),
filename = module.getFilename(),
global_context = global_context,
is_empty = module.getBody() is None
)
statement_sequence = module.getBody()
codes = generateStatementSequenceCode(
statement_sequence = statement_sequence,
allow_none = True,
context = context,
)
codes = codes or []
function_decl_codes = []
function_body_codes = []
extra_declarations = []
for function_body in module.getUsedFunctions():
function_code = generateFunctionBodyCode(
function_body = function_body,
context = context
)
assert type( function_code ) is str
function_body_codes.append( function_code )
if function_body.needsDirectCall():
function_decl = Generator.getFunctionDirectDecl(
function_identifier = function_body.getCodeName(),
closure_variables = function_body.getClosureVariables(),
parameter_variables = function_body.getParameters().getAllVariables(),
file_scope = Generator.getExportScopeCode(
cross_module = function_body.isCrossModuleUsed()
)
)
if function_body.isCrossModuleUsed():
extra_declarations.append( function_decl )
else:
function_decl_codes.append( function_decl )
for _identifier, code in sorted( iterItems( context.getHelperCodes() ) ):
function_body_codes.append( code )
for _identifier, code in sorted( iterItems( context.getDeclarations() ) ):
function_decl_codes.append( code )
function_body_codes = "\n\n".join( function_body_codes )
function_decl_codes = "\n\n".join( function_decl_codes )
metapath_loader_inittab = []
for other_module in other_modules:
metapath_loader_inittab.append(
Generator.getModuleMetapathLoaderEntryCode(
module_name = other_module.getFullName(),
is_shlib = other_module.isPythonShlibModule()
)
)
module_source_code = Generator.getModuleCode(
module_name = module_name,
codes = codes,
metapath_loader_inittab = metapath_loader_inittab,
function_decl_codes = function_decl_codes,
function_body_codes = function_body_codes,
temp_variables = module.getTempVariables(),
context = context,
)
extra_declarations = "\n".join( extra_declarations )
module_header_code = Generator.getModuleDeclarationCode(
module_name = module_name,
extra_declarations = extra_declarations
)
return module_source_code, module_header_code, context
def generateMainCode(codes, context):
return Generator.getMainCode(
context = context,
codes = codes
)
def generateConstantsDeclarationCode(context):
return Generator.getConstantsDeclarationCode(
context = context
)
def generateConstantsDefinitionCode(context):
return Generator.getConstantsDefinitionCode(
context = context
)
def generateHelpersCode():
header_code = Generator.getMakeTuplesCode() + \
Generator.getMakeListsCode() + \
Generator.getMakeDictsCode() + \
Generator.getMakeSetsCode() + \
Generator.getCallsDecls()
body_code = Generator.getCallsCode()
return header_code, body_code
def makeGlobalContext():
return Contexts.PythonGlobalContext()
Nuitka-0.5.0.1/nuitka/codegen/Generator.py 0000644 0001750 0001750 00000171165 12265264105 020555 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Generator for C++ and Python C/API.
This is the actual C++ code generator. It has methods and should be the only
place to know what C++ is like. Ideally it would be possible to replace the
target language by changing this one and the templates, and otherwise nothing
else.
"""
from .Identifiers import (
defaultToNullIdentifier,
defaultToNoneIdentifier,
KeeperAccessIdentifier,
HelperCallIdentifier,
EmptyDictIdentifier,
ThrowingIdentifier,
CallIdentifier,
Identifier
)
from .Indentation import (
getBlockCode,
indented
)
# imported from here pylint: disable=W0611
from .OrderedEvaluation import (
getOrderRelevanceEnforcedCallCode,
getOrderRelevanceEnforcedArgsCode,
_getAssignmentTempKeeperCode,
pickFirst
)
from .LineNumberCodes import (
mergeLineNumberBranches,
pushLineNumberBranch,
popLineNumberBranch,
getLineNumberCode,
resetLineNumber
)
from .TupleCodes import (
getTupleCreationCode,
getMakeTuplesCode
)
from .ListCodes import (
getListCreationCode,
getMakeListsCode
)
from .DictCodes import (
getDictionaryCreationCode,
getMakeDictsCode
)
from .SetCodes import (
getSetCreationCode,
getMakeSetsCode
)
from .CallCodes import (
getCallCodePosKeywordArgs,
getCallCodePosArgsQuick,
getCallCodeKeywordArgs,
getCallCodePosArgs,
getCallCodeNoArgs,
getCallsDecls,
getCallsCode
)
from .ConstantCodes import (
getConstantsInitCode,
getConstantsDeclCode,
getConstantAccess,
getConstantHandle,
getConstantCode,
needsPickleInit,
stream_data
)
from .FunctionCodes import (
getFunctionContextDefinitionCode,
getDirectionFunctionCallCode,
getGeneratorFunctionCode,
getFunctionCreationCode,
getFunctionDirectDecl,
getFunctionMakerCode,
getFunctionMakerDecl,
getFunctionCode,
)
from .ModuleCodes import (
getModuleMetapathLoaderEntryCode,
getModuleDeclarationCode,
getModuleAccessCode,
getModuleIdentifier,
getModuleCode
)
from .MainCodes import getMainCode
# imported from here pylint: enable=W0611
# These are here to be imported from here
# pylint: disable=W0611
from .VariableCodes import (
getVariableAssignmentCode,
getLocalVariableInitCode,
getVariableDelCode,
getVariableHandle,
getVariableCode
)
# pylint: enable=W0611
from .CodeObjectCodes import (
getCodeObjectsDeclCode,
getCodeObjectsInitCode,
)
from . import (
CodeTemplates,
OperatorCodes,
CppStrings
)
from nuitka import (
Constants,
Builtins,
Options,
Utils
)
def getReturnCode(identifier, via_exception, context):
if via_exception:
if identifier is None:
identifier = getConstantHandle(
context = context,
constant = None
)
return "throw ReturnValueException( %s );" % (
identifier.getCodeExportRef()
)
else:
if identifier is not None:
return "return %s;" % identifier.getCodeExportRef()
else:
return "return;"
def getYieldCode(identifier, in_handler):
if in_handler:
return Identifier(
"YIELD_IN_HANDLER( generator, %s )" % (
identifier.getCodeExportRef(),
),
0
)
else:
return Identifier(
"YIELD( generator, %s )" % identifier.getCodeExportRef(),
0
)
def getYieldFromCode(identifier, in_handler):
# TODO: Clarify, if the difference as in getYieldCode is needed.
if False and in_handler:
return Identifier(
"YIELD_FROM_IN_HANDLER( generator, %s )" % (
identifier.getCodeTemporaryRef(),
),
1
)
else:
return Identifier(
"YIELD_FROM( generator, %s )" % identifier.getCodeTemporaryRef(),
1
)
def getMetaclassVariableCode(context):
assert Utils.python_version < 300
return "GET_STRING_DICT_VALUE( moduledict_%s, (Nuitka_StringObject *)%s )" % (
context.getModuleCodeName(),
getConstantCode(
constant = "__metaclass__",
context = context
)
)
def getBuiltinImportCode( context, order_relevance, module_identifier,
globals_dict, locals_dict, import_list, level ):
assert type( module_identifier ) is not str
assert type( globals_dict ) is not str
assert type( locals_dict ) is not str
return getOrderRelevanceEnforcedArgsCode(
helper = "IMPORT_MODULE",
export_ref = 0,
ref_count = 1,
tmp_scope = "import",
order_relevance = order_relevance,
args = (
module_identifier,
globals_dict,
locals_dict,
import_list,
level
),
context = context
)
def getImportFromStarCode(context, module_identifier):
if not context.hasLocalsDict():
return "IMPORT_MODULE_STAR( %s, true, %s );" % (
getModuleAccessCode(
context = context
),
module_identifier.getCodeTemporaryRef()
)
else:
return "IMPORT_MODULE_STAR( locals_dict.asObject0(), false, %s );" % (
module_identifier.getCodeTemporaryRef()
)
def getMaxIndexCode():
return Identifier( "PY_SSIZE_T_MAX", 0 )
def getMinIndexCode():
return Identifier( "0", 0 )
def getIndexValueCode(number):
return Identifier( "%s" % number, 0 )
def getIndexCode(identifier):
return Identifier(
"CONVERT_TO_INDEX( %s )" % identifier.getCodeTemporaryRef(),
0
)
def getUnpackNextCode(iterator_identifier, count):
return Identifier(
"UNPACK_NEXT( %s, %d )" % (
iterator_identifier.getCodeTemporaryRef(),
count - 1
),
1
)
def getUnpackCheckCode(iterator_identifier, count):
return "UNPACK_ITERATOR_CHECK( %s, %d );" % (
iterator_identifier.getCodeTemporaryRef(),
count
)
def getSpecialAttributeLookupCode(attribute, source):
assert attribute.getConstant not in ( "__dict__", "__class__" )
return Identifier(
"LOOKUP_SPECIAL( %s, %s )" % (
source.getCodeTemporaryRef(),
attribute.getCodeTemporaryRef()
),
1
)
def getAttributeLookupCode(attribute, source):
assert attribute.getConstant not in ( "__dict__", "__class__" )
return Identifier(
"LOOKUP_ATTRIBUTE( %s, %s )" % (
source.getCodeTemporaryRef(),
attribute.getCodeTemporaryRef()
),
1
)
def getAttributeLookupDictSlotCode(source):
return Identifier(
"LOOKUP_ATTRIBUTE_DICT_SLOT( %s )" % (
source.getCodeTemporaryRef(),
),
1
)
def getAttributeLookupClassSlotCode(source):
return Identifier(
"LOOKUP_ATTRIBUTE_CLASS_SLOT( %s )" % (
source.getCodeTemporaryRef(),
),
1
)
def getAttributeCheckCode(context, order_relevance, attribute, source):
return getBoolFromCode(
code = getAttributeCheckBoolCode(
order_relevance = order_relevance,
source = source,
attribute = attribute,
context = context
)
)
def getAttributeCheckBoolCode(context, order_relevance, source, attribute):
return getOrderRelevanceEnforcedArgsCode(
helper = "HAS_ATTRIBUTE",
export_ref = 0,
ref_count = None,
tmp_scope = "hasattr",
order_relevance = order_relevance,
args = ( source, attribute ),
context = context
)
def getAttributeGetCode(context, order_relevance, source, attribute, default):
return getOrderRelevanceEnforcedArgsCode(
helper = "BUILTIN_GETATTR",
export_ref = 0,
ref_count = 1,
tmp_scope = "getattr",
order_relevance = order_relevance,
args = (
source,
attribute,
defaultToNullIdentifier(default)
),
context = context
)
def getAttributeSetCode(context, order_relevance, attribute, source, value):
result = getOrderRelevanceEnforcedArgsCode(
helper = "BUILTIN_SETATTR",
export_ref = 0,
ref_count = None,
tmp_scope = "setattr",
order_relevance = order_relevance,
args = (
source,
attribute,
value
),
context = context
)
# It's a void function "BUILTIN_SETATTR", but "setattr" returns "None".
return Identifier(
"( %s, Py_None )" % result,
0
)
def getImportNameCode(import_name, module):
return Identifier(
"IMPORT_NAME( %s, %s )" % (
module.getCodeTemporaryRef(),
import_name.getCodeTemporaryRef()
),
1
)
def getSubscriptLookupCode(context, order_relevance, subscript, source):
helper = "LOOKUP_SUBSCRIPT"
suffix_args = []
if subscript.isConstantIdentifier():
constant = subscript.getConstant()
if Constants.isIndexConstant( constant ):
constant_value = int( constant )
if abs( constant_value ) < 2**31:
helper = "LOOKUP_SUBSCRIPT_CONST"
suffix_args = [ "%d" % constant ]
return getOrderRelevanceEnforcedArgsCode(
helper = helper,
export_ref = 0,
ref_count = 1,
tmp_scope = "subscr",
order_relevance = order_relevance,
args = ( source, subscript ),
suffix_args = suffix_args,
context = context
)
def getHasKeyBoolCode(source, key):
return "HAS_KEY( %s, %s )" % (
source.getCodeTemporaryRef(),
key.getCodeTemporaryRef()
)
def getSliceLookupCode(order_relevance, source, lower, upper, context):
return getOrderRelevanceEnforcedArgsCode(
helper = "LOOKUP_SLICE",
export_ref = 0,
ref_count = 1,
tmp_scope = "slice",
order_relevance = order_relevance,
args = (
source,
defaultToNoneIdentifier(lower),
defaultToNoneIdentifier(upper)
),
context = context
)
def getSliceLookupIndexesCode(lower, upper, source):
return Identifier(
"LOOKUP_INDEX_SLICE( %s, %s, %s )" % (
source.getCodeTemporaryRef(),
lower.getCodeTemporaryRef(),
upper.getCodeTemporaryRef()
),
1
)
def getSliceObjectCode(order_relevance, lower, upper, step, context):
return getOrderRelevanceEnforcedArgsCode(
helper = "MAKE_SLICEOBJ",
export_ref = 0,
ref_count = 1,
tmp_scope = "sliceobj",
order_relevance = order_relevance,
args = (
defaultToNoneIdentifier(lower),
defaultToNoneIdentifier(upper),
defaultToNoneIdentifier(step)
),
context = context
)
def getStatementCode(identifier):
return identifier.getCodeDropRef() + ";"
def getOperationCode(context, order_relevance, operator, identifiers):
# This needs to have one return per operation of Python, and there are many
# of these, pylint: disable=R0911
prefix_args = []
ref_count = 1
if operator == "Pow":
helper = "POWER_OPERATION"
elif operator == "IPow":
helper = "POWER_OPERATION_INPLACE"
elif operator == "Add":
helper = "BINARY_OPERATION_ADD"
elif operator == "Sub":
helper = "BINARY_OPERATION_SUB"
elif operator == "Div":
helper = "BINARY_OPERATION_DIV"
elif operator == "Mult":
helper = "BINARY_OPERATION_MUL"
elif operator == "Mod":
helper = "BINARY_OPERATION_REMAINDER"
elif len( identifiers ) == 2:
helper = "BINARY_OPERATION"
prefix_args = [ OperatorCodes.binary_operator_codes[ operator ] ]
elif len( identifiers ) == 1:
impl_helper, ref_count = OperatorCodes.unary_operator_codes[ operator ]
helper = "UNARY_OPERATION"
prefix_args = [ impl_helper ]
else:
assert False, (operator, identifiers)
return getOrderRelevanceEnforcedArgsCode(
helper = helper,
export_ref = 0,
ref_count = ref_count,
tmp_scope = "op",
order_relevance = order_relevance,
prefix_args = prefix_args,
args = identifiers,
context = context
)
def getPrintCode(newline, identifiers, target_file):
print_elements_code = []
for identifier in identifiers:
print_elements_code.append(
CodeTemplates.template_print_value % {
"print_value" : identifier.getCodeTemporaryRef(),
"target_file" : "target_file"
if target_file is not None
else "NULL"
}
)
if newline:
print_elements_code.append(
CodeTemplates.template_print_newline % {
"target_file" : "target_file"
if target_file is not None
else "NULL"
}
)
if target_file is not None:
return CodeTemplates.template_print_statement % {
"target_file" : defaultToNullIdentifier(
target_file
).getCodeExportRef(),
"print_elements_code" : indented( print_elements_code )
}
else:
return "\n".join( print_elements_code )
def getConditionalExpressionCode( condition_code, identifier_no,
identifier_yes ):
if identifier_yes.getCheapRefCount() == identifier_no.getCheapRefCount():
if identifier_yes.getCheapRefCount() == 0:
codes_yes = identifier_yes.getCodeTemporaryRef()
codes_no = identifier_no.getCodeTemporaryRef()
ref_count = 0
else:
codes_yes = identifier_yes.getCodeExportRef()
codes_no = identifier_no.getCodeExportRef()
ref_count = 1
else:
codes_yes = identifier_yes.getCodeExportRef()
codes_no = identifier_no.getCodeExportRef()
ref_count = 1
return Identifier(
CodeTemplates.template_conditional_expression % {
"condition" : condition_code,
"yes" : codes_yes,
"no" : codes_no
},
ref_count
)
def getBranchCode(condition_code, yes_codes, no_codes):
assert yes_codes or no_codes
if no_codes is None:
return CodeTemplates.template_branch_one % {
"condition" : condition_code,
"branch_code" : indented(
yes_codes if yes_codes is not None else ""
)
}
else:
assert no_codes, no_codes
return CodeTemplates.template_branch_two % {
"condition" : condition_code,
"branch_yes_code" : indented(
yes_codes if yes_codes is not None else ""
),
"branch_no_code" : indented( no_codes )
}
def getLoopContinueCode(needs_exceptions):
if needs_exceptions:
return "throw ContinueException();"
else:
return "CONSIDER_THREADING(); continue;"
def getLoopBreakCode(needs_exceptions):
if needs_exceptions:
return "throw BreakException();"
else:
return "break;"
def getComparisonExpressionCode( context, comparator, order_relevance, left,
right ):
# There is an awful lot of cases, pylint: disable=R0912
if comparator in OperatorCodes.normal_comparison_codes:
helper = OperatorCodes.normal_comparison_codes[ comparator ]
assert helper.startswith("SEQUENCE_CONTAINS")
ref_count = 0
elif comparator in OperatorCodes.rich_comparison_codes:
helper = "RICH_COMPARE_%s" % (
OperatorCodes.rich_comparison_codes[ comparator ]
)
ref_count = 1
elif comparator == "Is":
# This is special, and "==" enforces order of evalulation already, or so
# we believe.
return getBoolFromCode(
code = "( %s == %s )" % (
left.getCodeTemporaryRef(),
right.getCodeTemporaryRef()
)
)
elif comparator == "IsNot":
# This is special, and "!=" enforces order of evalulation already, or so
# we believe.
return getBoolFromCode(
code = "( %s != %s )" % (
left.getCodeTemporaryRef(),
right.getCodeTemporaryRef()
)
)
else:
assert False, comparator
return getOrderRelevanceEnforcedArgsCode(
helper = helper,
export_ref = 0,
ref_count = ref_count,
tmp_scope = "cmp",
order_relevance = order_relevance,
args = ( left, right ),
context = context
)
def getComparisonExpressionBoolCode(context, comparator, order_relevance, left,
right):
# There is an awful lot of cases, pylint: disable=R0912
if comparator in OperatorCodes.normal_comparison_codes:
helper = "%s_BOOL" % (
OperatorCodes.normal_comparison_codes[ comparator ]
)
assert helper.startswith("SEQUENCE_CONTAINS")
elif comparator in OperatorCodes.rich_comparison_codes:
helper = "RICH_COMPARE_BOOL_%s" % (
OperatorCodes.rich_comparison_codes[ comparator ]
)
elif comparator == "Is":
# This is special, and "==" enforces order of evalulation already, or so
# we believe.
return "( %s == %s )" % (
left.getCodeTemporaryRef(),
right.getCodeTemporaryRef()
)
elif comparator == "IsNot":
# This is special, and "!=" enforces order of evalulation already, or so
# we believe.
return "( %s != %s )" % (
left.getCodeTemporaryRef(),
right.getCodeTemporaryRef()
)
else:
assert False, comparator
return getOrderRelevanceEnforcedArgsCode(
helper = helper,
export_ref = 0,
ref_count = None,
tmp_scope = "cmp",
order_relevance = order_relevance,
args = ( left, right ),
context = context
)
def getConditionNotBoolCode(condition):
return "(!( %s ))" % condition
def getConditionAndCode(operands):
return "( %s )" % " && ".join( operands )
def getConditionOrCode(operands):
return "( %s )" % " || ".join( operands )
def getConditionSelectionCode(condition_code, yes_code, no_code):
return "( %s ) ? ( %s ) : ( %s )" % (
condition_code,
yes_code,
no_code
)
def getConditionCheckTrueCode(condition):
return "CHECK_IF_TRUE( %s )" % condition.getCodeTemporaryRef()
def getConditionCheckFalseCode(condition):
return "CHECK_IF_FALSE( %s )" % condition.getCodeTemporaryRef()
def getTrueExpressionCode():
return "true"
def getFalseExpressionCode():
return "false"
def getAttributeAssignmentCode( order_relevance, target, attribute,
identifier ):
assert attribute.getConstant not in ( "__dict__", "__class__" )
return getOrderRelevanceEnforcedCallCode(
order_relevance = order_relevance,
helper = "SET_ATTRIBUTE",
names = ( "identifier", "target", "attribute" ),
values = ( identifier, target, attribute )
)
def getAttributeAssignmentDictSlotCode(order_relevance, target, identifier):
""" Get code for special case target.__dict__ = value """
return getOrderRelevanceEnforcedCallCode(
order_relevance = order_relevance,
helper = "SET_ATTRIBUTE_DICT_SLOT",
names = ( "identifier", "target" ),
values = ( identifier, target )
)
def getAttributeAssignmentClassSlotCode(order_relevance, target, identifier):
""" Get code for special case target.__class__ = value """
return getOrderRelevanceEnforcedCallCode(
order_relevance = order_relevance,
helper = "SET_ATTRIBUTE_CLASS_SLOT",
names = ( "identifier", "target" ),
values = ( identifier, target )
)
def getAttributeDelCode(target, attribute):
return "DEL_ATTRIBUTE( %s, %s );" % (
target.getCodeTemporaryRef(),
attribute.getCodeTemporaryRef()
)
def getSliceAssignmentIndexesCode(target, lower, upper, identifier):
return "SET_INDEX_SLICE( %s, %s, %s, %s );" % (
target.getCodeTemporaryRef(),
lower.getCodeTemporaryRef(),
upper.getCodeTemporaryRef(),
identifier.getCodeTemporaryRef()
)
def getSliceAssignmentCode(order_relevance, target, lower, upper, identifier):
return getOrderRelevanceEnforcedCallCode(
order_relevance = order_relevance,
helper = "SET_SLICE",
names = ( "identifier", "target", "lower", "upper" ),
values = (
identifier,
target,
defaultToNoneIdentifier(lower),
defaultToNoneIdentifier(upper)
)
)
def getSliceDelCode(target, lower, upper):
return "DEL_SLICE( %s, %s, %s );" % (
target.getCodeTemporaryRef(),
defaultToNoneIdentifier(lower).getCodeTemporaryRef(),
defaultToNoneIdentifier(upper).getCodeTemporaryRef()
)
def getLoopCode( loop_body_codes, needs_break_exception,
needs_continue_exception ):
if needs_break_exception and needs_continue_exception:
while_loop_template = \
CodeTemplates.template_loop_break_continue_catching
indentation = 2
elif needs_break_exception:
while_loop_template = CodeTemplates.template_loop_break_catching
indentation = 2
elif needs_continue_exception:
while_loop_template = CodeTemplates.template_loop_continue_catching
indentation = 2
else:
while_loop_template = CodeTemplates.template_loop_simple
indentation = 1
return while_loop_template % {
"loop_body_codes" : indented(
loop_body_codes if loop_body_codes is not None else "",
indentation
),
}
def getAssignmentTempKeeperCode(source_identifier, variable, context):
ref_count = source_identifier.getCheapRefCount()
variable_name = variable.getName()
assert not ref_count or variable.getReferenced().getNeedsFree(), \
( variable, variable.getReferenced().getNeedsFree(), ref_count,
source_identifier, source_identifier.__class__ )
return _getAssignmentTempKeeperCode(
source_identifier = source_identifier,
variable_name = variable_name,
context = context
)
def getTempKeeperHandle(variable, context):
variable_name = variable.getName()
ref_count = context.getTempKeeperRefCount( variable_name )
if ref_count == 1:
return KeeperAccessIdentifier(
"%s.asObject1()" % variable_name
)
else:
# TODO: Could create an identifier, where 0 is just cheap, and 1 is
# still available, may give nicer to read code occasionally.
return Identifier(
"%s.asObject0()" % variable_name,
0
)
def getSubscriptAssignmentCode( order_relevance, subscribed, subscript,
identifier ):
helper = "SET_SUBSCRIPT"
suffix_args = []
if subscript.isConstantIdentifier():
constant = subscript.getConstant()
if Constants.isIndexConstant( constant ):
constant_value = int( constant )
if abs( constant_value ) < 2**31:
helper = "SET_SUBSCRIPT_CONST"
suffix_args = [ "%d" % constant ]
return getOrderRelevanceEnforcedCallCode(
order_relevance = order_relevance,
helper = helper,
names = ( "identifier", "subscribed", "subscript" ),
values = ( identifier, subscribed, subscript ),
suffix_args = suffix_args
)
def getSubscriptDelCode(order_relevance, subscribed, subscript):
return getOrderRelevanceEnforcedCallCode(
order_relevance = order_relevance,
helper = "DEL_SUBSCRIPT",
names = ( "subscribed", "subscript" ),
values = ( subscribed, subscript )
)
def getTryFinallyCode( context, needs_continue, needs_break,
needs_return_value_catch, needs_return_value_reraise,
aborting, code_tried, code_final, try_count ):
tb_making = getTracebackMakingIdentifier( context )
rethrow_setups = ""
rethrow_catchers = ""
rethrow_raisers = ""
values = {
"try_count" : try_count
}
if needs_continue:
rethrow_setups += CodeTemplates.try_finally_template_setup_continue % (
values
)
rethrow_catchers += CodeTemplates.try_finally_template_catch_continue % values
rethrow_raisers += CodeTemplates.try_finally_template_reraise_continue % values
if needs_break:
rethrow_setups += CodeTemplates.try_finally_template_setup_break % values
rethrow_catchers += CodeTemplates.try_finally_template_catch_break % values
rethrow_raisers += CodeTemplates.try_finally_template_reraise_break % values
if needs_return_value_catch:
rethrow_setups += CodeTemplates.try_finally_template_setup_return_value % values
rethrow_catchers += CodeTemplates.try_finally_template_catch_return_value % values
if needs_return_value_reraise:
rethrow_raisers += CodeTemplates.try_finally_template_reraise_return_value % values
elif not aborting:
if context.getFunction().isGenerator():
rethrow_raisers += CodeTemplates.try_finally_template_indirect_generator_return_value % values
else:
rethrow_raisers += CodeTemplates.try_finally_template_indirect_return_value % values
else:
if context.getFunction().isGenerator():
rethrow_raisers += CodeTemplates.try_finally_template_direct_generator_return_value % values
else:
rethrow_raisers += CodeTemplates.try_finally_template_direct_return_value % values
result = CodeTemplates.try_finally_template % {
"try_count" : try_count,
"tried_code" : indented( code_tried ),
"final_code" : indented( code_final, 0 ),
"tb_making" : tb_making.getCodeExportRef(),
"rethrow_setups" : rethrow_setups,
"rethrow_catchers" : rethrow_catchers,
"rethrow_raisers" : rethrow_raisers,
}
if not rethrow_raisers:
result = result.rstrip()
return result
def getTryExceptHandlerCode( exception_identifiers, handler_code,
needs_frame_detach, first_handler ):
exception_code = []
cond_keyword = "if" if first_handler else "else if"
if exception_identifiers:
exception_code.append(
"%s ( %s )" % (
cond_keyword,
" || ".join(
"_exception.matches( %s )" % (
exception_identifier.getCodeTemporaryRef()
)
for exception_identifier in
exception_identifiers
)
)
)
else:
exception_code.append(
"%s (true)" % cond_keyword
)
if handler_code is None:
handler_code = []
if needs_frame_detach:
handler_code.insert(
0,
CodeTemplates.template_setup_except_handler_detaching % {
}
)
exception_code += getBlockCode(
handler_code
).split( "\n" )
return exception_code
def getTryExceptCode(context, code_tried, handler_codes):
exception_code = list( handler_codes )
exception_code += CodeTemplates.try_except_reraise_unmatched_template.split( "\n" )
tb_making = getTracebackMakingIdentifier( context )
return CodeTemplates.try_except_template % {
"tried_code" : indented( code_tried or "" ),
"exception_code" : indented( exception_code ),
"guard_class" : context.getFrameGuardClass(),
"tb_making" : tb_making.getCodeExportRef(),
}
def getTryNextExceptStopIterationIdentifier(context):
try_count = context.allocateTryNumber()
return Identifier( "_tmp_unpack_%d" % try_count, 1 )
def getTryNextExceptStopIterationCode( source_identifier, handler_code,
assign_code, temp_identifier ):
return CodeTemplates.template_try_next_except_stop_iteration % {
"temp_var" : temp_identifier.getCode(),
"handler_code" : indented( handler_code ),
"assignment_code" : assign_code,
"source_identifier" : source_identifier.getCodeTemporaryRef()
}
def getRaiseExceptionWithCauseCode( context, order_relevance, exception_type,
exception_cause ):
# Must enforce tb_maker to be last.
exception_tb_maker = getTracebackMakingIdentifier(
context = context
)
return getOrderRelevanceEnforcedCallCode(
order_relevance = order_relevance + [ None ],
helper = "RAISE_EXCEPTION_WITH_CAUSE",
names = (
"exception_type", "exception_cause", "exception_tb"
),
values = (
exception_type, exception_cause, exception_tb_maker
)
)
def getRaiseExceptionWithTypeCode(context, order_relevance, exception_type):
# Must enforce tb_maker to be last.
exception_tb_maker = getTracebackMakingIdentifier(
context = context
)
return getOrderRelevanceEnforcedCallCode(
order_relevance = order_relevance + [ None ],
helper = "RAISE_EXCEPTION_WITH_TYPE",
names = (
"exception_type", "exception_tb"
),
values = (
exception_type, exception_tb_maker
)
)
def getRaiseExceptionWithValueCode( context, order_relevance, exception_type,
exception_value, implicit ):
# Must enforce tb_maker to be last.
exception_tb_maker = getTracebackMakingIdentifier(
context = context
)
if implicit:
helper = "RAISE_EXCEPTION_WITH_VALUE_NO_NORMALIZE"
else:
helper = "RAISE_EXCEPTION_WITH_VALUE"
return getOrderRelevanceEnforcedCallCode(
order_relevance = order_relevance + [ None ],
helper = helper,
names = (
"exception_type", "exception_value", "exception_tb"
),
values = (
exception_type, exception_value, exception_tb_maker
)
)
def getRaiseExceptionWithTracebackCode( order_relevance, exception_type,
exception_value, exception_tb ):
return getOrderRelevanceEnforcedCallCode(
order_relevance = order_relevance,
helper = "RAISE_EXCEPTION_WITH_TRACEBACK",
names = (
"exception_type", "exception_value", "exception_tb"
),
values = (
exception_type, exception_value, exception_tb
)
)
def getReRaiseExceptionCode(local, final):
if local:
thrower_code = CodeTemplates.try_except_reraise_template % {}
else:
thrower_code = "RERAISE_EXCEPTION();"
if final:
return CodeTemplates.try_except_reraise_finally_template % {
"try_count" : final,
"thrower_code" : thrower_code
}
else:
return thrower_code
def getRaiseExceptionExpressionCode(context, exception_type, exception_value):
# Order is supposed to not matter, as these were run time detected and
# contain no side effects.
exception_tb_maker = getTracebackMakingIdentifier(
context = context
)
return ThrowingIdentifier(
"THROW_EXCEPTION( %s, %s, %s )" % (
exception_type.getCodeExportRef(),
exception_value.getCodeExportRef(),
exception_tb_maker.getCodeExportRef()
)
)
def getSideEffectsCode(side_effects, identifier):
assert side_effects
side_effects_code = ", ".join(
side_effect.getCodeTemporaryRef()
for side_effect in
side_effects
)
if identifier.getCheapRefCount() == 0:
return Identifier(
"( %s, %s )" % (
side_effects_code,
identifier.getCodeTemporaryRef()
),
0
)
else:
return Identifier(
"( %s, %s )" % (
side_effects_code,
identifier.getCodeExportRef()
),
1
)
def getBuiltinRefCode(context, builtin_name):
return Identifier(
"LOOKUP_BUILTIN( %s )" % getConstantCode(
constant = builtin_name,
context = context
),
0
)
def getBuiltinOriginalRefCode(builtin_name):
return Identifier(
"_python_original_builtin_value_%s" % builtin_name,
0
)
def getBuiltinAnonymousRefCode(builtin_name):
return Identifier(
"(PyObject *)%s" % Builtins.builtin_anon_codes[ builtin_name ],
0
)
def getExceptionRefCode(exception_type):
if exception_type == "NotImplemented":
return Identifier(
"Py_NotImplemented",
0
)
return Identifier(
"PyExc_%s" % exception_type,
0
)
def getMakeBuiltinExceptionCode(context, order_relevance, exception_type,
exception_args):
return getCallCodePosArgs(
called_identifier = getExceptionRefCode( exception_type ),
argument_tuple = getTupleCreationCode(
element_identifiers = exception_args,
order_relevance = order_relevance,
context = context,
),
order_relevance = ( None, None ),
context = context
)
def _getLocalVariableList(provider):
if provider.isExpressionFunctionBody():
# Sort parameter variables of functions to the end.
start_part = []
end_part = []
for variable in provider.getVariables():
if variable.isParameterVariable():
end_part.append( variable )
else:
start_part.append( variable )
variables = start_part + end_part
include_closure = not provider.isUnoptimized() and \
not provider.isClassDictCreation()
else:
variables = provider.getVariables()
include_closure = True
return [
variable
for variable in
variables
if not variable.isModuleVariable()
if not variable.isMaybeLocalVariable()
if ( not variable.isClosureReference() or include_closure )
]
def getLoadDirCode(context, provider):
if provider.isPythonModule():
globals_identifier = getLoadGlobalsCode(
context = context
)
return Identifier(
"PyDict_Keys( %s )" % (
globals_identifier.getCodeTemporaryRef(),
),
1
)
else:
if context.hasLocalsDict():
locals_identifier = getLoadLocalsCode(
context = context,
provider = provider,
mode = "updated"
)
return Identifier(
"PyDict_Keys( %s )" % (
locals_identifier.getCodeTemporaryRef()
),
1
)
else:
local_list = _getLocalVariableList(
provider = provider
)
result = getListCreationCode(
context = context,
order_relevance = (),
element_identifiers = (),
)
for local_var in local_list:
if local_var.isTempVariableReference():
result = Identifier(
"%s.updateLocalsDir( %s, %s )" % (
getVariableCode(
context = context,
variable = local_var
),
getConstantCode(
constant = local_var.getReferenced().getName(),
context = context
),
result.getCodeTemporaryRef()
),
0
)
else:
result = Identifier(
"%s.updateLocalsDir( %s )" % (
getVariableCode(
context = context,
variable = local_var
),
result.getCodeTemporaryRef()
),
0
)
return result
def getLoadVarsCode(identifier):
return Identifier(
"LOOKUP_VARS( %s )" % identifier.getCodeTemporaryRef(),
1
)
def getLoadGlobalsCode(context):
return Identifier(
"((PyModuleObject *)%(module_identifier)s)->md_dict" % {
"module_identifier" : getModuleAccessCode( context )
},
0
)
def getLoadLocalsCode(context, provider, mode):
def _getUpdateLocalsDictCode(context, result, local_var, ref_count):
if local_var.isTempVariableReference():
result = Identifier(
"%s.updateLocalsDict( %s, %s )" % (
getVariableCode(
context = context,
variable = local_var
),
getConstantCode(
constant = local_var.getReferenced().getName(),
context = context
),
result.getCodeExportRef() if ref_count else result.getCodeTemporaryRef()
),
ref_count
)
else:
result = Identifier(
"%s.updateLocalsDict( %s )" % (
getVariableCode(
context = context,
variable = local_var
),
result.getCodeExportRef() if ref_count else result.getCodeTemporaryRef()
),
ref_count
)
return result
if provider.isPythonModule():
return getLoadGlobalsCode( context )
elif not context.hasLocalsDict():
local_list = _getLocalVariableList(
provider = provider,
)
result = EmptyDictIdentifier()
for local_var in local_list:
result = _getUpdateLocalsDictCode(
local_var = local_var,
result = result,
context = context,
ref_count = result.getRefCount()
)
return result
else:
if mode == "copy":
return Identifier(
"PyDict_Copy( locals_dict.asObject0() )",
1
)
elif mode == "updated":
local_list = _getLocalVariableList(
provider = provider
)
result = Identifier(
"locals_dict.asObject0()",
0
)
for local_var in local_list:
result = _getUpdateLocalsDictCode(
local_var = local_var,
result = result,
context = context,
ref_count = result.getRefCount()
)
return result
else:
assert False
def getSetLocalsCode(new_locals_identifier):
return "locals_dict.assign1( %s );" % (
new_locals_identifier.getCodeExportRef()
)
def getStoreLocalsCode(context, source_identifier, provider):
assert not provider.isPythonModule()
code = ""
for variable in provider.getVariables():
if not variable.isModuleVariable() and \
not variable.isMaybeLocalVariable():
key_identifier = getConstantHandle(
context = context,
constant = variable.getName()
)
var_assign_code = getVariableAssignmentCode(
context = context,
variable = variable,
identifier = getSubscriptLookupCode(
order_relevance = ( None, None ),
subscript = key_identifier,
source = source_identifier,
context = context
)
)
# This ought to re-use the condition code stuff.
code += "if ( %s )\n" % getHasKeyBoolCode(
source = source_identifier,
key = key_identifier
)
code += getBlockCode( var_assign_code ) + "\n"
return code.rstrip( "\n" )
def getFutureFlagsCode(future_spec):
flags = future_spec.asFlags()
if flags:
return " | ".join( flags )
else:
return 0
def getCompileCode(context, order_relevance, source_identifier,
filename_identifier, mode_identifier, future_flags):
return getOrderRelevanceEnforcedArgsCode(
helper = "COMPILE_CODE",
export_ref = 0,
ref_count = 1,
tmp_scope = "compile",
order_relevance = order_relevance,
args = (
source_identifier,
filename_identifier,
mode_identifier,
future_flags
),
context = context
)
def getEvalCode(context, order_relevance, exec_code, filename_identifier,
globals_identifier, locals_identifier, mode_identifier,
future_flags):
code_identifier = getCompileCode(
order_relevance = [ None ] * 4, # TODO: Probably wrong.
source_identifier = exec_code,
filename_identifier = filename_identifier,
mode_identifier = mode_identifier,
future_flags = Identifier( str( future_flags ), 0 ),
context = context
)
return getOrderRelevanceEnforcedArgsCode(
helper = "EVAL_CODE",
export_ref = 0,
ref_count = 1,
tmp_scope = "eval",
order_relevance = order_relevance,
args = (
code_identifier,
globals_identifier,
locals_identifier
),
context = context
)
def getExecCode(context, exec_code, globals_identifier, locals_identifier, future_flags, provider, source_ref):
# Filename with origin if improved mode.
if Options.isFullCompat():
filename_identifier = getConstantCode(
constant = "",
context = context
)
else:
filename_identifier = getConstantCode(
constant = "" % source_ref.getAsString(),
context = context
)
result = CodeTemplates.exec_template % {
"globals_identifier" : globals_identifier.getCodeExportRef(),
"locals_identifier" : locals_identifier.getCodeExportRef(),
"source_identifier" : exec_code.getCodeTemporaryRef(),
"filename_identifier" : filename_identifier,
"mode_identifier" : getConstantCode(
constant = "exec",
context = context
),
"future_flags" : future_flags,
}
if provider.isExpressionFunctionBody() and provider.isUnqualifiedExec():
locals_temp_identifier = Identifier( "locals_source", 0 )
result += CodeTemplates.exec_copy_back_template % {
"store_locals_code" : indented(
getStoreLocalsCode(
context = context,
source_identifier = locals_temp_identifier,
provider = provider
),
)
}
return getBlockCode( result )
def getBuiltinSuperCode( order_relevance, type_identifier, object_identifier,
context ):
return getOrderRelevanceEnforcedArgsCode(
helper = "BUILTIN_SUPER",
export_ref = 0,
ref_count = 1,
tmp_scope = "super",
order_relevance = order_relevance,
args = (
defaultToNullIdentifier(type_identifier),
defaultToNullIdentifier(object_identifier)
),
context = context
)
def getBuiltinIsinstanceCode( context, order_relevance, inst_identifier,
cls_identifier ):
return getBoolFromCode(
code = getBuiltinIsinstanceBoolCode(
order_relevance = order_relevance,
inst_identifier = inst_identifier,
cls_identifier = cls_identifier,
context = context
)
)
def getBuiltinIsinstanceBoolCode( context, order_relevance, inst_identifier,
cls_identifier ):
return getOrderRelevanceEnforcedArgsCode(
helper = "BUILTIN_ISINSTANCE_BOOL",
export_ref = 0,
ref_count = None,
tmp_scope = "isinstance",
order_relevance = order_relevance,
args = (
inst_identifier,
cls_identifier
),
context = context
)
def getBuiltinOpenCode(context, order_relevance, filename, mode, buffering):
return getOrderRelevanceEnforcedArgsCode(
helper = "OPEN_FILE",
export_ref = 0,
ref_count = 1,
tmp_scope = "open",
order_relevance = order_relevance,
args = (
defaultToNullIdentifier(filename),
defaultToNullIdentifier(mode),
defaultToNullIdentifier(buffering)
),
context = context
)
def getBuiltinLenCode(identifier):
return HelperCallIdentifier(
"BUILTIN_LEN", identifier
)
def getBuiltinDir1Code(identifier):
return HelperCallIdentifier(
"BUILTIN_DIR1", identifier
)
def getBuiltinRange1Code(value):
return HelperCallIdentifier(
"BUILTIN_RANGE", value
)
def getBuiltinRange2Code(order_relevance, low, high, context):
return getOrderRelevanceEnforcedArgsCode(
helper = "BUILTIN_RANGE2",
export_ref = 0,
ref_count = 1,
tmp_scope = "range",
order_relevance = order_relevance,
args = (
low,
high
),
context = context
)
def getBuiltinRange3Code(order_relevance, low, high, step, context):
return getOrderRelevanceEnforcedArgsCode(
helper = "BUILTIN_RANGE3",
export_ref = 0,
ref_count = 1,
tmp_scope = "range",
order_relevance = order_relevance,
args = (
low,
high,
step
),
context = context
)
def getBuiltinXrangeCode(order_relevance, low, high, step, context):
return getOrderRelevanceEnforcedArgsCode(
helper = "BUILTIN_XRANGE",
export_ref = 0,
ref_count = 1,
tmp_scope = "xrange",
order_relevance = order_relevance,
args = (
low,
defaultToNullIdentifier(high),
defaultToNullIdentifier(step)
),
context = context
)
def getBuiltinChrCode(value):
return HelperCallIdentifier( "BUILTIN_CHR", value )
def getBuiltinOrdCode(value):
return HelperCallIdentifier( "BUILTIN_ORD", value )
def getBuiltinBinCode(value):
return HelperCallIdentifier( "BUILTIN_BIN", value )
def getBuiltinOctCode(value):
return HelperCallIdentifier( "BUILTIN_OCT", value )
def getBuiltinHexCode(value):
return HelperCallIdentifier( "BUILTIN_HEX", value )
def getBuiltinType1Code(value):
return HelperCallIdentifier( "BUILTIN_TYPE1", value )
def getBuiltinIter1Code(value):
return HelperCallIdentifier( "MAKE_ITERATOR", value )
def getBuiltinIter2Code( context, order_relevance, callable_identifier,
sentinel_identifier ):
return getOrderRelevanceEnforcedArgsCode(
helper = "BUILTIN_ITER2",
export_ref = 0,
ref_count = 1,
tmp_scope = "iter",
order_relevance = order_relevance,
args = (
callable_identifier,
sentinel_identifier
),
context = context
)
def getBuiltinNext1Code(value):
return HelperCallIdentifier( "BUILTIN_NEXT1", value )
def getBuiltinNext2Code( context, order_relevance, iterator_identifier,
default_identifier ):
return getOrderRelevanceEnforcedArgsCode(
helper = "BUILTIN_NEXT2",
export_ref = 0,
ref_count = 1,
tmp_scope = "next",
order_relevance = order_relevance,
args = (
iterator_identifier,
default_identifier
),
context = context
)
def getBuiltinType3Code( context, order_relevance, name_identifier,
bases_identifier, dict_identifier ):
return getOrderRelevanceEnforcedArgsCode(
helper = "BUILTIN_TYPE3",
export_ref = 0,
ref_count = 1,
tmp_scope = "type3",
order_relevance = order_relevance,
prefix_args = (
getConstantCode(
constant = context.getModuleName(),
context = context
),
),
args = (
name_identifier,
bases_identifier,
dict_identifier
),
context = context
)
def getBuiltinTupleCode(identifier):
return HelperCallIdentifier( "TO_TUPLE", identifier )
def getBuiltinListCode(identifier):
return HelperCallIdentifier( "TO_LIST", identifier )
def getBuiltinSetCode(identifier):
return HelperCallIdentifier( "TO_SET", identifier )
def getBuiltinDictCode(seq_identifier, dict_identifier):
if dict_identifier.isConstantIdentifier() and dict_identifier.getConstant() == {}:
dict_identifier = None
assert seq_identifier is not None or dict_identifier is not None
if seq_identifier is not None:
return Identifier(
"TO_DICT( %s, %s )" % (
seq_identifier.getCodeTemporaryRef(),
defaultToNullIdentifier(dict_identifier).getCodeTemporaryRef()
),
1
)
else:
return dict_identifier
def getBuiltinFloatCode(identifier):
return HelperCallIdentifier( "TO_FLOAT", identifier )
def getBuiltinLong1Code(context, identifier):
if identifier is None:
identifier = getConstantHandle( context = context, constant = "0" )
return HelperCallIdentifier( "TO_LONG", identifier )
def getBuiltinLong2Code(context, order_relevance, identifier, base):
if identifier is None:
identifier = getConstantHandle( context = context, constant = "0" )
return getOrderRelevanceEnforcedArgsCode(
helper = "TO_LONG2",
export_ref = 0,
ref_count = 1,
tmp_scope = "long",
order_relevance = order_relevance,
args = (
identifier,
base
),
context = context
)
def getBuiltinInt1Code(context, identifier):
if identifier is None:
identifier = getConstantHandle( context = context, constant = "0" )
return HelperCallIdentifier( "TO_INT", identifier )
def getBuiltinInt2Code(context, order_relevance, identifier, base):
if identifier is None:
identifier = getConstantHandle( context = context, constant = "0" )
return getOrderRelevanceEnforcedArgsCode(
helper = "TO_INT2",
export_ref = 0,
ref_count = 1,
tmp_scope = "int",
order_relevance = order_relevance,
args = (
identifier,
base
),
context = context
)
def getBuiltinStrCode(identifier):
return HelperCallIdentifier( "TO_STR", identifier )
def getBuiltinUnicode1Code(identifier):
return HelperCallIdentifier( "TO_UNICODE", identifier )
def getBuiltinUnicode3Code( context, order_relevance, identifier, encoding,
errors ):
return getOrderRelevanceEnforcedArgsCode(
helper = "TO_UNICODE3",
export_ref = 0,
ref_count = 1,
tmp_scope = "unicode",
order_relevance = order_relevance,
args = (
identifier,
defaultToNullIdentifier(encoding),
defaultToNullIdentifier(errors),
),
context = context
)
def getBoolFromCode(code):
assert type( code ) is str
return Identifier(
"BOOL_FROM( %s )" % code,
0
)
def getBuiltinBoolCode(identifier):
return Identifier(
"TO_BOOL( %s )" % identifier.getCodeTemporaryRef(),
0
)
def getFrameMakingIdentifier(context):
return context.getFrameHandle()
def getTracebackMakingIdentifier(context):
return Identifier(
"MAKE_TRACEBACK( %s )" % (
getFrameMakingIdentifier( context = context ).getCodeExportRef(),
),
1
)
def getExportScopeCode(cross_module):
if cross_module:
return "NUITKA_CROSS_MODULE"
else:
return "NUITKA_LOCAL_MODULE"
def getSelectMetaclassCode(metaclass_identifier, bases_identifier, context):
if Utils.python_version < 300:
assert metaclass_identifier is None
args = [
bases_identifier.getCodeTemporaryRef(),
getMetaclassVariableCode(context = context)
]
else:
args = [
metaclass_identifier.getCodeTemporaryRef(),
bases_identifier.getCodeTemporaryRef()
]
return CallIdentifier( "SELECT_METACLASS", args )
def getStatementTrace(source_desc, statement_repr):
return 'puts( "Execute: " %s );' % (
CppStrings.encodeString( source_desc + " " + statement_repr ),
)
def getConstantsDeclarationCode(context):
constant_declarations, _constant_locals = getConstantsDeclCode(
context = context,
for_header = True
)
constant_declarations += getCodeObjectsDeclCode(
for_header = True
)
header_body = CodeTemplates.template_constants_declaration % {
"constant_declarations" : "\n".join(constant_declarations)
}
return CodeTemplates.template_header_guard % {
"header_guard_name" : "__NUITKA_DECLARATIONS_H__",
"header_body" : header_body
}
def getConstantsDefinitionCode(context):
constant_inits = getConstantsInitCode(
context = context
)
constant_inits += getCodeObjectsInitCode(
context = context
)
constant_declarations, constant_locals = getConstantsDeclCode(
context = context,
for_header = False
)
constant_declarations += getCodeObjectsDeclCode(
for_header = False
)
return CodeTemplates.template_constants_reading % {
"constant_declarations" : "\n".join(constant_declarations),
"constant_inits" : indented(constant_inits),
"constant_locals" : indented(constant_locals)
}
def getCurrentExceptionTypeCode():
return Identifier(
"_exception.getType()",
0
)
def getCurrentExceptionValueCode():
return Identifier(
"_exception.getValue()",
0
)
def getCurrentExceptionTracebackCode():
return Identifier(
"(PyObject *)_exception.getTraceback()",
0
)
def getListOperationAppendCode(list_identifier, value_identifier):
return Identifier(
"APPEND_TO_LIST( %s, %s ), Py_None" % (
list_identifier.getCodeTemporaryRef(),
value_identifier.getCodeTemporaryRef()
),
0
)
def getSetOperationAddCode(set_identifier, value_identifier):
return Identifier(
"ADD_TO_SET( %s, %s ), Py_None" % (
set_identifier.getCodeTemporaryRef(),
value_identifier.getCodeTemporaryRef()
),
0
)
def getDictOperationSetCode( dict_identifier, key_identifier,
value_identifier ):
return Identifier(
"DICT_SET_ITEM( %s, %s, %s ), Py_None" % (
dict_identifier.getCodeTemporaryRef(),
key_identifier.getCodeTemporaryRef(),
value_identifier.getCodeTemporaryRef()
),
0
)
def getDictOperationGetCode(dict_identifier, key_identifier):
return Identifier(
"DICT_GET_ITEM( %s, %s )" % (
dict_identifier.getCodeTemporaryRef(),
key_identifier.getCodeTemporaryRef(),
),
1
)
def getDictOperationRemoveCode(dict_identifier, key_identifier):
return "DICT_REMOVE_ITEM( %s, %s );" % (
dict_identifier.getCodeTemporaryRef(),
key_identifier.getCodeTemporaryRef()
)
def getFrameLocalsUpdateCode(locals_identifier):
if locals_identifier.isConstantIdentifier() and \
locals_identifier.getConstant() == {}:
return ""
else:
return CodeTemplates.template_frame_locals_update % {
"locals_identifier" : locals_identifier.getCodeExportRef()
}
def getFrameGuardHeavyCode( frame_identifier, code_identifier, codes,
locals_code, needs_preserve, context ):
if context.isForDirectCall():
return_code = CodeTemplates.frame_guard_cpp_return
else:
return_code = CodeTemplates.frame_guard_python_return
tb_making = getTracebackMakingIdentifier( context )
if needs_preserve:
frame_class_name = "FrameGuardWithExceptionPreservation"
else:
frame_class_name = "FrameGuard"
return CodeTemplates.frame_guard_full_template % {
"frame_identifier" : frame_identifier,
"frame_class_name" : frame_class_name,
"code_identifier" : code_identifier.getCodeTemporaryRef(),
"codes" : indented( codes ),
"module_identifier" : getModuleAccessCode( context = context ),
"frame_locals" : indented( locals_code, vert_block = True ),
"tb_making" : tb_making.getCodeExportRef(),
"return_code" : return_code
}
def getFrameGuardOnceCode( frame_identifier, code_identifier, locals_identifier,
codes, needs_preserve, context ):
tb_making = getTracebackMakingIdentifier( context )
if needs_preserve:
frame_class_name = "FrameGuardWithExceptionPreservation"
else:
frame_class_name = "FrameGuard"
return CodeTemplates.frame_guard_once_template % {
"frame_identifier" : frame_identifier,
"frame_class_name" : frame_class_name,
"code_identifier" : code_identifier.getCodeTemporaryRef(),
"codes" : indented( codes ),
"module_identifier" : getModuleAccessCode( context = context ),
"frame_locals" : locals_identifier.getCodeExportRef(),
"tb_making" : tb_making.getCodeExportRef(),
"return_code" : indented( context.getReturnErrorCode() )
}
def getFrameGuardLightCode(frame_identifier, code_identifier, codes, context):
tb_making = getTracebackMakingIdentifier( context )
return CodeTemplates.frame_guard_genfunc_template % {
"frame_identifier" : frame_identifier,
"code_identifier" : code_identifier.getCodeTemporaryRef(),
"codes" : indented( codes ),
"module_identifier" : getModuleAccessCode( context = context ),
"tb_making" : tb_making.getCodeExportRef(),
}
def getFrameGuardVeryLightCode(codes):
return CodeTemplates.frame_guard_listcontr_template % {
"codes" : indented( codes, 0 ),
}
Nuitka-0.5.0.1/nuitka/codegen/ConstantCodes.py 0000644 0001750 0001750 00000045427 12265264105 021377 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Low level constant code generation.
"""
from .Pickling import getStreamedConstant
from .TupleCodes import addMakeTupleUse
from .ListCodes import addMakeListUse
from .DictCodes import addMakeDictUse
from .SetCodes import addMakeSetUse
from .BlobCodes import StreamData
from .Identifiers import (
EmptyDictIdentifier,
Identifier
)
# pylint: disable=W0622
from ..__past__ import unicode, long, iterItems
# pylint: enable=W0622
from ..Constants import HashableConstant, constant_builtin_types, isMutable
import re, struct
stream_data = StreamData()
def getConstantHandle(context, constant):
return context.getConstantHandle(
constant = constant
)
def getConstantCode(context, constant):
constant_identifier = context.getConstantHandle(
constant = constant
)
return constant_identifier.getCode()
# TODO: The determination of this should already happen in Building or in a
# helper not during code generation.
_match_attribute_names = re.compile( r"[a-zA-Z_][a-zA-Z0-9_]*$" )
def getConstantCodeName(context, constant):
return context.getConstantHandle( constant, real_use = False ).getCode()
def _isAttributeName(value):
return _match_attribute_names.match( value )
# Indicator to standalone mode code, if we need pickling module early on.
_needs_pickle = False
def needsPickleInit():
return _needs_pickle
def _getUnstreamCode(constant_value, constant_identifier):
saved = getStreamedConstant(
constant_value = constant_value
)
assert type( saved ) is bytes
# We need to remember having to use pickle, pylint: disable=W0603
global _needs_pickle
_needs_pickle = True
return "%s = UNSTREAM_CONSTANT( %s );" % (
constant_identifier,
stream_data.getStreamDataCode( saved )
)
def _packFloat(value):
return struct.pack( "= 0 and constant_value <= max_unsigned_long:
emit (
"%s = PyLong_FromUnsignedLong( %sul );" % (
constant_identifier,
constant_value
)
)
return
elif constant_value < 0 and constant_value >= min_signed_long:
emit (
"%s = PyLong_FromLong( %sl );" % (
constant_identifier,
constant_value
)
)
return
elif constant_value == min_signed_long-1:
emit(
"%s = PyLong_FromLong( %sl ); %s = PyNumber_InPlaceSubtract( %s, const_int_pos_1 );" % (
constant_identifier,
min_signed_long,
constant_identifier,
constant_identifier
)
)
return
elif constant_type is int:
if constant_value >= min_signed_long:
emit(
"%s = PyInt_FromLong( %sl );" % (
constant_identifier,
constant_value
)
)
return
else:
assert constant_value == min_signed_long-1
emit(
"%s = PyInt_FromLong( %sl ); %s = PyNumber_InPlaceSubtract( %s, const_int_pos_1 );" % (
constant_identifier,
min_signed_long,
constant_identifier,
constant_identifier
)
)
return
if constant_type is unicode:
try:
encoded = constant_value.encode( "utf-8" )
if str is not unicode:
emit(
"%s = UNSTREAM_UNICODE( %s );" % (
constant_identifier,
stream_data.getStreamDataCode(encoded)
)
)
else:
emit(
"%s = UNSTREAM_STRING( %s, %d );" % (
constant_identifier,
stream_data.getStreamDataCode(encoded),
1 if _isAttributeName(constant_value) else 0
)
)
return
except UnicodeEncodeError:
# So fall back to below code, which will unstream it then.
pass
elif constant_type is str:
# Python3: Strings that can be encoded as UTF-8 are done more or less
# directly. When they cannot be expressed as UTF-8, that is rare not we
# can indeed use pickling.
assert str is not unicode
if len(constant_value) == 1:
emit(
"%s = UNSTREAM_CHAR( %d, %d );" % (
constant_identifier,
ord(constant_value[0]),
1 if _isAttributeName(constant_value) else 0
)
)
else:
emit(
"%s = UNSTREAM_STRING( %s, %d );" % (
constant_identifier,
stream_data.getStreamDataCode(constant_value),
1 if _isAttributeName(constant_value) else 0
)
)
return
elif constant_type is bytes:
assert str is unicode
emit(
"%s = UNSTREAM_BYTES( %s );" % (
constant_identifier,
stream_data.getStreamDataCode( constant_value )
)
)
return
if constant_type is float:
emit(
"%s = UNSTREAM_FLOAT( %s );" % (
constant_identifier,
stream_data.getStreamDataCode(
value = _packFloat( constant_value ),
fixed_size = True
)
)
)
return
if constant_value is None:
return
if constant_value is False:
return
if constant_value is True:
return
if constant_value is Ellipsis:
return
if constant_type is dict:
if constant_value == {}:
emit( "%s = PyDict_New();" % constant_identifier )
else:
length = len( constant_value )
addMakeDictUse( length )
for key, value in iterItems( constant_value ):
_addConstantInitCode(
emit = emit,
constant_type = type( key ),
constant_value = key,
constant_identifier = getConstantCodeName( context, key ),
context = context
)
_addConstantInitCode(
emit = emit,
constant_type = type( value ),
constant_value = value,
constant_identifier = getConstantCodeName( context, value ),
context = context
)
emit(
"%s = MAKE_DICT%d( %s );" % (
constant_identifier,
length,
", ".join(
"%s, %s" % (
getConstantCodeName( context, value ),
getConstantCodeName( context, key )
)
for key, value
in
iterItems( constant_value )
)
)
)
return
if constant_type is tuple:
if constant_value == ():
emit( "%s = PyTuple_New( 0 );" % constant_identifier )
else:
length = len( constant_value )
addMakeTupleUse( length )
# Make elements earlier than tuple itself.
for element in constant_value:
_addConstantInitCode(
emit = emit,
constant_type = type( element ),
constant_value = element,
constant_identifier = getConstantCodeName(
context = context,
constant = element
),
context = context
)
emit(
"%s = MAKE_TUPLE%d( %s );" % (
constant_identifier,
length,
", ".join(
getConstantCodeName( context, element )
for element
in
constant_value
)
)
)
return
if constant_type is list:
if constant_value == []:
emit( "%s = PyList_New( 0 );" % constant_identifier )
else:
length = len( constant_value )
addMakeListUse( length )
# Make elements earlier than list itself.
for element in constant_value:
_addConstantInitCode(
emit = emit,
constant_type = type( element ),
constant_value = element,
constant_identifier = getConstantCodeName(
context = context,
constant = element
),
context = context
)
emit(
"%s = MAKE_LIST%d( %s );" % (
constant_identifier,
length,
", ".join(
getConstantCodeName(
context = context,
constant = element
)
for element
in
constant_value
)
)
)
return
if constant_type is set:
if constant_value == set():
emit( "%s = PySet_New( NULL );" % constant_identifier )
else:
length = len( constant_value )
addMakeSetUse( length )
# Make elements earlier than list itself.
for element in constant_value:
_addConstantInitCode(
emit = emit,
constant_type = type( element ),
constant_value = element,
constant_identifier = getConstantCodeName(
context = context,
constant = element
),
context = context
)
emit(
"%s = MAKE_SET%d( %s );" % (
constant_identifier,
length,
", ".join(
getConstantCodeName(
context = context,
constant = element
)
for element
in
constant_value
)
)
)
return
if constant_type in (frozenset, complex, unicode, long, range):
emit( _getUnstreamCode( constant_value, constant_identifier ) )
return
if constant_value in constant_builtin_types:
return
assert False, ( type(constant_value), constant_value, constant_identifier )
def _lengthKey(value):
return (
len(value[1]),
value[1]
)
the_contained_constants = {}
def getConstantsInitCode(context):
# There are many cases for constants to be created in the most efficient
# way, pylint: disable=R0912
statements = []
all_constants = the_contained_constants
all_constants.update(context.getConstants())
def receiveStatement(statement):
assert statement is not None
if statement not in statements:
statements.append(statement)
for (constant_type, constant_value), constant_identifier in \
sorted(all_constants.items(), key = _lengthKey):
_addConstantInitCode(
emit = receiveStatement,
constant_type = constant_type,
constant_value = constant_value.getConstant(),
constant_identifier = constant_identifier,
context = context
)
return statements
def getConstantsDeclCode(context, for_header):
# There are many cases for constants of different types.
# pylint: disable=R0912
statements = []
statements2 = []
constants = context.getConstants()
contained_constants = {}
def considerForDeferral(constant_value):
if constant_value is None:
return
if constant_value is False:
return
if constant_value is True:
return
if constant_value is Ellipsis:
return
constant_type = type(constant_value)
if constant_type is type:
return
key = constant_type, HashableConstant(constant_value)
if key not in contained_constants:
constant_identifier = getConstantCodeName(context, constant_value)
contained_constants[key] = constant_identifier
if constant_type in (tuple, list, set, frozenset):
for element in constant_value:
considerForDeferral(element)
elif constant_type is dict:
for key, value in iterItems(constant_value):
considerForDeferral(key)
considerForDeferral(value)
for (constant_type, constant_value), constant_identifier in \
sorted(constants.items(), key = _lengthKey):
# Need not declare built-in types.
if constant_type is type:
continue
declaration = "PyObject *%s;" % constant_identifier
if for_header:
declaration = "extern " + declaration
else:
if constant_type in (tuple, dict, list, set, frozenset):
considerForDeferral( constant_value.getConstant() )
statements.append(declaration)
for key, value in sorted(contained_constants.items(), key = _lengthKey):
if key not in constants:
declaration = "PyObject *%s;" % value
statements2.append(declaration)
if not for_header:
# Using global here, as this is really a singleton, in the form of a
# module, pylint: disable=W0603
global the_contained_constants
the_contained_constants = contained_constants
return statements, statements2
def getConstantAccess(context, constant):
# Many cases, because for each type, we may copy or optimize by creating
# empty. pylint: disable=R0911,R0912
if type(constant) is dict:
if constant:
for key, value in iterItems(constant):
# key cannot be mutable.
assert not isMutable(key)
if isMutable(value):
needs_deep = True
break
else:
needs_deep = False
if needs_deep:
return Identifier(
"DEEP_COPY( %s )" % getConstantCode(
constant = constant,
context = context
),
1
)
else:
return Identifier(
"PyDict_Copy( %s )" % getConstantCode(
constant = constant,
context = context
),
1
)
else:
return EmptyDictIdentifier()
elif type(constant) is set:
if constant:
return Identifier(
"PySet_New( %s )" % getConstantCode(
constant = constant,
context = context
),
1
)
else:
return Identifier(
"PySet_New( NULL )",
1
)
elif type(constant) is list:
if constant:
for value in constant:
if isMutable(value):
needs_deep = True
break
else:
needs_deep = False
if needs_deep:
return Identifier(
"DEEP_COPY( %s )" % getConstantCode(
constant = constant,
context = context
),
1
)
else:
return Identifier(
"LIST_COPY( %s )" % getConstantCode(
constant = constant,
context = context
),
1
)
else:
return Identifier(
"PyList_New( 0 )",
1
)
elif type(constant) is tuple:
for value in constant:
if isMutable(value):
needs_deep = True
break
else:
needs_deep = False
if needs_deep:
return Identifier(
"DEEP_COPY( %s )" % getConstantCode(
constant = constant,
context = context
),
1
)
else:
return getConstantHandle(
context = context,
constant = constant
)
else:
return getConstantHandle(
context = context,
constant = constant
)
Nuitka-0.5.0.1/nuitka/codegen/CodeObjectCodes.py 0000644 0001750 0001750 00000012015 12265264105 021572 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Code generation for code objects.
Right now only the creation is done here. But more should be added later on.
"""
from nuitka.Utils import python_version
from .ConstantCodes import getConstantHandle, getConstantCode
from .Identifiers import Identifier
import hashlib
from nuitka.__past__ import iterItems
# Code objects needed made unique by a key.
code_objects = {}
# False alarms about "hashlib.md5" due to its strange way of defining what is
# exported, pylint won't understand it. pylint: disable=E1101
if python_version < 300:
def _calcHash(key):
hash_value = hashlib.md5(
"%s%s%d%s%d%d%s%s%s%s" % key
)
return hash_value.hexdigest()
else:
def _calcHash(key):
hash_value = hashlib.md5(
("%s%s%d%s%d%d%s%s%s%s" % key).encode("utf-8")
)
return hash_value.hexdigest()
def _getCodeObjects():
return sorted(iterItems(code_objects))
# Sad but true, code objects have these many details that actually are fed from
# all different source, pylint: disable=R0913
def getCodeObjectHandle( context, filename, code_name, line_number, var_names,
arg_count, kw_only_count, is_generator, is_optimized,
has_starlist, has_stardict ):
var_names = tuple(var_names)
assert type(has_starlist) is bool
assert type(has_stardict) is bool
key = (
filename,
code_name,
line_number,
var_names,
arg_count,
kw_only_count,
is_generator,
is_optimized,
has_starlist,
has_stardict
)
if key not in code_objects:
code_objects[ key ] = Identifier(
"codeobj_%s" % _calcHash(key),
0
)
getConstantHandle(context, filename)
getConstantHandle(context, code_name)
getConstantHandle(context, var_names)
return code_objects[key]
def getCodeObjectsDeclCode(for_header):
# There are many cases for constants of different types.
# pylint: disable=R0912
statements = []
for _code_object_key, code_identifier in _getCodeObjects():
declaration = "PyCodeObject *%s;" % code_identifier.getCode()
if for_header:
declaration = "extern " + declaration
statements.append(declaration)
return statements
def getCodeObjectsInitCode(context):
statements = []
for code_object_key, code_identifier in _getCodeObjects():
co_flags = []
if code_object_key[2] != 0:
co_flags.append("CO_NEWLOCALS")
if code_object_key[6]:
co_flags.append("CO_GENERATOR")
if code_object_key[7]:
co_flags.append("CO_OPTIMIZED")
if code_object_key[8]:
co_flags.append("CO_VARARGS")
if code_object_key[9]:
co_flags.append("CO_VARKEYWORDS")
if python_version < 300:
code = "%s = MAKE_CODEOBJ( %s, %s, %d, %s, %d, %s );" % (
code_identifier.getCode(),
getConstantCode(
constant = code_object_key[0],
context = context
),
getConstantCode(
constant = code_object_key[1],
context = context
),
code_object_key[2],
getConstantCode(
constant = code_object_key[3],
context = context
),
code_object_key[4],
" | ".join(co_flags) or "0",
)
else:
code = "%s = MAKE_CODEOBJ( %s, %s, %d, %s, %d, %d, %s );" % (
code_identifier.getCode(),
getConstantCode(
constant = code_object_key[0],
context = context
),
getConstantCode(
constant = code_object_key[1],
context = context
),
code_object_key[2],
getConstantCode(
constant = code_object_key[3],
context = context
),
code_object_key[4],
code_object_key[5],
" | ".join(co_flags) or "0",
)
statements.append(code)
return statements
Nuitka-0.5.0.1/nuitka/codegen/CppStrings.py 0000644 0001750 0001750 00000004071 12265264105 020712 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" C++ string encoding
This contains the code to create string literals for C++ to represent the given
values and little more.
"""
from nuitka.__past__ import unicode # pylint: disable=W0622
def _encodeString(value):
""" Encode a string, so that it gives a C++ string literal.
This doesn't handle limits.
"""
assert type( value ) is bytes, type( value )
result = ""
octal = False
for c in value:
if str is not unicode:
cv = ord( c )
else:
cv = c
if c in b'\\\t\r\n"?':
result += r'\%o' % cv
octal = True
elif cv >= 32 and cv <= 127:
if octal and c in b'0123456789':
result += '" "'
result += chr( cv )
octal = False
else:
result += r'\%o' % cv
octal = True
result = result.replace( '" "\\', "\\" )
return '"%s"' % result
def encodeString(value):
""" Encode a string, so that it gives a C++ string literal.
"""
# Not all compilers don't allow arbitrary large C++ strings.
result = _encodeString( value[:16000 ] )
value = value[16000:]
while len( value ) > 0:
result += " "
result += _encodeString( value[:16000 ] )
value = value[16000:]
return result
Nuitka-0.5.0.1/nuitka/SyntaxErrors.py 0000644 0001750 0001750 00000006546 12265264105 017706 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Handling of syntax errors.
Format SyntaxError/IndentationError exception for output, as well as
raise it for the given source code reference.
"""
def formatOutput(e):
if len( e.args ) > 1:
reason, ( filename, lineno, colno, message ) = e.args
if message is None and colno is not None:
colno = None
else:
reason, = e.args
filename = None
lineno = None
colno = None
message = None
if colno is not None:
colno = colno - len( message ) + len( message.lstrip() )
return """\
File "%s", line %d
%s
%s^
%s: %s""" % (
filename,
lineno,
message.strip(),
" " * (colno-1) if colno is not None else "",
e.__class__.__name__,
reason
)
elif message is not None:
return """\
File "%s", line %d
%s
%s: %s""" % (
filename,
lineno,
message.strip(),
e.__class__.__name__,
reason
)
elif filename is not None:
return """\
File "%s", line %s
%s: %s""" % (
filename,
lineno,
e.__class__.__name__,
reason
)
else:
return """\
%s: %s""" % (
e.__class__.__name__,
reason
)
def raiseSyntaxError(reason, source_ref, col_offset = None, display_file = True,
display_line = True, source_line = None):
# TODO: This could could "linecache" module maybe.
def readSource():
source = open(source_ref.getFilename(), 'rU').readlines()
return source[source_ref.getLineNumber() - 1]
if display_file and display_line:
if source_line is None:
source_line = readSource()
raise SyntaxError(
reason,
(
source_ref.getFilename(),
source_ref.getLineNumber(),
col_offset,
source_line
)
)
else:
if source_ref is not None:
if source_line is None and display_line:
source_line = readSource()
raise SyntaxError(
reason,
(
source_ref.getFilename(),
source_ref.getLineNumber(),
None,
source_line
)
)
else:
raise SyntaxError(
reason,
(
None,
None,
None,
None
)
)
Nuitka-0.5.0.1/nuitka/Options.py 0000644 0001750 0001750 00000041037 12265270763 016657 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Options module """
version_string = """\
Nuitka V0.5.0.1
Copyright (C) 2013 Kay Hayen."""
from . import Utils
from optparse import OptionParser, OptionGroup, SUPPRESS_HELP
import sys, logging
# Indicator if we were called as "nuitka-run" in which case we assume some
# other defaults and work a bit different with parameters.
is_nuitka_run = Utils.basename(sys.argv[0]).lower().startswith("nuitka-run")
def getVersion():
return version_string.split()[1][1:]
if not is_nuitka_run:
usage = "usage: %prog [--module] [--execute] [options] main_module.py"
else:
usage = "usage: %prog [options] main_module.py"
parser = OptionParser(
usage = usage,
version = getVersion()
)
# This option is obsolete, and module should be used.
parser.add_option(
"--exe",
action = "store_true",
dest = "obsolete_executable",
default = False,
help = SUPPRESS_HELP
)
parser.add_option(
"--module",
action = "store_false",
dest = "executable",
default = True,
help = """\
Create an extension module executable instead of a program. Defaults to off."""
)
parser.add_option(
"--standalone", "--portable",
action = "store_true",
dest = "is_standalone",
default = False,
help = """\
Enable standalone mode in build. This allows you to transfer the created binary
to other machines without it relying on an existing Python installation. It
implies these options: "--recurse-all --recurse-stdlib". Defaults to off.""",
)
recurse_group = OptionGroup(
parser,
"Control the recursion into imported modules"
)
recurse_group.add_option(
"--recurse-stdlib",
action = "store_true",
dest = "recurse_stdlib",
default = False,
help = """\
Also descend into imported modules from standard library. Defaults to off."""
)
recurse_group.add_option(
"--recurse-none",
action = "store_true",
dest = "recurse_none",
default = False,
help = """\
When --recurse-none is used, do not descend into any imported modules at all,
overrides all other recursion options. Defaults to off."""
)
recurse_group.add_option(
"--recurse-all", "--recurse-on",
action = "store_true",
dest = "recurse_all",
default = False,
help = """\
When --recurse-all is used, attempt to descend into all imported modules.
Defaults to off."""
)
recurse_group.add_option(
"--recurse-to",
action = "append",
dest = "recurse_modules",
metavar = "MODULE/PACKAGE",
default = [],
help = """\
Recurse to that module, or if a package, to the whole package. Can be given
multiple times. Default empty."""
)
recurse_group.add_option(
"--recurse-not-to",
action = "append",
dest = "recurse_not_modules",
metavar = "MODULE/PACKAGE",
default = [],
help = """\
Do not recurse to that module, or if a package, to the whole package in any
case, overrides all other options. Can be given multiple times. Default
empty."""
)
recurse_group.add_option(
"--recurse-plugins", "--recurse-directory",
action = "append",
dest = "recurse_extra",
metavar = "MODULE/PACKAGE",
default = [],
help = """\
Recurse into that directory, no matter if it's used by the given main program
in a visible form. Overrides all other options. Can be given multiple times.
Default empty."""
)
parser.add_option_group( recurse_group )
execute_group = OptionGroup(
parser,
"Immediate execution after compilation"
)
execute_group.add_option(
"--execute",
action = "store_true",
dest = "immediate_execution",
default = is_nuitka_run,
help = """\
Execute immediately the created binary (or import the compiled module).
Defaults to %s.""" %
("on" if is_nuitka_run else "off")
)
execute_group.add_option(
"--execute-with-pythonpath", "--keep-pythonpath",
action = "store_true",
dest = "keep_pythonpath",
default = False,
help = """\
When immediately executing the created binary (--execute), don't reset
PYTHONPATH. When all modules are successfully included, you ought to not need
PYTHONPATH anymore."""
)
parser.add_option_group( execute_group )
dump_group = OptionGroup(
parser,
"Dump options for internal tree"
)
dump_group.add_option(
"--dump-xml",
action = "store_true",
dest = "dump_xml",
default = False,
help = """Dump the final result of optimization as XML, then exit."""
)
dump_group.add_option(
"--dump-tree",
action = "store_true",
dest = "dump_tree",
default = False,
help = """Dump the final result of optimization as text, then exit."""
)
dump_group.add_option(
"--display-tree",
action = "store_true",
dest = "display_tree",
default = False,
help = """\
Display the final result of optimization in a GUI, then exit."""
)
parser.add_option_group( dump_group )
parser.add_option(
"--python-version",
action = "store",
dest = "python_version",
choices = ( "2.6", "2.7", "3.2", "3.3" ),
default = None,
help = """Major version of Python to be used, one of '2.6', '2.7',
'3.2', or '3.3'."""
)
parser.add_option(
"--python-debug",
action = "store_true",
dest = "python_debug",
default = None,
help = """\
Use debug version or not. Default uses what you are using to run Nuitka, most
likely a non-debug version."""
)
parser.add_option(
"--python-flag",
action = "append",
dest = "python_flags",
default = [],
help = """\
Python flags to use. Default uses what you are using to run Nuitka, this
enforces a specific mode. These are options that also exist to standard
Python executable. Currently supported "-S" (alias nosite). Default empty."""
)
codegen_group = OptionGroup(
parser,
"Code generation choices"
)
codegen_group.add_option(
"--improved", "--enhanced",
action = "store_true",
dest = "improved",
default = False,
help = """\
Allow minor devitations from CPython behaviour, e.g. better tracebacks, which
are not really incompatible, but different.""",
)
codegen_group.add_option(
"--code-gen-no-statement-lines",
action ="store_false",
dest = "statement_lines",
default = True,
help = """\
Statements shall have their line numbers set. Disable this for less precise
exceptions and slightly faster code. Not recommended. Defaults to off."""
)
codegen_group.add_option(
"--no-optimization",
action = "store_true",
dest = "no_optimize",
default = False,
help = SUPPRESS_HELP
# """Disable all unnecessary optimizations on Python level. Defaults to off."""
)
parser.add_option_group( codegen_group )
outputdir_group = OptionGroup(
parser,
"Output directory choices"
)
outputdir_group.add_option(
"--output-dir",
action ="store",
dest = "output_dir",
metavar = "DIRECTORY",
default = "",
help = """\
Specify where intermediate and final output files should be put. DIRECTORY will
be populated with C++ files, object files, etc. Defaults to current directory.
"""
)
outputdir_group.add_option(
"--remove-output",
action = "store_true",
dest = "remove_build",
default = False,
help = """\
Removes the build directory after producing the module or exe file.
Defaults to off."""
)
parser.add_option_group( outputdir_group )
parser.add_option(
"--windows-disable-console",
action = "store_true",
dest = "win_disable_console",
default = False,
help = """\
When compiling for windows, disable the console window. Defaults to off."""
)
debug_group = OptionGroup(
parser,
"Debug features"
)
debug_group.add_option(
"--debug",
action = "store_true",
dest = "debug",
default = False,
help = """\
Executing all self checks possible to find errors in Nuitka, do not use for
production. Defaults to off."""
)
debug_group.add_option(
"--unstripped", "--no-strip", "--unstriped",
action = "store_true",
dest = "unstriped",
default = False,
help = """\
Keep debug info in the resulting object file for better gdb interaction.
Defaults to off."""
)
debug_group.add_option(
"--trace-execution",
action = "store_true",
dest = "trace_execution",
default = False,
help = """\
Traced execution output, output the line of code before executing it.
Defaults to off."""
)
debug_group.add_option(
"--c++-only",
action = "store_true",
dest = "cpp_only",
default = False,
help = """\
Compile the would-be regenerated source file. Allows compiling edited C++ files
with the C++ compiler for quick debugging changes to the generated source.
Defaults to off."""
)
debug_group.add_option(
"--experimental",
action = "store_true",
dest = "experimental",
default = False,
help = """\
Use features declared as 'experimental'. May have no effect if no experimental
features are present in the code. Defaults to off."""
)
parser.add_option_group( debug_group )
parser.add_option(
"--lto",
action = "store_true",
dest = "lto",
default = False,
help = """\
Use link time optimizations if available and usable (g++ 4.6 and higher).
Defaults to off."""
)
parser.add_option(
"--clang",
action = "store_true",
dest = "clang",
default = False,
help = """\
Enforce the use of clang (clang 3.0 or higher).
Defaults to off."""
)
parser.add_option(
"--mingw",
action = "store_true",
dest = "mingw",
default = False,
help = """\
Enforce the use of MinGW on Windows.
Defaults to off."""
)
tracing_group = OptionGroup(
parser,
"Tracing features"
)
tracing_group.add_option(
"--show-scons",
action = "store_true",
dest = "show_scons",
default = False,
help = """\
Operate Scons in non-quiet mode, showing the executed commands.
Defaults to off."""
)
tracing_group.add_option(
"--show-progress",
action = "store_true",
dest = "show_progress",
default = False,
help = """Provide progress information and statistics.
Defaults to off."""
)
tracing_group.add_option(
"--show-modules",
action = "store_true",
dest = "show_inclusion",
default = False,
help = """Provide a final summary on included modules.
Defaults to off."""
)
tracing_group.add_option(
"--verbose",
action = "store_true",
dest = "verbose",
default = False,
help = """\
Output details of actions take, esp. in optimizations. Can become a lot.
Defaults to off."""
)
parser.add_option_group( tracing_group )
parser.add_option(
"-j", "--jobs",
action ="store",
dest = "jobs",
metavar = "N",
default = Utils.getCoreCount(),
help = """\
Specify the allowed number of parallel C++ compiler jobs. Defaults to the
system CPU count.""",
)
parser.add_option(
"--warn-implicit-exceptions",
action = "store_true",
dest = "warn_implicit_exceptions",
default = False,
help = """\
Given warnings for implicit exceptions detected at compile time.""",
)
parser.add_option(
"--icon",
action = "store",
dest = "icon_path",
metavar = "ICON_PATH",
default = None,
help = """Add executable icon (windows only).""",
)
# First, isolate the first non-option arguments. TODO: Should repect "--"
# as a terminator to options.
if is_nuitka_run:
count = 0
for count, arg in enumerate( sys.argv ):
if count == 0:
continue
if arg[0] != "-":
break
if count > 0:
extra_args = sys.argv[count+1:]
sys.argv = sys.argv[0:count+1]
else:
extra_args = []
options, positional_args = parser.parse_args()
if not positional_args:
parser.print_help()
sys.exit( """
Error, need positional argument with python module or main program.""" )
if options.verbose:
logging.getLogger().setLevel( logging.DEBUG )
else:
logging.getLogger().setLevel( logging.INFO )
# Standalone mode implies an executable, not importing "site" module, which is
# only for this machine, recursing to all modules, and even including the
# standard library.
if options.is_standalone:
options.executable = True
options.recurse_all = True
options.recurse_stdlib = True
def shallTraceExecution():
return options.trace_execution
def shallExecuteImmediately():
return options.immediate_execution
def shallDumpBuiltTree():
return options.dump_tree
def shallDumpBuiltTreeXML():
return options.dump_xml
def shallDisplayBuiltTree():
return options.display_tree
def shallOnlyExecCppCall():
return options.cpp_only
def shallHaveStatementLines():
return options.statement_lines
def shallMakeModule():
return not options.executable
def shallFollowStandardLibrary():
return options.recurse_stdlib
def shallFollowNoImports():
return options.recurse_none
def shallFollowAllImports():
return options.recurse_all
def getShallFollowModules():
return sum( [ x.split( "," ) for x in options.recurse_modules ], [] )
for any_case_module in getShallFollowModules():
if any_case_module.startswith( "." ):
bad = True
else:
for char in "/\\:":
if char in any_case_module:
bad = True
break
else:
bad = False
if bad:
sys.exit( """
Error, '--recurse-to' takes only module names, not directory path '%s'.""" % \
any_case_module )
def getShallFollowInNoCase():
return sum( [ x.split( "," ) for x in options.recurse_not_modules ], [] )
for no_case_module in getShallFollowInNoCase():
if no_case_module.startswith( "." ):
bad = True
else:
for char in "/\\:":
if char in no_case_module:
bad = True
break
else:
bad = False
if bad:
sys.exit( """
Error, '--recurse-not-to' takes only module names, not directory path '%s'.""" % \
no_case_module )
def getShallFollowExtra():
return sum( [ x.split( "," ) for x in options.recurse_extra ], [] )
def shallWarnImplicitRaises():
return options.warn_implicit_exceptions
def isDebug():
return options.debug
def isPythonDebug():
return options.python_debug or sys.flags.debug
def isOptimize():
return not options.no_optimize
def isUnstriped():
return options.unstriped
def getOutputPath(path):
if options.output_dir:
return Utils.normpath( Utils.joinpath( options.output_dir, path ) )
else:
return path
def getOutputDir():
return options.output_dir if options.output_dir else "."
def getPositionalArgs():
return tuple( positional_args )
def getMainArgs():
return tuple( extra_args )
def shallOptimizeStringExec():
return False
def shallClearPythonPathEnvironment():
return not options.keep_pythonpath
def isShowScons():
return options.show_scons
def getJobLimit():
return int( options.jobs )
def isLto():
return options.lto
def isClang():
return options.clang
def isMingw():
return options.mingw
def shallDisableConsoleWindow():
return options.win_disable_console
def isFullCompat():
return not options.improved
def isShowProgress():
return options.show_progress
def isShowInclusion():
return options.show_inclusion
def isRemoveBuildDir():
return options.remove_build
def getIntendedPythonVersion():
return options.python_version
def isExperimental():
return hasattr( options, "experimental" ) and options.experimental
def isStandaloneMode():
return options.is_standalone
def getIconPath():
return options.icon_path
def getPythonFlags():
result = []
for part in options.python_flags:
if part in ( "-S", "nosite", "no_site" ):
result.append( "no_site" )
elif part in ( "-v", "trace_imports", "trace_import" ):
result.append( "trace_imports" )
else:
logging.warning( "Unsupported flag '%s'.", part )
return result
Nuitka-0.5.0.1/nuitka/Utils.py 0000644 0001750 0001750 00000010165 12265264105 016313 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Utility module.
Here the small things for file/dir names, Python version, CPU counting,
etc. that fit nowhere else and don't deserve their own names.
"""
import sys, os, subprocess
def _getPythonVersion():
big, major, minor = sys.version_info[0:3]
return big * 100 + major * 10 + minor
python_version = _getPythonVersion()
def getOS():
if os.name == "nt":
return "Windows"
elif os.name == "posix":
return os.uname()[0]
else:
assert False, os.name
def getArchitecture():
if getOS() == "Windows":
if "AMD64" in sys.version:
return "x86_64"
else:
return "x86"
else:
return os.uname()[4]
def relpath(path):
return os.path.relpath( path )
def abspath(path):
return os.path.abspath( path )
def joinpath(*parts):
return os.path.join( *parts )
def splitpath(path):
return tuple( element for element in os.path.split( path ) if element )
def basename(path):
return os.path.basename( path )
def dirname(path):
return os.path.dirname( path )
def normpath(path):
return os.path.normpath( path )
def normcase(path):
return os.path.normcase( path )
def getExtension(path):
return os.path.splitext( path )[1]
def isFile(path):
return os.path.isfile( path )
def isDir(path):
return os.path.isdir( path )
def listDir(path):
""" Give a sorted path, basename pairs of a directory."""
return sorted(
[
( joinpath( path, filename ), filename )
for filename in
os.listdir( path )
]
)
def deleteFile(path, must_exist):
if must_exist or isFile( path ):
os.unlink( path )
def makePath(path):
os.makedirs( path )
def getCoreCount():
cpu_count = 0
# Try to sum up the CPU cores, if the kernel shows them.
try:
# Try to get the number of logical processors
with open( "/proc/cpuinfo" ) as cpuinfo_file:
cpu_count = cpuinfo_file.read().count( "processor\t:" )
except IOError:
pass
if not cpu_count:
# false alarm, no re-import, just a function level import to avoid it
# unless it is absolutely necessary, pylint: disable=W0404
import multiprocessing
cpu_count = multiprocessing.cpu_count()
return cpu_count
def callExec(args):
""" Do exec in a portable way preserving exit code.
On Windows, unfortunately there is no real exec, so we have to spawn
a new process instead.
"""
# On Windows os.execl does not work properly
if getOS() != "Windows":
# The star arguments is the API of execl, pylint: disable=W0142
os.execl( *args )
else:
args = list( args )
del args[1]
sys.exit( subprocess.call( args ) )
def encodeNonAscii(var_name):
""" Encode variable name that is potentially not ASCII to ASCII only.
For Python3, unicode identifiers can be used, but these are not
possible in C++03, so we need to replace them.
"""
if python_version < 300:
return var_name
else:
var_name = var_name.encode( "ascii", "xmlcharrefreplace" )
var_name = var_name.decode( "ascii" )
# TODO: Is this truly safe of collisions, I think it is not. It might be
# necessary to use something that is not allowed otherwise.
return var_name.replace( "&#", "$$" ).replace( ";", "" )
Nuitka-0.5.0.1/nuitka/Builtins.py 0000644 0001750 0001750 00000012372 12265264105 017006 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Builtins module. Information about builtins of the currently running Python.
"""
from types import BuiltinFunctionType, FunctionType, GeneratorType
from nuitka import Utils
import sys
def _getBuiltinExceptionNames():
def isExceptionName(builtin_name):
if builtin_name.endswith( "Error" ) or \
builtin_name.endswith( "Exception" ):
return True
elif builtin_name in ( "StopIteration", "GeneratorExit", "SystemExit",
"NotImplemented", "KeyboardInterrupt" ):
return True
else:
return False
# Hide Python3 changes for builtin exception names
try:
import exceptions
names = [
str( name ) for name in dir( exceptions )
if isExceptionName( name )
]
values = {}
for key in names:
values[ key ] = getattr( exceptions, key )
for key in dir( sys.modules[ "__builtin__" ] ):
name = str( key )
if isExceptionName( name ):
names.append( key )
values[ name ] = getattr( sys.modules[ "__builtin__" ], key )
except ImportError:
exceptions = {}
for key, value in sys.modules[ "builtins" ].__dict__.items():
if isExceptionName( key ):
exceptions[ key ] = value
names = [
key for key, value in exceptions.items()
]
values = {}
for key, value in exceptions.items():
values[ key ] = value
return names, values
builtin_exception_names, builtin_exception_values = _getBuiltinExceptionNames()
# Just to make sure it's covering these cases correctly.
assert "ValueError" in builtin_exception_names
assert "StopIteration" in builtin_exception_names
assert "GeneratorExit" in builtin_exception_names
assert "AssertionError" in builtin_exception_names
assert "BaseException" in builtin_exception_names
assert "Exception" in builtin_exception_names
assert "NotImplemented" in builtin_exception_names
def _getBuiltinNames():
names = [
str( x )
for x in __builtins__.keys()
]
for builtin_exception_name in builtin_exception_names:
if builtin_exception_name in names:
names.remove( builtin_exception_name )
names.remove( "__doc__" )
names.remove( "__name__" )
names.remove( "__package__" )
warnings = []
for builtin_name in names:
if builtin_name.endswith( "Warning" ):
warnings.append( builtin_name )
for builtin_name in warnings:
names.remove( builtin_name )
return names, warnings
builtin_names, builtin_warnings = _getBuiltinNames()
assert "__import__" in builtin_names
assert "int" in builtin_names
assert "__doc__" not in builtin_names
assert "sys" not in builtin_names
builtin_all_names = builtin_names + builtin_exception_names + builtin_warnings
def _getAnonBuiltins():
anon_names = {
# Strangely not Python3 types module
"NoneType" : type( None ),
"ellipsis" : type( Ellipsis ), # see above
"NotImplementedType" : type( NotImplemented ),
"function" : FunctionType,
"builtin_function_or_method" : BuiltinFunctionType,
# Can't really have it any better way.
"compiled_function" : BuiltinFunctionType,
"generator" : GeneratorType,
"compiled_generator" : GeneratorType, # see above
}
anon_codes = {
"NoneType" : "Py_TYPE( Py_None )",
"ellipsis" : "&PyEllipsis_Type",
"NotImplementedType" : "Py_TYPE( Py_NotImplemented )",
"function" : "&PyFunction_Type",
"builtin_function_or_method" : "&PyCFunction_Type",
"compiled_function" : "&Nuitka_Function_Type",
"compiled_generator" : "&Nuitka_Generator_Type",
}
if Utils.python_version < 300:
class Temp:
def __init__(self):
pass
def method(self):
pass
anon_names[ "classobj" ] = type(Temp)
anon_codes[ "classobj" ] = "&PyClass_Type"
anon_names[ "instance" ] = type( Temp() )
anon_codes[ "instance" ] = "&PyInstance_Type"
anon_names[ "instancemethod" ] = type( Temp().method )
anon_codes[ "instancemethod" ] = "&PyMethod_Type"
return anon_names, anon_codes
builtin_anon_names, builtin_anon_codes = _getAnonBuiltins()
Nuitka-0.5.0.1/nuitka/MainControl.py 0000644 0001750 0001750 00000054053 12265270763 017453 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" This is the main actions of Nuitka.
This can do all the steps to translate one module to a target language using
the Python C/API, to compile it to either an executable or an extension module.
"""
from .tree import (
Recursion,
Building
)
from . import (
ModuleRegistry,
SyntaxErrors,
Importing,
Tracing,
TreeXML,
Options,
Utils
)
from .build import SconsInterface
from .codegen import CodeGeneration, ConstantCodes
from .optimizations import Optimization
from .finalizations import Finalization
from nuitka.freezer.Standalone import (
detectEarlyImports,
detectLateImports,
detectUsedDLLs
)
from nuitka.freezer.BytecodeModuleFreezer import (
generateBytecodeFrozenCode,
getFrozenModuleCount,
addFrozenModule
)
import sys, os, subprocess, shutil
from logging import warning, info
def createNodeTree(filename):
""" Create a node tree.
Turn that source code into a node tree structure. If recursion into
imported modules is available, more trees will be available during
optimization, or immediately through recursed directory paths.
"""
# First, build the raw node tree from the source code.
main_module = Building.buildModuleTree(
filename = filename,
package = None,
is_top = True,
is_main = not Options.shallMakeModule()
)
ModuleRegistry.addRootModule(main_module)
# First remove old object files and old generated files, old binary or
# module, and standalone mode program directory if any, they can only do
# harm.
source_dir = getSourceDirectoryPath(main_module)
if not Options.shallOnlyExecCppCall():
cleanSourceDirectory(source_dir)
if Options.isStandaloneMode():
standalone_dir = getStandaloneDirectoryPath(main_module)
shutil.rmtree(standalone_dir, ignore_errors = True)
Utils.makePath(standalone_dir)
Utils.deleteFile(
path = getResultFullpath(main_module),
must_exist = False
)
# Second, do it for the directories given.
for plugin_filename in Options.getShallFollowExtra():
Recursion.checkPluginPath(
plugin_filename = plugin_filename,
module_package = None
)
# Then optimize the tree and potentially recursed modules.
Optimization.optimize()
return main_module
def dumpTree(tree):
Tracing.printLine( "Analysis -> Tree Result" )
Tracing.printSeparator()
Tracing.printSeparator()
Tracing.printSeparator()
tree.dump()
Tracing.printSeparator()
Tracing.printSeparator()
Tracing.printSeparator()
def dumpTreeXML(tree):
xml_root = tree.asXml()
TreeXML.dump(xml_root)
def displayTree(tree):
# Import only locally so the Qt4 dependency doesn't normally come into play
# when it's not strictly needed, pylint: disable=W0404
from .gui import TreeDisplay
TreeDisplay.displayTreeInspector(tree)
def getTreeFilenameWithSuffix(tree, suffix):
return tree.getOutputFilename() + suffix
def getSourceDirectoryPath(main_module):
assert main_module.isPythonModule()
return Options.getOutputPath(
path = Utils.basename(
getTreeFilenameWithSuffix( main_module, ".build" )
)
)
def getStandaloneDirectoryPath(main_module):
return Options.getOutputPath(
path = Utils.basename(
getTreeFilenameWithSuffix( main_module, ".dist" )
)
)
def getResultBasepath(main_module):
assert main_module.isPythonModule()
if Options.isStandaloneMode():
return Utils.joinpath(
getStandaloneDirectoryPath( main_module ),
Utils.basename(
getTreeFilenameWithSuffix( main_module, "" )
)
)
else:
return Options.getOutputPath(
path = Utils.basename(
getTreeFilenameWithSuffix( main_module, "" )
)
)
def getResultFullpath(main_module):
result = getResultBasepath(main_module)
if Options.shallMakeModule():
if Utils.getOS() == "Windows":
result += ".pyd"
else:
result += ".so"
else:
result += ".exe"
return result
def cleanSourceDirectory(source_dir):
if Utils.isDir(source_dir):
for path, _filename in Utils.listDir(source_dir):
if Utils.getExtension(path) in (".cpp", ".hpp", ".c", ".o", ".os",
".obj", ".bin", ".res", ".rc",
".manifest"):
Utils.deleteFile(path, True)
else:
Utils.makePath( source_dir )
static_source_dir = Utils.joinpath(
source_dir,
"static"
)
if Utils.isDir(static_source_dir):
for path, _filename in Utils.listDir(static_source_dir):
if Utils.getExtension(path) in (".o", ".os", ".obj"):
Utils.deleteFile(path, True)
def pickSourceFilenames(source_dir, modules):
collision_filenames = set()
seen_filenames = set()
for module in sorted( modules, key = lambda x : x.getFullName() ):
base_filename = Utils.joinpath( source_dir, module.getFullName() )
# Note: Could detect if the filesystem is cases sensitive in source_dir
# or not, but that's probably not worth the effort.
collision_filename = Utils.normcase( base_filename )
if collision_filename in seen_filenames:
collision_filenames.add( collision_filename )
seen_filenames.add( collision_filename )
collision_counts = {}
module_filenames = {}
for module in sorted(modules, key = lambda x : x.getFullName()):
if module.isPythonShlibModule():
continue
base_filename = Utils.joinpath(
source_dir,
"module." + module.getFullName()
if not module.isInternalModule() else
module.getFullName()
)
collision_filename = Utils.normcase( base_filename )
if collision_filename in collision_filenames:
collision_counts[ collision_filename ] = \
collision_counts.get( collision_filename, 0 ) + 1
hash_suffix = "@%d" % collision_counts[ collision_filename ]
else:
hash_suffix = ""
base_filename += hash_suffix
cpp_filename = base_filename + ".cpp"
hpp_filename = base_filename + ".hpp"
module_filenames[module] = (cpp_filename, hpp_filename)
return module_filenames
standalone_entry_points = []
def makeSourceDirectory(main_module):
# We deal with a lot of details here, but rather one by one, and split makes
# no sense, pylint: disable=R0914,R0912
assert main_module.isPythonModule()
# The global context used to generate code.
global_context = CodeGeneration.makeGlobalContext()
# Get the full list of modules imported, create code for all of them.
modules = ModuleRegistry.getDoneModules()
assert main_module in modules
# Sometimes we need to talk about all modules except main module.
other_modules = ModuleRegistry.getDoneUserModules()
# Lets check if the recurse-to modules are actually present, and warn the
# user if one was not found.
for any_case_module in Options.getShallFollowModules():
for module in other_modules:
if module.getFullName() == any_case_module:
break
else:
warning(
"Didn't recurse to '%s', apparently not used." % \
any_case_module
)
# Prepare code generation, i.e. execute finalization for it.
for module in sorted(modules, key = lambda x : x.getFullName()):
if module.isPythonModule():
Finalization.prepareCodeGeneration(module)
# Pick filenames.
source_dir = getSourceDirectoryPath(main_module)
module_filenames = pickSourceFilenames(
source_dir = source_dir,
modules = modules
)
module_hpps = []
for module in sorted(modules, key = lambda x : x.getFullName()):
if module.isPythonModule():
cpp_filename, hpp_filename = module_filenames[module]
source_code, header_code, module_context = \
CodeGeneration.generateModuleCode(
global_context = global_context,
module = module,
module_name = module.getFullName(),
other_modules = other_modules
if module is main_module else
()
)
# The main of an executable module gets a bit different code.
if module is main_module and not Options.shallMakeModule():
source_code = CodeGeneration.generateMainCode(
context = module_context,
codes = source_code
)
module_hpps.append( hpp_filename )
writeSourceCode(
filename = cpp_filename,
source_code = source_code
)
writeSourceCode(
filename = hpp_filename,
source_code = header_code
)
if Options.isShowInclusion():
info("Included compiled module '%s'.", module.getFullName())
elif module.isPythonShlibModule():
target_filename = Utils.joinpath(
getStandaloneDirectoryPath(main_module),
*module.getFullName().split(".")
)
if Utils.getOS() == "Windows":
target_filename += ".pyd"
else:
target_filename += ".so"
target_dir = Utils.dirname(target_filename)
if not Utils.isDir(target_dir):
Utils.makePath(target_dir)
shutil.copy(
module.getFilename(),
target_filename
)
standalone_entry_points.append(
(target_filename,module.getPackage())
)
else:
assert False, module
writeSourceCode(
filename = Utils.joinpath( source_dir, "__constants.hpp" ),
source_code = CodeGeneration.generateConstantsDeclarationCode(
context = global_context
)
)
writeSourceCode(
filename = Utils.joinpath( source_dir, "__constants.cpp" ),
source_code = CodeGeneration.generateConstantsDefinitionCode(
context = global_context
)
)
helper_decl_code, helper_impl_code = CodeGeneration.generateHelpersCode()
writeSourceCode(
filename = Utils.joinpath( source_dir, "__helpers.hpp" ),
source_code = helper_decl_code
)
writeSourceCode(
filename = Utils.joinpath( source_dir, "__helpers.cpp" ),
source_code = helper_impl_code
)
module_hpp_include = [
'#include "%s"\n' % Utils.basename( module_hpp )
for module_hpp in
module_hpps
]
writeSourceCode(
filename = Utils.joinpath( source_dir, "__modules.hpp" ),
source_code = "".join( module_hpp_include )
)
def runScons(main_module, quiet):
# Scons gets transported many details, that we express as variables, and
# have checks for them, leading to many branches, pylint: disable=R0912
python_version = "%d.%d" % ( sys.version_info[0], sys.version_info[1] )
if hasattr(sys, "abiflags"):
# The Python3 for some platforms has sys.abiflags pylint: disable=E1101
if Options.isPythonDebug() or \
hasattr(sys, "getobjects"):
if sys.abiflags.startswith("d"):
python_version += sys.abiflags
else:
python_version += "d" + sys.abiflags
else:
python_version += sys.abiflags
def asBoolStr(value):
return "true" if value else "false"
options = {
"name" : Utils.basename(
getTreeFilenameWithSuffix(main_module, "")
),
"result_name" : getResultBasepath(main_module),
"source_dir" : getSourceDirectoryPath(main_module),
"debug_mode" : asBoolStr(Options.isDebug()),
"python_debug" : asBoolStr(Options.isPythonDebug()),
"unstriped_mode" : asBoolStr(Options.isUnstriped()),
"module_mode" : asBoolStr(Options.shallMakeModule()),
"optimize_mode" : asBoolStr(Options.isOptimize()),
"full_compat" : asBoolStr(Options.isFullCompat()),
"experimental" : asBoolStr(Options.isExperimental()),
"python_version" : python_version,
"target_arch" : Utils.getArchitecture(),
"python_prefix" : sys.prefix,
"nuitka_src" : SconsInterface.getSconsDataPath(),
"module_count" : "%d" % (
len(ModuleRegistry.getDoneUserModules()) + 1
)
}
# Ask Scons to cache on Windows, except where the directory is thrown
# away. On non-Windows you can should use ccache instead.
if not Options.isRemoveBuildDir() and Utils.getOS() == "Windows":
options["cache_mode"] = "true"
if Options.isLto():
options["lto_mode"] = "true"
if Options.shallDisableConsoleWindow():
options["win_disable_console"] = "true"
if Options.isStandaloneMode():
options["standalone_mode"] = "true"
if getFrozenModuleCount():
options["frozen_modules"] = str(
getFrozenModuleCount()
)
if Options.isShowScons():
options["show_scons"] = "true"
if Options.isMingw():
options["mingw_mode"] = "true"
if Options.isClang():
options["clang_mode"] = "true"
if Options.getIconPath():
options["icon_path"] = Options.getIconPath()
return SconsInterface.runScons( options, quiet ), options
def writeSourceCode(filename, source_code):
# Prevent accidental overwriting. When this happens the collision detection
# or something else has failed.
assert not Utils.isFile(filename), filename
if Utils.python_version >= 300:
with open(filename, "wb") as output_file:
output_file.write(source_code.encode("latin1"))
else:
with open(filename, "w") as output_file:
output_file.write(source_code)
def writeBinaryData(filename, binary_data):
# Prevent accidental overwriting. When this happens the collision detection
# or something else has failed.
assert not Utils.isFile(filename), filename
assert type(binary_data) is bytes
with open(filename, "wb") as output_file:
output_file.write(binary_data)
def callExec(args, clean_path, add_path):
old_python_path = os.environ.get("PYTHONPATH", None)
if clean_path and old_python_path is not None:
os.environ["PYTHONPATH"] = ""
if add_path:
if "PYTHONPATH" in os.environ:
os.environ["PYTHONPATH"] += ":" + Options.getOutputDir()
else:
os.environ["PYTHONPATH"] = Options.getOutputDir()
# We better flush these, "os.execl" won't do it anymore.
sys.stdout.flush()
sys.stderr.flush()
args += Options.getMainArgs()
# That's the API of execl, pylint: disable=W0142
Utils.callExec(args)
def executeMain(binary_filename, tree, clean_path):
main_filename = tree.getFilename()
if Options.isStandaloneMode():
name = binary_filename
elif main_filename.endswith( ".py" ):
name = main_filename[:-3]
else:
name = main_filename
name = Utils.abspath(name)
args = (binary_filename, name)
callExec(
clean_path = clean_path,
add_path = False,
args = args
)
def executeModule(tree, clean_path):
python_command = "__import__('%s')" % tree.getName()
if Utils.getOS() == "Windows":
python_command = '"%s"' % python_command
args = (
sys.executable,
"python",
"-c",
python_command,
)
callExec(
clean_path = clean_path,
add_path = True,
args = args
)
def compileTree(main_module):
source_dir = getSourceDirectoryPath(main_module)
if not Options.shallOnlyExecCppCall():
# Now build the target language code for the whole tree.
makeSourceDirectory(
main_module = main_module
)
if Options.isStandaloneMode():
for late_import in detectLateImports():
addFrozenModule(late_import)
if getFrozenModuleCount():
frozen_code = generateBytecodeFrozenCode()
writeSourceCode(
filename = Utils.joinpath(
source_dir,
"__frozen.cpp"
),
source_code = frozen_code
)
writeBinaryData(
filename = Utils.joinpath(source_dir, "__constants.bin"),
binary_data = ConstantCodes.stream_data.getBytes()
)
else:
source_dir = getSourceDirectoryPath(main_module)
if not Utils.isFile( Utils.joinpath(source_dir, "__helpers.hpp")):
sys.exit("Error, no previous build directory exists.")
# Run the Scons to build things.
result, options = runScons(
main_module = main_module,
quiet = not Options.isShowScons()
)
return result, options
def main():
""" Main program flow of Nuitka
At this point, options will be parsed already, Nuitka will be executing
in the desired version of Python with desired flags, and we just get
to execute the task assigned.
We might be asked to only re-compile generated C++, dump only an XML
representation of the internal node tree after optimization, etc.
"""
# Main has to fullfil many options, leading to many branches
# pylint: disable=R0912
positional_args = Options.getPositionalArgs()
assert len( positional_args ) > 0
filename = Options.getPositionalArgs()[0]
# Inform the importing layer about the main script directory, so it can use
# it when attempting to follow imports.
Importing.setMainScriptDirectory(
main_dir = Utils.dirname(Utils.abspath(filename))
)
# Detect to be frozen modules if any, so we can consider to not recurse
# to them.
if Options.isStandaloneMode():
for early_import in detectEarlyImports():
addFrozenModule(early_import)
# Turn that source code into a node tree structure.
try:
main_module = createNodeTree(
filename = filename
)
except (SyntaxError, IndentationError) as e:
if Options.isFullCompat() and \
e.args[0].startswith("unknown encoding:"):
if Utils.python_version >= 333 or \
(
Utils.python_version >= 276 and \
Utils.python_version < 300
) or \
"2.7.5+" in sys.version or \
"3.3.2+" in sys.version: # Debian backports have "+" versions
complaint = "no-exist"
else:
complaint = "with BOM"
e.args = (
"encoding problem: %s" % complaint,
(e.args[1][0], 1, None, None)
)
sys.exit(
SyntaxErrors.formatOutput(e)
)
if Options.shallDumpBuiltTree():
dumpTree(main_module)
elif Options.shallDumpBuiltTreeXML():
dumpTreeXML(main_module)
elif Options.shallDisplayBuiltTree():
displayTree(main_module)
else:
result, options = compileTree(
main_module = main_module
)
# Exit if compilation failed.
if not result:
sys.exit(1)
# Remove the source directory (now build directory too) if asked to.
if Options.isRemoveBuildDir():
shutil.rmtree(
getSourceDirectoryPath(main_module)
)
if Options.isStandaloneMode():
binary_filename = options["result_name"] + ".exe"
standalone_entry_points.insert(
0,
(binary_filename, None)
)
if Utils.getOS() == "NetBSD":
warning("Standalone mode on NetBSD is not functional, due to $ORIGIN linkage not being supported.")
for early_dll in detectUsedDLLs(standalone_entry_points):
shutil.copy(
early_dll,
Utils.joinpath(
getStandaloneDirectoryPath(main_module),
Utils.basename(early_dll)
)
)
if Options.isShowInclusion():
info("Included used shared library '%s'.", early_dll)
# Modules should not be executable, but Scons creates them like it, fix
# it up here.
if Utils.getOS() != "Windows" and Options.shallMakeModule():
subprocess.call(
(
"chmod",
"-x",
getResultFullpath(main_module)
)
)
# Execute the module immediately if option was given.
if Options.shallExecuteImmediately():
if Options.shallMakeModule():
executeModule(
tree = main_module,
clean_path = Options.shallClearPythonPathEnvironment()
)
else:
executeMain(
binary_filename = getResultFullpath(main_module),
tree = main_module,
clean_path = Options.shallClearPythonPathEnvironment()
)
Nuitka-0.5.0.1/nuitka/build/ 0000755 0001750 0001750 00000000000 12265271051 015733 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/build/SingleExe.scons 0000644 0001750 0001750 00000074561 12265264105 020704 0 ustar hayen hayen 0000000 0000000 # -*- python -*-
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# The Nuitka scons file. If you have Scons or platform knowledge, please be
# especially invited and contribute improvements.
#
# This file is used to build an executable or shared library. Nuitka needs no
# build process for itself, although it can be compiled using the same method.
import os, subprocess, sys, re, platform
# The directory containing the C++ files generated by Nuitka to be built using
# scons. They are referred to as sources from here on. The ARGUMENTS is what
# Scons provides to us.
source_name = ARGUMENTS[ "name" ]
source_dir = ARGUMENTS[ "source_dir" ]
# The directory containing Nuitka provided C++ files to be built and where it
# should be used.
nuitka_src = ARGUMENTS[ "nuitka_src" ]
static_src = os.path.join( source_dir, "static" )
# We tell Scons that it is a variant to be built, so object files don't end up
# inside that directory which need not be writable.
VariantDir(static_src, os.path.join(nuitka_src, "static_src"), 0)
# The name of executable or loadable module that we produce.
result_basepath = ARGUMENTS["result_name"]
def getBoolOption( option_name, default ):
""" Small helper for boolean mode flags."""
value = ARGUMENTS.get( option_name, "True" if default else "False" )
return value.lower() in ( "yes", "true", "1" )
# The directory to use for cache directory.
cache_mode = getBoolOption( "cache_mode", False )
# Module mode: Create a Python extension module, create an executable otherwise.
module_mode = getBoolOption( "module_mode", False )
# Debug mode: Less optimizations, debug information in the resulting binary.
debug_mode = getBoolOption( "debug_mode", False )
# Python version to target.
python_version = ARGUMENTS[ "python_version" ]
# Python debug mode: refcount checking, assertions in CPython core.
python_debug = getBoolOption( "python_debug", False )
# Optimization mode: Optimize as much as currently possible.
optimize_mode = getBoolOption( "optimize_mode", True )
# Full compatibility, even where it's stupid, i.e. do not provide information,
# even if available, in order to assert maximum compatibility. Intended to
# control level of compatability to absurd.
full_compat_mode = getBoolOption( "full_compat", False )
# Experimental mode. Do things that are not yet safe to do.
experimental_mode = getBoolOption( "experimental", False )
# LTO mode: Use link time optimizations of g++ compiler if available and known
# good with the compiler in question. The 4.5 one didn't have good enough
# support, the compiled result would not run correctly.
lto_mode = getBoolOption( "lto_mode", False )
# Windows target mode: Cross compile for Windows or compiling on windows.
win_target = getBoolOption( "win_target", os.name == "nt" )
# Windows subsystem mode: Disable console for windows builds.
win_disable_console = getBoolOption( "win_disable_console", False )
# Unstriped mode: Do not remove debug symbols.
unstriped_mode = getBoolOption( "unstriped_mode", False )
# Clang compiler mode, default on MacOS X and FreeBSD, optional on Linux.
clang_mode = getBoolOption( "clang_mode", False )
if sys.platform == "darwin" or "freebsd" in sys.platform:
clang_mode = True
# MinGW compiler mode, optional and interesting to Windows only.
mingw_mode = getBoolOption( "mingw_mode", False )
# Shared library and compiled modules count, determines the need for the
# compiled module loader.
module_count = int( ARGUMENTS[ "module_count" ] )
# Frozen modules count, determines the need for the bytecode frozen
# modules loader.
frozen_modules = int( ARGUMENTS.get( "frozen_modules", 0 ) )
# Standalone mode
standalone_mode = getBoolOption( "standalone_mode", False )
# Show scons mode, output information about Scons operation
show_scons_mode = getBoolOption( "show_scons", False )
# Home of Python to be compiled against, used to find include files and libs
# to link against.
python_prefix = ARGUMENTS[ "python_prefix" ]
# Target arch, uses for compiler choice and quick linking of constants binary
# data.
target_arch = ARGUMENTS[ "target_arch" ]
# Icon for executable (windows-only)
icon_path = ARGUMENTS.get( "icon_path", None )
def createEnvironment( compiler_tools ):
args = {}
if os.name == "nt" and \
not mingw_mode and \
getExecutablePath( "cl", initial = True ) is not None:
args[ "MSVC_USE_SCRIPT" ] = False
return Environment(
# We want the outside environment to be passed through.
ENV = os.environ,
# Extra tools configuration for scons.
tools = compiler_tools,
# The shared libraries should not be named "lib...", because CPython
# requires the filename "module_name.so" to load it.
SHLIBPREFIX = "",
# Under windows, specify the target architecture is needed for Scons
# to pick up MSVC.
TARGET_ARCH = target_arch,
# If we are on Windows, and MinGW is not enforced, lets see if we can
# find "cl.exe", and if we do, disable automatic scan.
**args
)
def getExecutablePath( filename, initial ):
# Variable substitution from environment is needed, because this can contain
# "$CC" which should be looked up too.
if filename.startswith( "$" ):
filename = env[ filename[1:] ]
# Append ".exe" suffix on Windows if not already present.
if os.name == "nt" and not filename.lower().endswith( ".exe" ):
filename += ".exe"
# Search in "PATH" environment for a file named like it.
if os.name == "nt":
separator = ";"
else:
separator = ":"
# Either look at the initial "PATH" as given from the outside or look at the
# current environment.
if initial:
search_path = os.environ[ "PATH" ]
else:
search_path = env._dict[ "ENV" ][ "PATH" ]
# Now check in each path element, much like the shell will.
path_elements = search_path.split( separator )
for path_element in path_elements:
full = os.path.join( path_element, filename )
if os.path.exists( full ):
return full
else:
return None
if mingw_mode:
# Force usage of MinGW.
compiler_tools = ["mingw"]
else:
# Everything else should use default.
compiler_tools = ["default"]
# Create Scons environment, the main control tool. Don't include "mingw" on
# Windows immediately, we will default to MSVC if available.
env = createEnvironment(
compiler_tools = compiler_tools
)
# On Windows, in case MSVC was not found and not previously forced, retry with
# it and use that instead then as a fallback. Using both tools in one call
# seems to not work in terms of fallback when both exist.
if os.name == "nt" and \
compiler_tools == ["default"] and \
getExecutablePath(env[ "CXX" ], initial = False) is None:
env = createEnvironment(
compiler_tools = ["mingw"]
)
if clang_mode:
# If requested by the user, use the clang compiler, overriding what was
# said in environment.
env["CXX"] = "clang"
elif "CXX" in os.environ:
# If the environment variable CXX is set, use that.
env["CXX"] = os.environ["CXX"]
# To work around Windows not supporting command lines of greater than 10K by
# default:
def setupSpawn(env):
def spawn( sh, escape, cmd, args, env ):
# The "del" appears to not work, but is used with large amounts of
# files to link. So, lets do this outselves, plus it avoids a process
# spawn.
if cmd == "del":
os.unlink(args[1])
return 0
# For quoted arguments that end in a backslash, things don't work well
# this is a workaround for it.
def removeTrailingSlashQuote( arg ):
if arg.endswith( r'\"' ):
return arg[:-1] + '\\"'
else:
return arg
newargs = ' '.join(
removeTrailingSlashQuote( arg )
for arg in
args[1:]
)
cmdline = cmd + " " + newargs
startupinfo = subprocess.STARTUPINFO()
# CPython2.6 compatibility
try:
from subprocess import STARTF_USESHOWWINDOW
except ImportError:
from _subprocess import STARTF_USESHOWWINDOW
startupinfo.dwFlags |= STARTF_USESHOWWINDOW
proc = subprocess.Popen(
cmdline,
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE,
startupinfo = startupinfo,
shell = False,
env = env
)
data, err = proc.communicate()
rv = proc.wait()
if cmd == "cl":
data = data[ data.find( "\r\n" ) + 2 : ]
source_basenames = [
os.path.basename( source_file )
for source_file in
source_files
]
def check( line ):
return line in ( "", "Generating Code..." ) or \
line in source_basenames
data = "\r\n".join(
line
for line in
data.split( "\r\n" )
if not check( line )
)
elif cmd == "rc":
data = data[ data.find( "reserved.\r" ) + 13 : ]
data = "\n".join(
line
for line in
data.split( "\n")
if not "identifier truncated to"
)
elif cmd == "link" and module_mode:
data = "\r\n".join(
line
for line in
data.split( "\r\n" )
if " Creating library" not in line
)
if data.rstrip():
if not show_scons_mode:
print cmdline
print data,
return rv
env["SPAWN"] = spawn
def getGccVersion():
# Update CXXVERSION in env, after we changed it.
import SCons
pipe = SCons.Action._subproc(
env, [ env[ "CXX" ], '--version' ],
stdin = 'devnull',
stderr = 'devnull',
stdout = subprocess.PIPE
)
line = pipe.stdout.readline()
match = re.search( r'[0-9]+(\.[0-9]+){2}', line )
if match:
return match.group(0)
else:
return None
def getClangVersion():
import SCons
pipe = SCons.Action._subproc(
env, [ env[ "CXX" ], '--version' ],
stdin = 'devnull',
stderr = 'devnull',
stdout = subprocess.PIPE
)
line = pipe.stdout.readline()
match = re.search( r'LLVM ([0-9]+(\.[0-9]+){1})', line )
if match:
return match.group(1)
match = re.search( r'([0-9]+(\.[0-9]+){1})', line )
if match:
return match.group(1)
else:
return None
# Some versions of Scons have unset or None, make it empty string.
if env.get( "CXX", None ) is None:
env[ "CXX" ] = None
orig_cxx = env[ "CXX" ]
orig_cxx_version = env.get( "CXXVERSION", None )
# Remove "g++" as the compiler, in case it's not really existing or not high
# enough version.
if "g++" in ( env[ "CXX" ] or "" ):
gpp_version = getGccVersion()
if gpp_version is None or int( getGccVersion().replace( ".", "" ) ) < 440:
env[ "CXX" ] = None
# Detect compiler to be used from supported ones, if there is no usable g++
# link.
if os.name != "nt" and env["CXX"] is None:
for candidate in ( "g++-4.8", "g++-4.7", "g++-4.6", "g++-4.5", "g++-4.4",
"clang" ):
if os.path.exists( os.path.join( "/usr/bin", candidate ) ):
env[ "CXX" ] = candidate
env[ "CC" ] = candidate.replace( "++", "cc" )
if "clang" not in candidate:
# Update CXXVERSION in env, after we changed it.
env[ "CXXVERSION" ] = getGccVersion()
break
else:
sys.exit( """\
The g++ compiler '%s' (version %s) doesn't have the sufficient \
version (>= 4.4).""" % ( orig_cxx, orig_cxx_version ) )
if show_scons_mode:
print "Scons compiler: Using",
print getExecutablePath( env[ "CXX" ], initial = False )
if env["CXX"] is None or \
getExecutablePath( env[ "CXX" ], initial = False ) is None:
if os.name == "nt":
sys.exit( """\
Error, cannot locate suitable C++ compiler. You have the following options:
a) If a suitable Visual Studio version is installed, it not located,
automatically unless you install pywin32 for the Python installation
below "%s".
b) To make it find Visual Studio execute from Start Menu the 'Visual Studio
Command Prompt' or "vcvarsall.bat". That will add Visual Studio to the
PATH.
b) Install MinGW to 'C:\\MinGW' then it is automatically picked up.
""" % sys.exec_prefix )
else:
sys.exit( "Error, cannot locate suitable C++ compiler." )
gcc_mode = "g++" in env[ "CXX" ] or "clang" in env[ "CXX" ]
msvc_mode = win_target and not gcc_mode
if os.name == "nt" and msvc_mode:
setupSpawn( env )
if os.name == "nt" and gcc_mode and target_arch == "x86_64":
sys.exit( """\
Error, MinGW does not support 64 bit Python. Use MSVC or 32 bit Python instead.""" )
env[ "BUILD_DIR" ] = source_dir
# Store the file signatures database with the rest of the source files
sconsign_dir = os.path.abspath( os.path.join( source_dir, '.sconsign' ) )
if not os.path.exists( sconsign_dir ):
os.makedirs( sconsign_dir )
env.SConsignFile( sconsign_dir )
# Support for clang.
if "clang" in env[ "CXX" ]:
env.Append( CCFLAGS = [ "-w" ] )
env.Append( CPPDEFINES = [ "_XOPEN_SOURCE" ] )
env.Append( LINKFLAGS = [ "-lstdc++" ])
# Don't export anything by default, this should create smaller executables.
env.Append(
CCFLAGS = [
"-fvisibility=hidden",
"-fvisibility-inlines-hidden"
]
)
if debug_mode and not clang_mode:
env.Append( CCFLAGS = [ "-Wunused-but-set-variable" ] )
clang_version = getClangVersion()
clang_version = int( clang_version.replace( ".", "" ) + "0" )
# Support for g++.
if "g++" in env[ "CXX" ]:
# Don't export anything by default, this should create smaller executables.
if not win_target:
env.Append( CCFLAGS = [
"-fvisibility=hidden",
]
)
env.Append( CXXFLAGS = [
"-fvisibility-inlines-hidden"
]
)
env[ "CXXVERSION" ] = getGccVersion()
# Version dependent options.
gpp_version = int( env[ "CXXVERSION" ].replace( ".", "" ) )
# Enforce the minimum version, selecting a potentially existing g++-4.5
# binary if it's not high enough. This is esp. useful under Debian which
# allows all compiler to exist next to each other and where g++ might not be
# good enough, but g++-4.5 would be.
if gpp_version < 440:
sys.exit( """\
The g++ compiler %s (version %s) doesn't have the sufficient \
version (>= 4.4).""" % ( env[ "CXX" ], env[ "CXXVERSION" ] ) )
# Older g++ complains about aliasing with Py_True and Py_False, but we don't
# care.
if gpp_version < 450:
env.Append( CCFLAGS = [ "-fno-strict-aliasing" ] )
# For LTO mode, the version requirement is even higher, so try that too.
if lto_mode and gpp_version < 460:
sys.exit( """\
The g++ compiler %s (version %s) doesn't have the sufficient \
version for lto mode (>= 4.6).""" % ( env[ "CXX" ], env[ "CXXVERSION" ] ) )
# For g++ 4.6 there are some new interesting functions.
if gpp_version >= 460:
env.Append( CCFLAGS = [ "-fpartial-inlining" ] )
if debug_mode:
env.Append( CCFLAGS = [ "-Wunused-but-set-variable" ] )
# Use link time optimizations so that gcc can help with optimization across
# files, but unfortunately at this time it seriously slows down the compiled
# code. This may be because it needs -O3 option to be effective.
if gpp_version >= 460 and lto_mode:
env.Append( CCFLAGS = [ "-flto" ] )
env.Append( LINKFLAGS = [ "-flto=%d" % GetOption( "num_jobs" ) ] )
# env.Append( LINKFLAGS = [ "-Wsuggest-attribute=noreturn" ] )
# env.Append( LINKFLAGS = [ "-Wsuggest-attribute=pure" ] )
# env.Append( LINKFLAGS = [ "-Wsuggest-attribute=const" ] )
# env.Append( CCFLAGS = [ "-Wnoexcept" ] )
if debug_mode:
env.Append( LINKFLAGS = [ "-O2" ] )
if optimize_mode:
env.Append( LINKFLAGS = [
"-O3",
"-fpartial-inlining",
"-freorder-functions",
]
)
# Give a warning if LTO mode was specified, but won't be used.
if lto_mode and gpp_version < 460:
print >> sys.stderr, "Warning, LTO mode specified, but not available."
# The var-tracking does not scale, disable it. Should we really need it, we
# can enable it.
env.Append( CCFLAGS = [ "-fno-var-tracking" ] )
if msvc_mode:
env.Append( CCFLAGS = [ "/EHsc", "/J", "/Gd" ] )
env.Append( LINKFLAGS = [ "/INCREMENTAL:NO" ] )
if debug_mode:
if gcc_mode:
# Allow g++/clang to point out all kinds of inconsistency to us by
# raising an error.
env.Append(
CCFLAGS = [
"-Wall",
"-Werror"
]
)
# MinGW gives some non-sensical warnings it seems.
if os.name == "nt":
env.Append(
CCFLAGS = [
"-Wno-error=format",
"-Wno-format"
]
)
elif msvc_mode:
# Disable warnings that system headers already show.
env.Append(
CCFLAGS = [
"/W4", "/wd4505", "/wd4127", "/wd4100", "/wd4702", "/wd4189",
"/wd4211"
]
)
# As for sequence points, we are abusing it, so we have to allow it.
if "g++" in env["CXX"]:
env.Append(
CCFLAGS = ["-Wno-sequence-point"]
)
# Benefit from clang checking memory accesses.
if "clang" in env[ "CXX" ] and clang_version >= 330:
env.Append(
CCFLAGS = ["-fsanatize=address,bounds,return"]
)
if full_compat_mode:
env.Append(
CPPDEFINES = ["_NUITKA_FULL_COMPAT"]
)
if experimental_mode:
env.Append(
CPPDEFINES = ["_NUITKA_EXPERIMENTAL"]
)
if standalone_mode:
env.Append(
CPPDEFINES = ["_NUITKA_STANDALONE"]
)
if win_target:
# This is used for "PathRemoveFileSpec" used in standalone mode
# Windows code.
env.Append(
LIBS = ["Shlwapi"]
)
elif "linux" in sys.platform:
env.Append(
LIBS = ["dl"]
)
if win_target:
# For MinGW and cross compilation, we need to tell the subsystem
# to target as well as to automatically import everything used.
if gcc_mode:
env.Append(
LINKFLAGS = ["-Wl,--enable-auto-import"]
)
if win_disable_console:
env.Append(
LINKFLAGS = ["-Wl,--subsystem,windows"]
)
if win_disable_console:
env.Append(
CPPDEFINES = ["_NUITKA_WINMAIN_ENTRY_POINT"]
)
if python_debug:
env.Append(
CPPDEFINES = ["Py_DEBUG"]
)
def detectHostMultiarch():
import commands
for line in commands.getoutput( "dpkg-architecture" ).split("\n"):
if line.startswith("DEB_HOST_MULTIARCH="):
return line.split( "=", 1 )[1]
else:
return None
if gcc_mode and "linux" in sys.platform:
if python_version.startswith("3.3"):
host_multiarch = detectHostMultiarch()
if host_multiarch is not None:
env.Append(
CCFLAGS = [
"-I" + os.path.join(
"/usr/include/",
host_multiarch,
"python" + python_version
)
]
)
if os.name == "nt":
# On Windows, the installation layout is relatively fixed.
python_header_path = os.path.join(python_prefix, "include")
else:
# The python header path is a combination of python version and debug
# indication, make sure the headers are found by adding it to the C++
# include path.
python_header_path = os.path.join(
python_prefix,
"include",
"python" + python_version
)
if not os.path.exists( os.path.join( python_header_path, "Python.h" ) ):
# Not, for --python-debug other headers are used than for normal
# compilation.
sys.exit(
"""\
Error, no 'Python.h' %s headers can be found at '%s', dependency \
not satisfied!""" % (
"debug" if python_debug else "development",
python_header_path
)
)
env.Append( CPPPATH = [ python_header_path ] )
if win_target:
env.Append( LIBPATH = [ os.path.join( python_prefix, "libs" ) ] )
env.Append( LIBS = [ "python" + python_version.replace( ".", "" ) ] )
else:
# Debian and Ubuntu distinguish the system libraries like this.
if python_debug and \
python_prefix == "/usr" and \
not python_version.startswith( "3" ) and \
platform.dist()[0].lower() in ( "debian", "ubuntu" ):
env.Append( LIBS = [ "python" + python_version + "_d" ] )
else:
env.Append( LIBS = [ "python" + python_version ] )
if python_prefix != "/usr" and "linux" in sys.platform:
env.Append(
LIBS = ["dl", "pthread", "util", "m"]
)
if gcc_mode:
env.Append(
LINKFLAGS = ["-export-dynamic"]
)
# Add the python library path to the library path
python_lib_path = os.path.join( python_prefix, "lib" )
env.Append( LIBPATH = [ python_lib_path ] )
# For NetBSD the rpath is required, on FreeBSD it's warned as unused.
if "netbsd" in sys.platform:
env.Append(
LINKFLAGS = ["-rpath=" + python_lib_path]
)
# The static include files reside in Nuitka installation, which may be where
# the "nuitka.build" package lives.
nuitka_include = os.path.join(
nuitka_src,
"include"
)
# We have include files in the build directory and the static include directory
# that is located inside Nuitka installation.
env.Append(
CPPPATH = [
source_dir,
nuitka_include
]
)
if debug_mode or unstriped_mode:
# Use debug format that wine understands, so we get good tracebacks from it.
if gcc_mode:
env.Append(
CCFLAGS = ["-g"]
)
env.Append(
ASFLAGS = ["-g"]
)
if "g++" in env[ "CXX" ]:
env.Append(
CCFLAGS = ["-feliminate-unused-debug-types"]
)
elif msvc_mode:
env.Append(
CCFLAGS = ["/Zi"]
)
env.Append(
LINKFLAGS = ["/DEBUG"]
)
# When debugging, optimize less than when optimizing, when not remove
# assertions.
if debug_mode:
if not optimize_mode:
if gcc_mode or msvc_mode:
env.Append(
CCFLAGS = ["-O2"]
)
else:
env.Append(
CPPDEFINES = ["__NUITKA_NO_ASSERT__"]
)
if optimize_mode:
if gcc_mode:
env.Append(
CCFLAGS = ["-O3"]
)
# Check inlining of calls, except in debug mode, where it will all be
# wrong due to additional code.
# if not debug_mode:
# env.Append( CCFLAGS = [ "-Winline" ] )
elif msvc_mode:
env.Append(
CCFLAGS = ["/Ox"]
)
# Set load libpython from binary directory default
if standalone_mode and gcc_mode:
env.Append( LINKFLAGS = [ "-Wl,-R,'$$ORIGIN'" ] )
def getLinkerArch():
if "linux" in sys.platform:
if target_arch == "x86_64":
return "elf64-x86-64"
elif target_arch == "armv5tel":
return "elf32-littlearm"
else:
# TODO: Detect from objdump mayhaps.
return None
elif os.name == "nt":
# This is MinGW, MSVC is handled by other code.
return "pe-i386"
else:
# TODO: Missing for MacOS, Linux
return None
constants_bin_filename = os.path.join(source_dir,"__constants.bin")
if gcc_mode and getLinkerArch() is not None:
env.Append(
LINKFLAGS = [
"-Wl,-b", "-Wl,binary",
"-Wl,%s" % constants_bin_filename,
"-Wl,-b", "-Wl,%s" % getLinkerArch(),
"-Wl,-defsym",
"-Wl,%sconstant_bin=_binary_%s___constants_bin_start" % (
"_" if mingw_mode else "",
"".join(re.sub("[^a-zA-Z0-9_]","_",c) for c in source_dir)
)
]
)
constants_generated_filename = None
else:
constants_generated_filename = os.path.join(
source_dir,
"__constants_data.c"
)
with open(constants_generated_filename, "w") as output:
output.write("""const unsigned char constant_bin[] =\n{\n""")
for count, stream_byte in enumerate(
open(constants_bin_filename, "rb").read()
):
if count % 16 == 0:
if count > 0:
output.write("\n")
output.write(" ")
output.write(" 0x%02x," % ord(stream_byte))
output.write("\n};\n");
env.Append(
CPPDEFINES = ["_NUITKA_FROZEN=%d" % frozen_modules]
)
# Tell compiler to create a shared library or program.
if module_mode:
if "g++" in env["CXX"]:
env.Append(
CCFLAGS = ["-shared"]
)
elif "clang" in env["CXX"]:
pass
elif msvc_mode:
env.Append(
CCFLAGS = ["/LD"]
)
else:
assert False
else:
if msvc_mode:
env.Append(
CCFLAGS = ["/MT"]
)
if module_mode:
env.Append(
CPPDEFINES = ["_NUITKA_MODULE"]
)
else:
env.Append(
CPPDEFINES = ["_NUITKA_EXE"]
)
def discoverSourceFiles():
result = []
def getStatic( sub_path ):
return os.path.join( static_src, sub_path.replace( "/", os.path.sep ) )
result.append(getStatic("CompiledFunctionType.cpp"))
result.append(getStatic("CompiledGeneratorType.cpp"))
result.append(getStatic("CompiledMethodType.cpp"))
result.append(getStatic("CompiledFrameType.cpp"))
result.append(getStatic("CompiledCodeHelpers.cpp"))
result.append(getStatic("InspectPatcher.cpp"))
if win_target:
result.append(getStatic("win32_ucontext_src/fibers_win32.cpp"))
elif target_arch == "x86_64":
result.append(getStatic("x64_ucontext_src/fibers_x64.cpp"))
result.append(getStatic("x64_ucontext_src/swapfiber.S"))
elif "arm" in target_arch:
result.append(getStatic("arm_ucontext_src/fibers_arm.cpp"))
result.append(getStatic("arm_ucontext_src/ucontext.cpp"))
result.append(getStatic("arm_ucontext_src/getcontext.asm"))
else:
# Variant based on getcontext/setcontext/swapcontext/makecontext
result.append(getStatic("gen_ucontext_src/fibers_gen.cpp"))
for filename in os.listdir( source_dir ):
if filename.endswith( ".cpp" ):
result.append( os.path.join( source_dir, filename ) )
# If more than one module is included, we need the unfreezer.
if module_count > 1:
result.append( getStatic( "MetaPathBasedLoader.cpp" ) )
if constants_generated_filename is not None:
result.append(constants_generated_filename)
return result
source_targets = []
# Prepare the use of a custom specs file for windows targets. We change the used
# specs for linking to avoid the use of the wrong (for CPython) run time
# library.
if msvc_mode and not module_mode:
rc_content = []
if python_version < "3.0":
manifest_filename = os.path.join( source_dir, "resources.manifest" )
assert 0 == subprocess.call(
(
"mt",
"-inputresource:%s;#1" % os.path.join(
python_prefix,
"python.exe"
),
"-out:%s" % manifest_filename
),
stdout = subprocess.PIPE
)
rc_content.append( "1 RT_MANIFEST resources.manifest" )
if icon_path:
rc_content.append(
'2 ICON MOVEABLE PURE LOADONCALL DISCARDABLE "%s"' % (
icon_path.replace( "\\", "/" )
)
)
if rc_content:
rc_filename = os.path.join( source_dir, "resources.rc" )
rc_file = open( rc_filename, "w" )
rc_content.insert( 0, '#include "winuser.h"' )
rc_file.write( "\n".join( rc_content ) )
rc_file.close()
source_targets.append(
env.RES( rc_filename )
)
source_files = discoverSourceFiles()
if module_mode:
# For Python modules, the standard shared library extension is not what
# gets used.
if win_target:
module_suffix = ".pyd"
else:
module_suffix = ".so"
env["SHLIBSUFFIX"] = module_suffix
target = env.SharedLibrary(
result_basepath,
source_files + source_targets
)
else:
# Avoid dependency on MinGW libraries.
if win_target and gcc_mode:
env.Append( LINKFLAGS = [ "-static-libgcc", "-static-libstdc++" ] )
target = env.Program(
result_basepath + ".exe",
source_files + source_targets
)
# Avoid IO for compilation as much as possible, this should make the
# compilation more memory hungry, but also faster.
if gcc_mode:
env.Append( CCFLAGS = "-pipe" )
if "CCFLAGS" in os.environ:
env.Append(CCFLAGS = os.environ[ "CCFLAGS" ].split())
if "LDFLAGS" in os.environ:
env.Append(LINKFLAGS = os.environ[ "LDFLAGS" ].split())
# Remove the target file to avoid cases where it falsely doesn't get rebuild
# and then lingers from previous builds,
if os.path.exists(target[0].abspath):
os.unlink(target[0].abspath)
if show_scons_mode:
print "scons: Told to run compilation on %d CPUs." % GetOption( 'num_jobs' )
# Cached, when done, by the fastest possible algorithm and right inside the
# build directory. Makes no sense of course, if that is removed later on by
# Nuitka.
if cache_mode:
CacheDir(os.path.join(source_dir, "cache"))
Decider("MD5")
Default(target)
Nuitka-0.5.0.1/nuitka/build/include/ 0000755 0001750 0001750 00000000000 12265271051 017356 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/build/include/nuitka/ 0000755 0001750 0001750 00000000000 12265271051 020651 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/build/include/nuitka/builtins.hpp 0000644 0001750 0001750 00000007226 12265264105 023224 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_BUILTINS_H__
#define __NUITKA_BUILTINS_H__
#include "__helpers.hpp"
extern PyModuleObject *module_builtin;
extern PyDictObject *dict_builtin;
#include "nuitka/calling.hpp"
NUITKA_MAY_BE_UNUSED static PyObject *LOOKUP_BUILTIN( PyObject *name )
{
assertObject( (PyObject *)dict_builtin );
assertObject( name );
assert( Nuitka_String_Check( name ) );
PyObject *result = GET_STRING_DICT_VALUE(
dict_builtin,
(Nuitka_StringObject *)name
);
assertObject( result );
return result;
}
class PythonBuiltin
{
public:
explicit PythonBuiltin( PyObject **name )
{
this->name = (Nuitka_StringObject **)name;
this->value = NULL;
}
PyObject *asObject0()
{
if ( this->value == NULL )
{
this->value = LOOKUP_BUILTIN( (PyObject *)*this->name );
}
assertObject( this->value );
return this->value;
}
void update( PyObject *new_value )
{
assertObject( new_value );
this->value = new_value;
}
PyObject *call()
{
return CALL_FUNCTION_NO_ARGS(
this->asObject0()
);
}
PyObject *call1( PyObject *arg )
{
return CALL_FUNCTION_WITH_ARGS1(
this->asObject0(),
arg
);
}
PyObject *call2( PyObject *arg1, PyObject *arg2 )
{
return CALL_FUNCTION_WITH_ARGS2(
this->asObject0(),
arg1,
arg2
);
}
PyObject *call3( PyObject *arg1, PyObject *arg2, PyObject *arg3 )
{
return CALL_FUNCTION_WITH_ARGS3(
this->asObject0(),
arg1,
arg2,
arg3
);
}
PyObject *call_args( PyObject *args )
{
return CALL_FUNCTION_WITH_POSARGS(
this->asObject0(),
PyObjectTemporary( args ).asObject0()
);
}
PyObject *call_kw( PyObject *kw )
{
return CALL_FUNCTION_WITH_KEYARGS(
this->asObject0(),
kw
);
}
PyObject *call_args_kw( PyObject *args, PyObject *kw )
{
return CALL_FUNCTION(
this->asObject0(),
args,
kw
);
}
private:
PythonBuiltin( PythonBuiltin const & ) { assert( false ); }
Nuitka_StringObject **name;
PyObject *value;
};
extern void _initBuiltinModule();
#ifdef _NUITKA_EXE
// Original builtin values, currently only used for assertions.
extern PyObject *_python_original_builtin_value_type;
extern PyObject *_python_original_builtin_value_len;
extern PyObject *_python_original_builtin_value_range;
extern PyObject *_python_original_builtin_value_repr;
extern PyObject *_python_original_builtin_value_int;
extern PyObject *_python_original_builtin_value_iter;
extern PyObject *_python_original_builtin_value_long;
extern void _initBuiltinOriginalValues();
#endif
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/variables_temporary.hpp 0000644 0001750 0001750 00000015212 12265270513 025437 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_VARIABLES_TEMPORARY_H__
#define __NUITKA_VARIABLES_TEMPORARY_H__
// Wraps a "PyObject *" you received or acquired from another container to
// simplify refcount handling when you're not going to use the object beyond the
// local scope. It will hold a reference to the wrapped object as long as the
// PyObjectTemporary is alive, and will release the reference when the wrapper
// is destroyed: this eliminates the need for manual DECREF calls on Python
// objects before returning from a method call.
//
// In effect, wrapping an object inside a PyObjectTemporary is equivalent to a
// deferred Py_DECREF() call on the wrapped object.
class PyObjectTemporary
{
public:
explicit PyObjectTemporary( PyObject *object )
{
assertObject( object );
this->object = object;
}
~PyObjectTemporary()
{
assertObject( this->object );
Py_DECREF( this->object );
}
PyObject *asObject0() const
{
assertObject( this->object );
return this->object;
}
PyObject *asObject1() const
{
return INCREASE_REFCOUNT( this->object );
}
void assign0( PyObject *object )
{
assertObject( object );
assertObject( this->object );
PyObject *old = this->object;
this->object = INCREASE_REFCOUNT( object );
Py_DECREF( old );
}
void assign1( PyObject *object )
{
assertObject( object );
assertObject( this->object );
PyObject *old_object = this->object;
this->object = object;
Py_XDECREF( old_object );
}
private:
PyObjectTemporary( const PyObjectTemporary &object ) { assert( false ); }
PyObjectTemporary() { assert( false ); };
PyObject *object;
};
class PyObjectTemporaryWithDel
{
public:
explicit PyObjectTemporaryWithDel( PyObject *object )
{
assertObject( object );
this->object = object;
}
~PyObjectTemporaryWithDel()
{
if ( this->object ) assertObject( this->object );
Py_XDECREF( this->object );
}
PyObject *asObject0() const
{
assertObject( this->object );
return this->object;
}
PyObject *asObject1() const
{
return INCREASE_REFCOUNT( this->object );
}
void assign0( PyObject *object )
{
assertObject( object );
if ( this->object ) assertObject( this->object );
PyObject *old = this->object;
this->object = INCREASE_REFCOUNT( object );
Py_XDECREF( old );
}
void assign1( PyObject *object )
{
assertObject( object );
if ( this->object ) assertObject( this->object );
PyObject *old_object = this->object;
this->object = object;
Py_XDECREF( old_object );
}
// TODO: Tolerance must die, really and this method should be avoidable, as
// it's only used to emulate block scopes.
void del( bool tolerant )
{
if ( this->object ) assertObject( this->object );
Py_XDECREF( this->object );
this->object = NULL;
}
private:
PyObjectTemporaryWithDel( const PyObjectTemporary &object ) { assert( false ); }
PyObjectTemporaryWithDel() { assert( false ); };
PyObject *object;
};
class PyObjectTempVariable
{
public:
explicit PyObjectTempVariable()
{
this->object = NULL;
}
~PyObjectTempVariable()
{
if ( this->object ) assertObject( this->object );
Py_XDECREF( this->object );
}
PyObject *asObject0() const
{
assertObject( this->object );
return this->object;
}
PyObject *asObject1() const
{
assertObject( this->object );
return INCREASE_REFCOUNT( this->object );
}
void assign1( PyObject *object )
{
if ( this->object ) assertObject( this->object );
Py_XDECREF( this->object );
assertObject( object );
this->object = object;
}
void assign0( PyObject *object )
{
if ( this->object ) assertObject( this->object );
Py_XDECREF( this->object );
assertObject( object );
this->object = INCREASE_REFCOUNT( object );
}
void del( bool tolerant )
{
if ( this->object ) assertObject( this->object );
Py_XDECREF( this->object );
this->object = NULL;
}
private:
PyObjectTempVariable( const PyObjectTempVariable &object )
{
assert( false );
}
PyObject *object;
};
class PyObjectTempKeeper1
{
public:
explicit PyObjectTempKeeper1()
{
this->object = NULL;
}
~PyObjectTempKeeper1()
{
Py_XDECREF( this->object );
}
inline PyObject *asObject1() const
{
assertObject( this->object );
return INCREASE_REFCOUNT( this->object );
}
inline PyObject *asObject0() const
{
assertObject( this->object );
return this->object;
}
inline PyObject *assign( PyObject *value )
{
assertObject( value );
this->object = value;
return this->object;
}
bool isKeeping() const
{
return this->object != NULL;
}
private:
PyObjectTempKeeper1( const PyObjectTempKeeper1 &other ) { assert( false ); }
PyObject *object;
};
class PyObjectTempKeeper0
{
public:
explicit PyObjectTempKeeper0()
{
this->object = NULL;
}
~PyObjectTempKeeper0()
{}
inline PyObject *asObject0() const
{
assertObject( this->object );
return this->object;
}
inline PyObject *asObject1() const
{
assertObject( this->object );
return INCREASE_REFCOUNT( this->object );
}
inline PyObject *assign( PyObject *value )
{
assertObject( value );
this->object = value;
return this->object;
}
private:
PyObjectTempKeeper0( const PyObjectTempKeeper0 &other ) { assert( false ); }
PyObject *object;
};
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/importing.hpp 0000644 0001750 0001750 00000002102 12265264105 023367 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_IMPORTING_H__
#define __NUITKA_IMPORTING_H__
extern PyObject *IMPORT_MODULE( PyObject *module_name, PyObject *globals, PyObject *locals, PyObject *import_items, PyObject *level );
extern void IMPORT_MODULE_STAR( PyObject *target, bool is_module, PyObject *module );
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/variables_locals.hpp 0000644 0001750 0001750 00000010102 12265264105 024663 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_VARIABLES_LOCALS_H__
#define __NUITKA_VARIABLES_LOCALS_H__
class PyObjectLocalVariable
{
public:
explicit PyObjectLocalVariable( PyObject *var_name, PyObject *object = NULL )
{
this->var_name = var_name;
this->object = object;
}
explicit PyObjectLocalVariable()
{
this->var_name = NULL;
this->object = NULL;
}
~PyObjectLocalVariable()
{
Py_XDECREF( this->object );
}
void setVariableName( PyObject *var_name )
{
assertObject( var_name );
assert( this->var_name == NULL);
this->var_name = var_name;
}
void assign0( PyObject *object )
{
assertObject( object );
PyObject *old_object = this->object;
this->object = INCREASE_REFCOUNT( object );
Py_XDECREF( old_object );
}
void assign1( PyObject *object )
{
assertObject( object );
PyObject *old_object = this->object;
this->object = object;
Py_XDECREF( old_object );
}
PyObject *asObject0() const
{
if ( this->object == NULL && this->var_name != NULL )
{
PyErr_Format( PyExc_UnboundLocalError, "local variable '%s' referenced before assignment", Nuitka_String_AsString( this->var_name ) );
throw PythonException();
}
assertObject( this->object );
return this->object;
}
PyObject *asObject1() const
{
return INCREASE_REFCOUNT( this->asObject0() );
}
bool isInitialized() const
{
return this->object != NULL;
}
void del( bool tolerant )
{
if ( this->object == NULL )
{
if ( tolerant == false )
{
PyErr_Format( PyExc_UnboundLocalError, "local variable '%s' referenced before assignment", Nuitka_String_AsString( this->var_name ) );
throw PythonException();
}
}
else
{
PyObject *old_object = this->object;
this->object = NULL;
Py_DECREF( old_object );
}
}
PyObject *getVariableName() const
{
return this->var_name;
}
PyObject *updateLocalsDict( PyObject *locals_dict ) const
{
assert( PyDict_Check( locals_dict ) );
if ( this->isInitialized() )
{
#if PYTHON_VERSION < 300
int status = PyDict_SetItem(
#else
int status = PyObject_SetItem(
#endif
locals_dict,
this->getVariableName(),
this->asObject0()
);
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
return locals_dict;
}
PyObject *updateLocalsDir( PyObject *locals_list ) const
{
assert( PyList_Check( locals_list ) );
if ( this->isInitialized() )
{
int status = PyList_Append(
locals_list,
this->getVariableName()
);
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
return locals_list;
}
private:
PyObjectLocalVariable( const PyObjectLocalVariable &other ) { assert( false ); }
PyObject *var_name;
PyObject *object;
};
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/compiled_method.hpp 0000644 0001750 0001750 00000003313 12265264105 024520 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_COMPILED_METHOD_H__
#define __NUITKA_COMPILED_METHOD_H__
// Compiled function and compile generator types may be referenced.
#include "compiled_function.hpp"
#include "compiled_generator.hpp"
// The backbone of the integration into CPython. Try to behave as well as normal
// method objects, or even better.
// The Nuitka_MethodObject is the storage associated with a compiled method
// instance of which there can be many for each code.
typedef struct {
PyObject_HEAD
Nuitka_FunctionObject *m_function;
PyObject *m_weakrefs;
PyObject *m_object;
PyObject *m_class;
} Nuitka_MethodObject;
extern PyTypeObject Nuitka_Method_Type;
// Make a method out of a function.
extern PyObject *Nuitka_Method_New( Nuitka_FunctionObject *function, PyObject *object, PyObject *klass );
static inline bool Nuitka_Method_Check( PyObject *object )
{
return Py_TYPE( object ) == &Nuitka_Method_Type;
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/fibers.hpp 0000644 0001750 0001750 00000002412 12265264105 022635 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_FIBERS_H__
#define __NUITKA_FIBERS_H__
#if defined( _WIN32 )
#include
#else
#include
#endif
typedef struct _Fiber
{
#if defined( _WIN32 )
LPVOID fiber;
#else
ucontext_t f_context;
void *start_stack;
#endif
} Fiber;
extern "C" void initFiber( Fiber *to );
extern "C" void swapFiber( Fiber *to, Fiber *from );
extern "C" void prepareFiber( Fiber *to, void *code, unsigned long arg );
extern "C" void releaseFiber( Fiber *to );
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/prelude.hpp 0000644 0001750 0001750 00000012215 12265264105 023025 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_PRELUDE_H__
#define __NUITKA_PRELUDE_H__
#ifdef __NUITKA_NO_ASSERT__
#define NDEBUG
#endif
// Include the Python version numbers, and define our own take of what
// versions should be
#include "patchlevel.h"
#define PYTHON_VERSION (PY_MAJOR_VERSION*100+PY_MINOR_VERSION*10+PY_MICRO_VERSION)
// This is needed or else we can't create modules name "proc" or "func". For
// Python3, the name collision can't happen, so we can limit it to Python2.
#if PYTHON_VERSION < 300
#define initproc python_initproc
#define initfunc python_initfunc
#endif
// Include the Python C/API header files
#include "Python.h"
#include "methodobject.h"
#include "frameobject.h"
#include "pydebug.h"
#if PYTHON_VERSION < 300
#undef initproc
#undef initfunc
#endif
// Include the C header files most often used.
#include
// An idea I first saw used with Cython, hint the compiler about branches that
// are more or less likely to be taken. And hint the compiler about things that
// we assume to be normally true. If other compilers can do similar, I would be
// grateful for howtos.
#ifdef __GNUC__
#define likely(x) __builtin_expect(!!(x), 1)
#define unlikely(x) __builtin_expect(!!(x), 0)
#else
#define likely(x) (x)
#define unlikely(x) (x)
#endif
// A way to not give warnings about things that are declared, but might not be
// used like inline helper functions in headers or static per module variables
// from headers.
#ifdef __GNUC__
#define NUITKA_MAY_BE_UNUSED __attribute__((__unused__))
#else
#define NUITKA_MAY_BE_UNUSED
#endif
#ifdef __GNUC__
#define NUITKA_NO_RETURN __attribute__((__noreturn__))
#else
#define NUITKA_NO_RETURN
#endif
#ifdef __GNUC__
#define NUITKA_FORCE_INLINE __attribute__((always_inline))
#else
#define NUITKA_FORCE_INLINE
#endif
NUITKA_MAY_BE_UNUSED static PyObject *_eval_globals_tmp;
NUITKA_MAY_BE_UNUSED static PyObject *_eval_locals_tmp;
#if PYTHON_VERSION >= 300
// Python3 removed PyInt instead of renaming PyLong.
#define PyInt_FromString PyLong_FromString
#define PyInt_FromLong PyLong_FromLong
#define PyInt_AsLong PyLong_AsLong
#define PyInt_FromSsize_t PyLong_FromSsize_t
#define PyNumber_Int PyNumber_Long
#define PyObject_Unicode PyObject_Str
#endif
#if PYTHON_VERSION < 300
#define Nuitka_String_AsString PyString_AsString
#define Nuitka_String_AsString_Unchecked PyString_AS_STRING
#define Nuitka_String_Check PyString_Check
#define Nuitka_StringObject PyStringObject
#define Nuitka_StringIntern PyString_InternInPlace
#else
#define Nuitka_String_AsString _PyUnicode_AsString
// Note: There seems to be no variant that does it without checks, so rolled our
// own.
#define Nuitka_String_AsString_Unchecked _PyUnicode_AS_STRING
#define Nuitka_String_Check PyUnicode_Check
#define Nuitka_StringObject PyUnicodeObject
#define Nuitka_StringIntern PyUnicode_InternInPlace
#endif
// With the idea to reduce the amount of exported symbols in the DLLs, make it
// clear that the module init function should of course be exported, but not for
// executable, where we call it ourselves.
#if defined( _NUITKA_EXE )
#if PYTHON_VERSION < 300
#define NUITKA_MODULE_INIT_FUNCTION void
#else
#define NUITKA_MODULE_INIT_FUNCTION PyObject *
#endif
#elif defined( __GNUC__ )
#if PYTHON_VERSION < 300
#define NUITKA_MODULE_INIT_FUNCTION PyMODINIT_FUNC __attribute__(( visibility( "default" )))
#else
#define NUITKA_MODULE_INIT_FUNCTION extern "C" __attribute__(( visibility( "default" ))) PyObject *
#endif
#else
#define NUITKA_MODULE_INIT_FUNCTION PyMODINIT_FUNC
#endif
// The name of the entry point for DLLs changed between Python versions, and
// this is.
#if PYTHON_VERSION < 300
#define MOD_INIT_NAME( name ) init##name
#define MOD_INIT_DECL( name ) NUITKA_MODULE_INIT_FUNCTION init##name( void )
#define MOD_RETURN_VALUE( value )
#else
#define MOD_INIT_NAME( name ) PyInit_##name
#define MOD_INIT_DECL( name ) NUITKA_MODULE_INIT_FUNCTION PyInit_##name( void )
#define MOD_RETURN_VALUE( value ) value
#endif
// These two express if a directly called function should be exported (C++
// level) or if it can be local to the file.
#define NUITKA_CROSS_MODULE
#define NUITKA_LOCAL_MODULE static
#include "nuitka/helpers.hpp"
#include "nuitka/compiled_function.hpp"
// Sentinel PyObject to be used for all our call iterator endings.
extern PyObject *_sentinel_value;
#include "nuitka/compiled_generator.hpp"
#include "nuitka/compiled_method.hpp"
#include "nuitka/compiled_frame.hpp"
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/printing.hpp 0000644 0001750 0001750 00000002421 12265270763 023224 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_PRINTING_H__
#define __NUITKA_PRINTING_H__
// Helper functions for print. Need to play nice with Python softspace behaviour.
#include "nuitka/exceptions.hpp"
extern void PRINT_ITEM_TO( PyObject *file, PyObject *object );
extern void PRINT_NEW_LINE_TO( PyObject *file );
extern void PRINT_NEW_LINE( void );
extern PyObject *GET_STDOUT();
extern PyObject *GET_STDERR();
// Helper functions to debug the compiler operation.
extern void PRINT_REFCOUNT( PyObject *object );
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/compiled_function.hpp 0000644 0001750 0001750 00000013033 12265264105 025065 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_COMPILED_FUNCTION_H__
#define __NUITKA_COMPILED_FUNCTION_H__
#include "Python.h"
#include "frameobject.h"
// Compiled function type.
// The backbone of the integration into CPython. Try to behave as well as normal
// functions and builtin functions, or even better.
// Cleanup function to be called when the function object is released.
typedef void (*releaser)( void * );
struct Nuitka_FunctionObject;
// Standard Python entry point, accepting argument tuple and keyword dict.
typedef PyObject *(*function_arg_parser)( Nuitka_FunctionObject *, PyObject **, Py_ssize_t, PyObject * );
// Quick call variant, only plain arguments are present and not formed as a
// tuple, allowing shortcuts.
typedef PyObject *(*direct_arg_parser)( Nuitka_FunctionObject *, PyObject **, int );
// The Nuitka_FunctionObject is the storage associated with a compiled function
// instance of which there can be many for each code.
struct Nuitka_FunctionObject {
PyObject_HEAD
PyObject *m_name;
void *m_context;
releaser m_cleanup;
PyObject *m_module;
PyObject *m_doc;
PyCodeObject *m_code_object;
function_arg_parser m_code;
direct_arg_parser m_direct_arg_parser;
PyObject *m_dict;
PyObject *m_weakrefs;
// List of defaults, for use in __defaults__ and parameter parsing.
PyObject *m_defaults;
Py_ssize_t m_defaults_given;
#if PYTHON_VERSION >= 300
// List of keyword only defaults, for use in __kwdefaults__ and parameter
// parsing.
PyObject *m_kwdefaults;
// Annotations to the function arguments and return value.
PyObject *m_annotations;
#endif
#if PYTHON_VERSION >= 330
PyObject *m_qualname;
#endif
long m_counter;
};
extern PyTypeObject Nuitka_Function_Type;
// Make a function without context.
#if PYTHON_VERSION < 300
extern PyObject *Nuitka_Function_New( function_arg_parser code, direct_arg_parser, PyObject *name, PyCodeObject *code_object, PyObject *defaults, PyObject *module, PyObject *doc );
#elif PYTHON_VERSION < 330
extern PyObject *Nuitka_Function_New( function_arg_parser code, direct_arg_parser, PyObject *name, PyCodeObject *code_object, PyObject *defaults, PyObject *kwdefaults, PyObject *annotations, PyObject *module, PyObject *doc );
#else
extern PyObject *Nuitka_Function_New( function_arg_parser code, direct_arg_parser, PyObject *name, PyObject *qualname, PyCodeObject *code_object, PyObject *defaults, PyObject *kwdefaults, PyObject *annotations, PyObject *module, PyObject *doc );
#endif
// Make a function with context.
#if PYTHON_VERSION < 300
extern PyObject *Nuitka_Function_New( function_arg_parser code, direct_arg_parser dparse, PyObject *name, PyCodeObject *code_object, PyObject *defaults, PyObject *module, PyObject *doc, void *context, releaser cleanup );
#elif PYTHON_VERSION < 330
extern PyObject *Nuitka_Function_New( function_arg_parser code, direct_arg_parser dparse, PyObject *name, PyCodeObject *code_object, PyObject *defaults, PyObject *kwdefaults, PyObject *annotations, PyObject *module, PyObject *doc, void *context, releaser cleanup );
#else
extern PyObject *Nuitka_Function_New( function_arg_parser code, direct_arg_parser dparse, PyObject *name, PyObject *qualname, PyCodeObject *code_object, PyObject *defaults, PyObject *kwdefaults, PyObject *annotations, PyObject *module, PyObject *doc, void *context, releaser cleanup );
#endif
static inline bool Nuitka_Function_Check( PyObject *object )
{
return Py_TYPE( object ) == &Nuitka_Function_Type;
}
static inline PyObject *Nuitka_Function_GetName( PyObject *object )
{
return ((Nuitka_FunctionObject *)object)->m_name;
}
extern void ERROR_MULTIPLE_VALUES( Nuitka_FunctionObject *function,
Py_ssize_t index );
extern void ERROR_TOO_MANY_ARGUMENTS( struct Nuitka_FunctionObject *function,
Py_ssize_t given
#if PYTHON_VERSION < 270
, Py_ssize_t kw_size
#endif
#if PYTHON_VERSION >= 330
, Py_ssize_t kw_only
#endif
);
#if PYTHON_VERSION < 330
extern void ERROR_TOO_FEW_ARGUMENTS( Nuitka_FunctionObject *function,
#if PYTHON_VERSION < 270
Py_ssize_t kw_size,
#endif
Py_ssize_t given );
#else
extern void ERROR_TOO_FEW_ARGUMENTS( Nuitka_FunctionObject *function,
PyObject **values);
#endif
extern void ERROR_NO_ARGUMENTS_ALLOWED( struct Nuitka_FunctionObject *function,
#if PYTHON_VERSION >= 330
PyObject *kw,
#endif
Py_ssize_t given );
#if PYTHON_VERSION >= 330
extern void ERROR_TOO_FEW_KWONLY( struct Nuitka_FunctionObject *function,
PyObject **kw_vars );
#endif
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/helper/ 0000755 0001750 0001750 00000000000 12265271051 022130 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/build/include/nuitka/helper/richcomparisons.hpp 0000644 0001750 0001750 00000021554 12265264105 026055 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_HELPER_RICHCOMPARISONS_H__
#define __NUITKA_HELPER_RICHCOMPARISONS_H__
static inline bool IS_SANE_TYPE( PyTypeObject *type )
{
return
#if PYTHON_VERSION < 300
type == &PyString_Type ||
type == &PyInt_Type ||
#endif
type == &PyLong_Type ||
type == &PyList_Type ||
type == &PyTuple_Type;
}
extern PyObject *MY_RICHCOMPARE( PyObject *v, PyObject *w, int op );
NUITKA_MAY_BE_UNUSED static PyObject *RICH_COMPARE_LT( PyObject *operand1, PyObject *operand2 )
{
PyObject *result = MY_RICHCOMPARE( operand1, operand2, Py_LT );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *RICH_COMPARE_LE( PyObject *operand1, PyObject *operand2 )
{
// Quick path for avoidable checks, compatible with CPython.
if ( operand1 == operand2 && IS_SANE_TYPE( Py_TYPE( operand1 ) ) )
{
return INCREASE_REFCOUNT( Py_True );
}
PyObject *result = MY_RICHCOMPARE( operand1, operand2, Py_LE );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *RICH_COMPARE_EQ( PyObject *operand1, PyObject *operand2 )
{
// Quick path for avoidable checks, compatible with CPython.
if ( operand1 == operand2 && IS_SANE_TYPE( Py_TYPE( operand1 ) ) )
{
return INCREASE_REFCOUNT( Py_True );
}
PyObject *result = MY_RICHCOMPARE( operand1, operand2, Py_EQ );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *RICH_COMPARE_NE( PyObject *operand1, PyObject *operand2 )
{
// Quick path for avoidable checks, compatible with CPython.
if ( operand1 == operand2 && IS_SANE_TYPE( Py_TYPE( operand1 ) ) )
{
return INCREASE_REFCOUNT( Py_False );
}
PyObject *result = MY_RICHCOMPARE( operand1, operand2, Py_NE );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *RICH_COMPARE_GT( PyObject *operand1, PyObject *operand2 )
{
PyObject *result = MY_RICHCOMPARE( operand1, operand2, Py_GT );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *RICH_COMPARE_GE( PyObject *operand1, PyObject *operand2 )
{
PyObject *result = MY_RICHCOMPARE( operand1, operand2, Py_GE );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static bool RICH_COMPARE_BOOL_LT( PyObject *operand1, PyObject *operand2 )
{
PyObject *rich_result = MY_RICHCOMPARE( operand1, operand2, Py_LT );
if (unlikely( rich_result == NULL ))
{
throw PythonException();
}
bool result;
// Doing the quick tests on the outside spares the function call, with
// "partial inline" this should become unneeded.
if ( rich_result == Py_True )
{
result = true;
}
else if ( rich_result == Py_False || rich_result == Py_None )
{
result = false;
}
else
{
result = CHECK_IF_TRUE( rich_result );
}
Py_DECREF( rich_result );
return result;
}
NUITKA_MAY_BE_UNUSED static bool RICH_COMPARE_BOOL_LE( PyObject *operand1, PyObject *operand2 )
{
// Quick path for avoidable checks, compatible with CPython.
if ( operand1 == operand2 && IS_SANE_TYPE( Py_TYPE( operand1 ) ) )
{
return true;
}
PyObject *rich_result = MY_RICHCOMPARE( operand1, operand2, Py_LE );
if (unlikely( rich_result == NULL ))
{
throw PythonException();
}
bool result;
// Doing the quick tests on the outside spares the function call, with
// "partial inline" this should become unneeded.
if ( rich_result == Py_True )
{
result = true;
}
else if ( rich_result == Py_False || rich_result == Py_None )
{
result = false;
}
else
{
result = CHECK_IF_TRUE( rich_result );
}
Py_DECREF( rich_result );
return result;
}
NUITKA_MAY_BE_UNUSED static bool RICH_COMPARE_BOOL_EQ_PARAMETERS( PyObject *operand1, PyObject *operand2 )
{
assertObject( operand1 );
assertObject( operand2 );
PyObject *rich_result = MY_RICHCOMPARE( operand1, operand2, Py_EQ );
// String comparisons cannot fail they say.
assertObject( rich_result );
bool result;
// Doing the quick tests on the outside spares the function call, with
// "partial inline" this should become unneeded.
if ( rich_result == Py_True )
{
result = true;
}
else if ( rich_result == Py_False || rich_result == Py_None )
{
result = false;
}
else
{
result = CHECK_IF_TRUE( rich_result );
}
Py_DECREF( rich_result );
return result;
}
NUITKA_MAY_BE_UNUSED static bool RICH_COMPARE_BOOL_EQ( PyObject *operand1, PyObject *operand2 )
{
// Quick path for avoidable checks, compatible with CPython.
if ( operand1 == operand2 && IS_SANE_TYPE( Py_TYPE( operand1 ) ) )
{
return true;
}
PyObject *rich_result = MY_RICHCOMPARE( operand1, operand2, Py_EQ );
if (unlikely( rich_result == NULL ))
{
throw PythonException();
}
bool result;
// Doing the quick tests on the outside spares the function call, with
// "partial inline" this should become unneeded.
if ( rich_result == Py_True )
{
result = true;
}
else if ( rich_result == Py_False || rich_result == Py_None )
{
result = false;
}
else
{
result = CHECK_IF_TRUE( rich_result );
}
Py_DECREF( rich_result );
return result;
}
NUITKA_MAY_BE_UNUSED static bool RICH_COMPARE_BOOL_NE( PyObject *operand1, PyObject *operand2 )
{
// Quick path for avoidable checks, compatible with CPython.
if ( operand1 == operand2 && IS_SANE_TYPE( Py_TYPE( operand1 ) ) )
{
return false;
}
PyObject *rich_result = MY_RICHCOMPARE( operand1, operand2, Py_NE );
if (unlikely( rich_result == NULL ))
{
throw PythonException();
}
bool result;
// Doing the quick tests on the outside spares the function call, with
// "partial inline" this should become unneeded.
if ( rich_result == Py_True )
{
result = true;
}
else if ( rich_result == Py_False || rich_result == Py_None )
{
result = false;
}
else
{
result = CHECK_IF_TRUE( rich_result );
}
Py_DECREF( rich_result );
return result;
}
NUITKA_MAY_BE_UNUSED static bool RICH_COMPARE_BOOL_GT( PyObject *operand1, PyObject *operand2 )
{
PyObject *rich_result = MY_RICHCOMPARE( operand1, operand2, Py_GT );
if (unlikely( rich_result == NULL ))
{
throw PythonException();
}
bool result;
// Doing the quick tests on the outside spares the function call, with
// "partial inline" this should become unneeded.
if ( rich_result == Py_True )
{
result = true;
}
else if ( rich_result == Py_False || rich_result == Py_None )
{
result = false;
}
else
{
result = CHECK_IF_TRUE( rich_result );
}
Py_DECREF( rich_result );
return result;
}
NUITKA_MAY_BE_UNUSED static bool RICH_COMPARE_BOOL_GE( PyObject *operand1, PyObject *operand2 )
{
// Quick path for avoidable checks, compatible with CPython.
if ( operand1 == operand2 && IS_SANE_TYPE( Py_TYPE( operand1 ) ) )
{
return true;
}
PyObject *rich_result = MY_RICHCOMPARE( operand1, operand2, Py_GE );
if (unlikely( rich_result == NULL ))
{
throw PythonException();
}
bool result;
// Doing the quick tests on the outside spares the function call, with
// "partial inline" this should become unneeded.
if ( rich_result == Py_True )
{
result = true;
}
else if ( rich_result == Py_False || rich_result == Py_None )
{
result = false;
}
else
{
result = CHECK_IF_TRUE( rich_result );
}
Py_DECREF( rich_result );
return result;
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/helper/subscripts.hpp 0000644 0001750 0001750 00000023610 12265264105 025046 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_HELPER_SUBSCRIPTS_H__
#define __NUITKA_HELPER_SUBSCRIPTS_H__
extern PyObject *BUILTIN_CHR( unsigned char c );
NUITKA_MAY_BE_UNUSED static PyObject *LOOKUP_SUBSCRIPT_CONST( PyObject *source, PyObject *const_subscript, Py_ssize_t int_subscript )
{
assertObject( source );
assertObject( const_subscript );
PyTypeObject *type = Py_TYPE( source );
PyMappingMethods *mapping_methods = type->tp_as_mapping;
PyObject *result;
if ( mapping_methods && mapping_methods->mp_subscript )
{
if ( PyList_CheckExact( source ) )
{
Py_ssize_t list_size = PyList_GET_SIZE( source );
if ( int_subscript < 0 )
{
if ( -int_subscript > list_size )
{
PyErr_Format( PyExc_IndexError, "list index out of range" );
throw PythonException();
}
int_subscript += list_size;
}
else
{
if ( int_subscript >= list_size )
{
PyErr_Format( PyExc_IndexError, "list index out of range" );
throw PythonException();
}
}
return INCREASE_REFCOUNT( ((PyListObject *)source)->ob_item[ int_subscript ] );
}
#if PYTHON_VERSION < 300
else if ( PyString_CheckExact( source ) )
{
Py_ssize_t string_size = PyString_GET_SIZE( source );
if ( int_subscript < 0 )
{
if ( -int_subscript > string_size )
{
PyErr_Format( PyExc_IndexError, "string index out of range" );
throw PythonException();
}
int_subscript += string_size;
}
else
{
if ( int_subscript >= string_size )
{
PyErr_Format( PyExc_IndexError, "string index out of range" );
throw PythonException();
}
}
unsigned char c = ((PyStringObject *)source)->ob_sval[ int_subscript ];
return BUILTIN_CHR( c );
}
#else
else if ( PyUnicode_CheckExact( source ) )
{
if ( int_subscript < 0 )
{
#if PYTHON_VERSION < 330
int_subscript += PyUnicode_GET_SIZE( source );
#else
int_subscript += PyUnicode_GET_LENGTH( source );
#endif
}
result = type->tp_as_sequence->sq_item( source, int_subscript );
}
#endif
else
{
result = mapping_methods->mp_subscript( source, const_subscript );
}
}
else if ( type->tp_as_sequence )
{
result = PySequence_GetItem( source, int_subscript );
}
else
{
PyErr_Format(
PyExc_TypeError,
#if PYTHON_VERSION < 270
"'%s' object is unsubscriptable",
#else
"'%s' object has no attribute '__getitem__'",
#endif
Py_TYPE( source )->tp_name
);
throw PythonException();
}
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *LOOKUP_SUBSCRIPT( PyObject *source, PyObject *subscript )
{
assertObject( source );
assertObject( subscript );
PyTypeObject *type = Py_TYPE( source );
PyMappingMethods *mapping = type->tp_as_mapping;
PyObject *result;
if ( mapping != NULL && mapping->mp_subscript != NULL )
{
result = mapping->mp_subscript( source, subscript );
}
else if ( type->tp_as_sequence != NULL )
{
if ( PyIndex_Check( subscript ) )
{
result = PySequence_GetItem( source, CONVERT_TO_INDEX( subscript ) );
}
else if ( type->tp_as_sequence->sq_item )
{
PyErr_Format( PyExc_TypeError, "sequence index must be integer, not '%s'", Py_TYPE( subscript )->tp_name );
throw PythonException();
}
else
{
PyErr_Format(
PyExc_TypeError,
#if PYTHON_VERSION < 270
"'%s' object is unsubscriptable",
#else
"'%s' object has no attribute '__getitem__'",
#endif
Py_TYPE( source )->tp_name
);
throw PythonException();
}
}
else
{
PyErr_Format(
PyExc_TypeError,
#if PYTHON_VERSION < 270
"'%s' object is unsubscriptable",
#else
"'%s' object has no attribute '__getitem__'",
#endif
Py_TYPE( source )->tp_name
);
throw PythonException();
}
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static void SET_SUBSCRIPT_CONST( PyObject *value, PyObject *target, PyObject *subscript, Py_ssize_t int_subscript )
{
assertObject( value );
assertObject( target );
assertObject( subscript );
PyMappingMethods *mapping_methods = Py_TYPE( target )->tp_as_mapping;
if ( mapping_methods != NULL && mapping_methods->mp_ass_subscript )
{
if ( PyList_CheckExact( target ) )
{
Py_ssize_t list_size = PyList_GET_SIZE( target );
if ( int_subscript < 0 )
{
if ( -int_subscript > list_size )
{
PyErr_Format(
PyExc_IndexError,
"list assignment index out of range"
);
throw PythonException();
}
int_subscript += list_size;
}
PyListObject *target_list = (PyListObject *)target;
PyObject *old_value = target_list->ob_item[ int_subscript ];
target_list->ob_item[ int_subscript ] = INCREASE_REFCOUNT( value );
Py_DECREF( old_value );
}
else
{
int res = mapping_methods->mp_ass_subscript( target, subscript, value );
if (unlikely( res == -1 ))
{
throw PythonException();
}
}
}
else if ( Py_TYPE( target )->tp_as_sequence )
{
if ( PyIndex_Check( subscript ) )
{
Py_ssize_t key_value = PyNumber_AsSsize_t( subscript, PyExc_IndexError );
if ( key_value == -1 )
{
THROW_IF_ERROR_OCCURED();
}
SEQUENCE_SETITEM( target, key_value, value );
}
else if ( Py_TYPE( target )->tp_as_sequence->sq_ass_item )
{
PyErr_Format(
PyExc_TypeError,
"sequence index must be integer, not '%s'",
Py_TYPE( subscript )->tp_name
);
throw PythonException();
}
else
{
PyErr_Format(
PyExc_TypeError,
"'%s' object does not support item assignment",
Py_TYPE( target )->tp_name
);
throw PythonException();
}
}
else
{
PyErr_Format(
PyExc_TypeError,
"'%s' object does not support item assignment",
Py_TYPE( target )->tp_name
);
throw PythonException();
}
}
NUITKA_MAY_BE_UNUSED static void SET_SUBSCRIPT( PyObject *value, PyObject *target, PyObject *subscript )
{
assertObject( value );
assertObject( target );
assertObject( subscript );
PyMappingMethods *mapping_methods = Py_TYPE( target )->tp_as_mapping;
if ( mapping_methods != NULL && mapping_methods->mp_ass_subscript )
{
int res = mapping_methods->mp_ass_subscript( target, subscript, value );
if (unlikely( res == -1 ))
{
throw PythonException();
}
}
else if ( Py_TYPE( target )->tp_as_sequence )
{
if ( PyIndex_Check( subscript ) )
{
Py_ssize_t key_value = PyNumber_AsSsize_t( subscript, PyExc_IndexError );
if ( key_value == -1 )
{
THROW_IF_ERROR_OCCURED();
}
SEQUENCE_SETITEM( target, key_value, value );
}
else if ( Py_TYPE( target )->tp_as_sequence->sq_ass_item )
{
PyErr_Format(
PyExc_TypeError,
"sequence index must be integer, not '%s'",
Py_TYPE( subscript )->tp_name
);
throw PythonException();
}
else
{
PyErr_Format(
PyExc_TypeError,
"'%s' object does not support item assignment",
Py_TYPE( target )->tp_name
);
throw PythonException();
}
}
else
{
PyErr_Format(
PyExc_TypeError,
"'%s' object does not support item assignment",
Py_TYPE( target )->tp_name
);
throw PythonException();
}
}
NUITKA_MAY_BE_UNUSED static void DEL_SUBSCRIPT( PyObject *target, PyObject *subscript )
{
assertObject( target );
assertObject( subscript );
int status = PyObject_DelItem( target, subscript );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/helper/boolean.hpp 0000644 0001750 0001750 00000004736 12265264105 024274 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_HELPER_BOOLEAN_H__
#define __NUITKA_HELPER_BOOLEAN_H__
#if PYTHON_VERSION >= 300
#define nb_nonzero nb_bool
#endif
NUITKA_MAY_BE_UNUSED static bool CHECK_IF_TRUE( PyObject *object )
{
assertObject( object );
if ( object == Py_True )
{
return true;
}
else if ( object == Py_False || object == Py_None )
{
return false;
}
else
{
Py_ssize_t result;
if ( Py_TYPE( object )->tp_as_number != NULL && Py_TYPE( object )->tp_as_number->nb_nonzero != NULL )
{
result = (*Py_TYPE( object )->tp_as_number->nb_nonzero)( object );
}
else if ( Py_TYPE( object )->tp_as_mapping != NULL && Py_TYPE( object )->tp_as_mapping->mp_length != NULL )
{
result = (*Py_TYPE( object )->tp_as_mapping->mp_length)( object );
}
else if ( Py_TYPE( object )->tp_as_sequence != NULL && Py_TYPE( object )->tp_as_sequence->sq_length != NULL )
{
result = (*Py_TYPE( object )->tp_as_sequence->sq_length)( object );
}
else
{
return true;
}
if ( result > 0 )
{
return true;
}
else if ( result == 0 )
{
return false;
}
else
{
throw PythonException();
}
}
}
NUITKA_MAY_BE_UNUSED static bool CHECK_IF_FALSE( PyObject *object )
{
return CHECK_IF_TRUE( object ) == false;
}
NUITKA_MAY_BE_UNUSED static PyObject *BOOL_FROM( bool value )
{
return value ? Py_True : Py_False;
}
NUITKA_MAY_BE_UNUSED static PyObject *UNARY_NOT( PyObject *object )
{
return BOOL_FROM( CHECK_IF_FALSE( object ) );
}
#undef nb_nonzero
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/helper/slices.hpp 0000644 0001750 0001750 00000013203 12265264105 024124 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_HELPER_SLICES_H__
#define __NUITKA_HELPER_SLICES_H__
static inline bool IS_INDEXABLE( PyObject *value )
{
return
value == Py_None ||
#if PYTHON_VERSION < 300
PyInt_Check( value ) ||
#endif
PyLong_Check( value ) ||
PyIndex_Check( value );
}
#if PYTHON_VERSION < 300
// Note: It appears that Python3 has no index slicing operations anymore, but
// uses slice objects all the time. That's fine and make sure we adhere to it by
// guarding the presence of the helpers.
NUITKA_MAY_BE_UNUSED static PyObject *LOOKUP_SLICE( PyObject *source, PyObject *lower, PyObject *upper )
{
assertObject( source );
assertObject( lower );
assertObject( upper );
PySequenceMethods *tp_as_sequence = Py_TYPE( source )->tp_as_sequence;
if ( tp_as_sequence && tp_as_sequence->sq_slice && IS_INDEXABLE( lower ) && IS_INDEXABLE( upper ) )
{
Py_ssize_t ilow = 0;
if ( lower != Py_None )
{
ilow = CONVERT_TO_INDEX( lower );
}
Py_ssize_t ihigh = PY_SSIZE_T_MAX;
if ( upper != Py_None )
{
ihigh = CONVERT_TO_INDEX( upper );
}
PyObject *result = PySequence_GetSlice( source, ilow, ihigh );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
else
{
PyObject *slice = PySlice_New( lower, upper, NULL );
if (unlikely( slice == NULL ))
{
throw PythonException();
}
PyObject *result = PyObject_GetItem( source, slice );
Py_DECREF( slice );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
}
NUITKA_MAY_BE_UNUSED static PyObject *LOOKUP_INDEX_SLICE( PyObject *source, Py_ssize_t lower, Py_ssize_t upper )
{
assertObject( source );
PyObject *result = PySequence_GetSlice( source, lower, upper );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static void SET_SLICE( PyObject *value, PyObject *target, PyObject *lower, PyObject *upper )
{
assertObject( target );
assertObject( lower );
assertObject( upper );
assertObject( value );
PySequenceMethods *tp_as_sequence = Py_TYPE( target )->tp_as_sequence;
if ( tp_as_sequence && tp_as_sequence->sq_ass_slice && IS_INDEXABLE( lower ) && IS_INDEXABLE( upper ) )
{
Py_ssize_t lower_int = 0;
if ( lower != Py_None )
{
lower_int = CONVERT_TO_INDEX( lower );
}
Py_ssize_t upper_int = PY_SSIZE_T_MAX;
if ( upper != Py_None )
{
upper_int = CONVERT_TO_INDEX( upper );
}
int status = PySequence_SetSlice( target, lower_int, upper_int, value );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
else
{
PyObject *slice = PySlice_New( lower, upper, NULL );
if (unlikely( slice == NULL ))
{
throw PythonException();
}
int status = PyObject_SetItem( target, slice, value );
Py_DECREF( slice );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
}
NUITKA_MAY_BE_UNUSED static void SET_INDEX_SLICE( PyObject *target, Py_ssize_t lower, Py_ssize_t upper, PyObject *value )
{
assertObject( target );
assertObject( value );
int status = PySequence_SetSlice( target, lower, upper, value );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
NUITKA_MAY_BE_UNUSED static void DEL_SLICE( PyObject *target, Py_ssize_t lower, Py_ssize_t upper )
{
assertObject( target );
if ( Py_TYPE( target )->tp_as_sequence && Py_TYPE( target )->tp_as_sequence->sq_ass_slice )
{
int status = PySequence_DelSlice( target, lower, upper );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
else
{
PyObjectTemporary lower_obj( PyInt_FromSsize_t( lower ) );
PyObjectTemporary upper_obj( PyInt_FromSsize_t( upper ) );
PyObject *slice = PySlice_New( lower_obj.asObject0(), upper_obj.asObject0(), NULL );
if (unlikely( slice == NULL ))
{
throw PythonException();
}
int status = PyObject_DelItem( target, slice );
Py_DECREF( slice );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
}
#endif
NUITKA_MAY_BE_UNUSED static PyObject *MAKE_SLICEOBJ( PyObject *start, PyObject *stop, PyObject *step )
{
assertObject( start );
assertObject( stop );
assertObject( step );
PyObject *result = PySlice_New( start, stop, step );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/helper/dictionaries.hpp 0000644 0001750 0001750 00000016615 12265264105 025331 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_DICTIONARIES_H__
#define __NUITKA_DICTIONARIES_H__
static inline PyDictObject *MODULE_DICT( PyModuleObject *module )
{
PyDictObject *dict = (PyDictObject *)(module->md_dict);
return dict;
}
static inline PyDictObject *MODULE_DICT( PyObject *module )
{
return MODULE_DICT( (PyModuleObject *)module );
}
#if PYTHON_VERSION < 330
// Quick dictionary lookup for a string value.
typedef PyDictEntry *Nuitka_DictEntryHandle;
static PyDictEntry *GET_STRING_DICT_ENTRY( PyDictObject *dict, Nuitka_StringObject *key )
{
assert( PyDict_CheckExact( dict ) );
assert( Nuitka_String_Check( key ) );
#if PYTHON_VERSION < 300
long hash = key->ob_shash;
#else
long hash = key->hash;
#endif
// Only improvement would be to identify how to ensure that the hash is
// computed already. Calling hash early on could do that potentially.
if ( hash == -1 )
{
#if PYTHON_VERSION < 300
hash = PyString_Type.tp_hash( (PyObject *)key );
key->ob_shash = hash;
#else
hash = PyUnicode_Type.tp_hash( (PyObject *)key );
key->hash = hash;
#endif
}
PyDictEntry *entry = dict->ma_lookup( dict, (PyObject *)key, hash );
// The "entry" cannot be NULL, it can only be empty for a string dict
// lookup, but at least assert it.
assert( entry != NULL );
return entry;
}
NUITKA_MAY_BE_UNUSED static PyObject *GET_DICT_ENTRY_VALUE( Nuitka_DictEntryHandle handle )
{
return handle->me_value;
}
NUITKA_MAY_BE_UNUSED static void SET_DICT_ENTRY_VALUE( Nuitka_DictEntryHandle handle, PyObject *value )
{
handle->me_value = value;
}
static PyObject *GET_STRING_DICT_VALUE( PyDictObject *dict, Nuitka_StringObject *key )
{
return GET_STRING_DICT_ENTRY( dict, key )->me_value;
}
#else
// Quick dictionary lookup for a string value.
struct PyDictKeyEntry
{
/* Cached hash code of me_key. */
Py_hash_t me_hash;
PyObject *me_key;
PyObject *me_value; /* This field is only meaningful for combined tables */
};
typedef PyDictKeyEntry *(*dict_lookup_func)(PyDictObject *mp, PyObject *key, Py_hash_t hash, PyObject ***value_addr);
// Stolen from CPython3.3 dictobject.c
struct _dictkeysobject
{
Py_ssize_t dk_refcnt;
Py_ssize_t dk_size;
dict_lookup_func dk_lookup;
Py_ssize_t dk_usable;
PyDictKeyEntry dk_entries[1];
};
typedef PyObject **Nuitka_DictEntryHandle;
static Nuitka_DictEntryHandle GET_STRING_DICT_ENTRY( PyDictObject *dict, Nuitka_StringObject *key )
{
assert( PyDict_CheckExact( dict ) );
assert( Nuitka_String_Check( key ) );
Py_hash_t hash = key->_base._base.hash;
// Only improvement would be to identify how to ensure that the hash is computed
// already. Calling hash early on could do that potentially.
if ( hash == -1 )
{
hash = PyUnicode_Type.tp_hash( (PyObject *)key );
key->_base._base.hash = hash;
}
PyObject **value_addr;
PyDictKeyEntry *entry = dict->ma_keys->dk_lookup( dict, (PyObject *)key, hash, &value_addr );
// The "entry" cannot be NULL, it can only be empty for a string dict lookup, but at
// least assert it.
assert( entry != NULL );
return value_addr;
}
NUITKA_MAY_BE_UNUSED static PyObject *GET_DICT_ENTRY_VALUE( Nuitka_DictEntryHandle handle )
{
return *handle;
}
NUITKA_MAY_BE_UNUSED static void SET_DICT_ENTRY_VALUE( Nuitka_DictEntryHandle handle, PyObject *value )
{
*handle = value;
}
NUITKA_MAY_BE_UNUSED static PyObject *GET_STRING_DICT_VALUE( PyDictObject *dict, Nuitka_StringObject *key )
{
Nuitka_DictEntryHandle handle = GET_STRING_DICT_ENTRY( dict, key );
return GET_DICT_ENTRY_VALUE( handle );
}
#endif
NUITKA_MAY_BE_UNUSED static void DICT_SET_ITEM( PyObject *dict, PyObject *key, PyObject *value )
{
int status = PyDict_SetItem( dict, key, value );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
NUITKA_MAY_BE_UNUSED static inline void DICT_SET_ITEM( PyDictObject *dict, PyObject *key, PyObject *value )
{
return DICT_SET_ITEM( (PyObject *)dict, key, value );
}
NUITKA_MAY_BE_UNUSED static void DICT_REMOVE_ITEM( PyObject *dict, PyObject *key )
{
int status = PyDict_DelItem( dict, key );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
NUITKA_MAY_BE_UNUSED static PyObject *DICT_GET_ITEM( PyObject *dict, PyObject *key )
{
assertObject( dict );
assert( PyDict_Check( dict ) );
assertObject( key );
PyObject *result = PyDict_GetItem( dict, key );
if ( result == NULL )
{
if (unlikely( PyErr_Occurred() ))
{
throw PythonException();
}
PyErr_SetObject( PyExc_KeyError, key );
throw PythonException();
}
else
{
return INCREASE_REFCOUNT( result );
}
}
// Convert to dictionary, helper for builtin dict mainly.
NUITKA_MAY_BE_UNUSED static PyObject *TO_DICT( PyObject *seq_obj, PyObject *dict_obj )
{
PyObject *result = PyDict_New();
if ( seq_obj != NULL )
{
int res;
if ( PyObject_HasAttrString( seq_obj, "keys" ) )
{
res = PyDict_Merge( result, seq_obj, 1 );
}
else
{
res = PyDict_MergeFromSeq2( result, seq_obj, 1 );
}
if ( res == -1 )
{
throw PythonException();
}
}
if ( dict_obj != NULL )
{
int res = PyDict_Merge( result, dict_obj, 1 );
if ( res == -1 )
{
throw PythonException();
}
}
return result;
}
NUITKA_MAY_BE_UNUSED static void UPDATE_STRING_DICT0( PyDictObject *dict, Nuitka_StringObject *key, PyObject *value )
{
Nuitka_DictEntryHandle entry = GET_STRING_DICT_ENTRY( dict, key );
PyObject *old = GET_DICT_ENTRY_VALUE( entry );
// Values are more likely (more often) set than not set, in that case speculatively
// try the quickest access method.
if (likely( old != NULL ))
{
SET_DICT_ENTRY_VALUE( entry, INCREASE_REFCOUNT( value ) );
Py_DECREF( old );
}
else
{
DICT_SET_ITEM( dict, (PyObject *)key, value );
}
}
NUITKA_MAY_BE_UNUSED static void UPDATE_STRING_DICT1( PyDictObject *dict, Nuitka_StringObject *key, PyObject *value )
{
Nuitka_DictEntryHandle entry = GET_STRING_DICT_ENTRY( dict, key );
PyObject *old = GET_DICT_ENTRY_VALUE( entry );
// Values are more likely (more often) set than not set, in that case speculatively
// try the quickest access method.
if (likely( old != NULL ))
{
SET_DICT_ENTRY_VALUE( entry, value );
Py_DECREF( old );
}
else
{
DICT_SET_ITEM( dict, (PyObject *)key, value );
Py_DECREF( value );
}
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/helper/indexes.hpp 0000644 0001750 0001750 00000003006 12265264105 024301 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_HELPER_INDEXES_H__
#define __NUITKA_HELPER_INDEXES_H__
#include "nuitka/helper/raising.hpp"
NUITKA_MAY_BE_UNUSED static Py_ssize_t CONVERT_TO_INDEX( PyObject *value )
{
assertObject( value );
#if PYTHON_VERSION < 300
if ( PyInt_Check( value ) )
{
return PyInt_AS_LONG( value );
}
else
#endif
if ( PyIndex_Check( value ) )
{
Py_ssize_t result = PyNumber_AsSsize_t( value, NULL );
if (unlikely( result == -1 ))
{
THROW_IF_ERROR_OCCURED();
}
return result;
}
else
{
PyErr_Format( PyExc_TypeError, "slice indices must be integers or None or have an __index__ method" );
throw PythonException();
}
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/helper/operations.hpp 0000644 0001750 0001750 00000045030 12265264105 025030 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_OPERATIONS_H__
#define __NUITKA_OPERATIONS_H__
#if PYTHON_VERSION < 300
#define NEW_STYLE_NUMBER( o ) PyType_HasFeature( Py_TYPE( o ), Py_TPFLAGS_CHECKTYPES )
#else
#define NEW_STYLE_NUMBER( o ) (true)
#endif
typedef PyObject *(unary_api)( PyObject * );
NUITKA_MAY_BE_UNUSED static PyObject *UNARY_OPERATION( unary_api api, PyObject *operand )
{
PyObject *result = api( operand );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
typedef PyObject *(binary_api)( PyObject *, PyObject * );
NUITKA_MAY_BE_UNUSED static PyObject *BINARY_OPERATION( binary_api api, PyObject *operand1, PyObject *operand2 )
{
assertObject( operand1 );
assertObject( operand2 );
PyObject *result = api( operand1, operand2 );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *BINARY_OPERATION_ADD( PyObject *operand1, PyObject *operand2 )
{
assertObject( operand1 );
assertObject( operand2 );
binaryfunc slot1 = NULL;
binaryfunc slot2 = NULL;
PyTypeObject *type1 = Py_TYPE( operand1 );
PyTypeObject *type2 = Py_TYPE( operand2 );
if ( type1->tp_as_number != NULL && NEW_STYLE_NUMBER( operand1 ) )
{
slot1 = type1->tp_as_number->nb_add;
}
if ( type1 != type2 )
{
if ( type2->tp_as_number != NULL && NEW_STYLE_NUMBER( operand2 ) )
{
slot2 = type2->tp_as_number->nb_add;
if ( slot1 == slot2 )
{
slot2 = NULL;
}
}
}
if ( slot1 != NULL )
{
if ( slot2 && PyType_IsSubtype( type2, type1 ) )
{
PyObject *x = slot2( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
slot2 = NULL;
}
PyObject *x = slot1( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
}
if ( slot2 != NULL )
{
PyObject *x = slot2( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
}
#if PYTHON_VERSION < 300
if ( !NEW_STYLE_NUMBER( operand1 ) || !NEW_STYLE_NUMBER( operand2 ) )
{
int err = PyNumber_CoerceEx( &operand1, &operand2 );
if ( err < 0 )
{
throw PythonException();
}
if ( err == 0 )
{
PyNumberMethods *mv = Py_TYPE( operand1 )->tp_as_number;
if ( mv )
{
binaryfunc slot = mv->nb_add;
if ( slot != NULL )
{
PyObject *x = slot( operand1, operand2 );
Py_DECREF( operand1 );
Py_DECREF( operand2 );
if ( x == NULL )
{
throw PythonException();
}
return x;
}
}
// CoerceEx did that
Py_DECREF( operand1 );
Py_DECREF( operand2 );
}
}
#endif
// Special case for "+", also works as sequence concat.
PySequenceMethods *seq_methods = Py_TYPE( operand1 )->tp_as_sequence;
if ( seq_methods && seq_methods->sq_concat )
{
PyObject *result = (*seq_methods->sq_concat)( operand1, operand2 );
if ( result == NULL )
{
throw PythonException();
}
return result;
}
PyErr_Format(
PyExc_TypeError,
"unsupported operand type(s) for +: '%s' and '%s'",
type1->tp_name,
type2->tp_name
);
throw PythonException();
}
static PyObject *SEQUENCE_REPEAT( ssizeargfunc repeatfunc, PyObject *seq, PyObject *n )
{
if (unlikely( !PyIndex_Check( n ) ))
{
PyErr_Format(
PyExc_TypeError,
"can't multiply sequence by non-int of type '%s'",
Py_TYPE( n )->tp_name
);
throw PythonException();
}
PyObject *index_value = PyNumber_Index( n );
if (unlikely( index_value == NULL ))
{
throw PythonException();
}
/* We're done if PyInt_AsSsize_t() returns without error. */
#if PYTHON_VERSION < 300
Py_ssize_t count = PyInt_AsSsize_t( index_value );
#else
Py_ssize_t count = PyLong_AsSsize_t( index_value );
#endif
Py_DECREF( index_value );
if (unlikely( count == -1 )) // Note: -1 is an unlikely repetition count
{
PyObject *exception = GET_ERROR_OCCURED();
if (unlikely( exception ))
{
if ( !PyErr_GivenExceptionMatches( exception, PyExc_OverflowError ) )
{
throw PythonException();
}
PyErr_Format(
PyExc_OverflowError,
"cannot fit '%s' into an index-sized integer",
Py_TYPE( n )->tp_name
);
throw PythonException();
}
}
PyObject *result = (*repeatfunc)( seq, count );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *BINARY_OPERATION_MUL( PyObject *operand1, PyObject *operand2 )
{
assertObject( operand1 );
assertObject( operand2 );
binaryfunc slot1 = NULL;
binaryfunc slot2 = NULL;
PyTypeObject *type1 = Py_TYPE( operand1 );
PyTypeObject *type2 = Py_TYPE( operand2 );
if ( type1->tp_as_number != NULL && NEW_STYLE_NUMBER( operand1 ) )
{
slot1 = type1->tp_as_number->nb_multiply;
}
if ( type1 != type2 )
{
if ( type2->tp_as_number != NULL && NEW_STYLE_NUMBER( operand2 ) )
{
slot2 = type2->tp_as_number->nb_multiply;
if ( slot1 == slot2 )
{
slot2 = NULL;
}
}
}
if ( slot1 != NULL )
{
if ( slot2 && PyType_IsSubtype( type2, type1 ) )
{
PyObject *x = slot2( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
slot2 = NULL;
}
PyObject *x = slot1( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
}
if ( slot2 != NULL )
{
PyObject *x = slot2( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
}
#if PYTHON_VERSION < 300
if ( !NEW_STYLE_NUMBER( operand1 ) || !NEW_STYLE_NUMBER( operand2 ) )
{
int err = PyNumber_CoerceEx( &operand1, &operand2 );
if ( err < 0 )
{
throw PythonException();
}
if ( err == 0 )
{
PyNumberMethods *mv = Py_TYPE( operand1 )->tp_as_number;
if ( mv )
{
binaryfunc slot = mv->nb_multiply;
if ( slot != NULL )
{
PyObject *x = slot( operand1, operand2 );
Py_DECREF( operand1 );
Py_DECREF( operand2 );
if ( x == NULL )
{
throw PythonException();
}
return x;
}
}
// CoerceEx did that
Py_DECREF( operand1 );
Py_DECREF( operand2 );
}
}
#endif
// Special case for "+", also works as sequence concat.
PySequenceMethods *seq_methods1 = Py_TYPE( operand1 )->tp_as_sequence;
PySequenceMethods *seq_methods2 = Py_TYPE( operand2 )->tp_as_sequence;
if ( seq_methods1 != NULL && seq_methods1->sq_repeat )
{
return SEQUENCE_REPEAT( seq_methods1->sq_repeat, operand1, operand2 );
}
if ( seq_methods2 != NULL && seq_methods2->sq_repeat )
{
return SEQUENCE_REPEAT( seq_methods2->sq_repeat, operand2, operand1 );
}
PyErr_Format(
PyExc_TypeError,
"unsupported operand type(s) for *: '%s' and '%s'",
type1->tp_name,
type2->tp_name
);
throw PythonException();
}
NUITKA_MAY_BE_UNUSED static PyObject *BINARY_OPERATION_SUB( PyObject *operand1, PyObject *operand2 )
{
assertObject( operand1 );
assertObject( operand2 );
binaryfunc slot1 = NULL;
binaryfunc slot2 = NULL;
PyTypeObject *type1 = Py_TYPE( operand1 );
PyTypeObject *type2 = Py_TYPE( operand2 );
if ( type1->tp_as_number != NULL && NEW_STYLE_NUMBER( operand1 ) )
{
slot1 = type1->tp_as_number->nb_subtract;
}
if ( type1 != type2 )
{
if ( type2->tp_as_number != NULL && NEW_STYLE_NUMBER( operand2 ) )
{
slot2 = type2->tp_as_number->nb_subtract;
if ( slot1 == slot2 )
{
slot2 = NULL;
}
}
}
if ( slot1 != NULL )
{
if ( slot2 && PyType_IsSubtype( type2, type1 ) )
{
PyObject *x = slot2( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
slot2 = NULL;
}
PyObject *x = slot1( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
}
if ( slot2 != NULL )
{
PyObject *x = slot2( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
}
#if PYTHON_VERSION < 300
if ( !NEW_STYLE_NUMBER( operand1 ) || !NEW_STYLE_NUMBER( operand2 ) )
{
int err = PyNumber_CoerceEx( &operand1, &operand2 );
if ( err < 0 )
{
throw PythonException();
}
if ( err == 0 )
{
PyNumberMethods *mv = Py_TYPE( operand1 )->tp_as_number;
if ( mv )
{
binaryfunc slot = mv->nb_subtract;
if ( slot != NULL )
{
PyObject *x = slot( operand1, operand2 );
Py_DECREF( operand1 );
Py_DECREF( operand2 );
if ( x == NULL )
{
throw PythonException();
}
return x;
}
}
// CoerceEx did that
Py_DECREF( operand1 );
Py_DECREF( operand2 );
}
}
#endif
PyErr_Format(
PyExc_TypeError,
"unsupported operand type(s) for -: '%s' and '%s'",
type1->tp_name,
type2->tp_name
);
throw PythonException();
}
#if PYTHON_VERSION < 300
NUITKA_MAY_BE_UNUSED static PyObject *BINARY_OPERATION_DIV( PyObject *operand1, PyObject *operand2 )
{
assertObject( operand1 );
assertObject( operand2 );
binaryfunc slot1 = NULL;
binaryfunc slot2 = NULL;
PyTypeObject *type1 = Py_TYPE( operand1 );
PyTypeObject *type2 = Py_TYPE( operand2 );
if ( type1->tp_as_number != NULL && NEW_STYLE_NUMBER( operand1 ) )
{
slot1 = type1->tp_as_number->nb_divide;
}
if ( type1 != type2 )
{
if ( type2->tp_as_number != NULL && NEW_STYLE_NUMBER( operand2 ) )
{
slot2 = type2->tp_as_number->nb_divide;
if ( slot1 == slot2 )
{
slot2 = NULL;
}
}
}
if ( slot1 != NULL )
{
if ( slot2 && PyType_IsSubtype( type2, type1 ) )
{
PyObject *x = slot2( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
slot2 = NULL;
}
PyObject *x = slot1( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
}
if ( slot2 != NULL )
{
PyObject *x = slot2( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
}
#if PYTHON_VERSION < 300
if ( !NEW_STYLE_NUMBER( operand1 ) || !NEW_STYLE_NUMBER( operand2 ) )
{
int err = PyNumber_CoerceEx( &operand1, &operand2 );
if ( err < 0 )
{
throw PythonException();
}
if ( err == 0 )
{
PyNumberMethods *mv = Py_TYPE( operand1 )->tp_as_number;
if ( mv )
{
binaryfunc slot = mv->nb_divide;
if ( slot != NULL )
{
PyObject *x = slot( operand1, operand2 );
Py_DECREF( operand1 );
Py_DECREF( operand2 );
if ( x == NULL )
{
throw PythonException();
}
return x;
}
}
// CoerceEx did that
Py_DECREF( operand1 );
Py_DECREF( operand2 );
}
}
#endif
PyErr_Format(
PyExc_TypeError,
"unsupported operand type(s) for /: '%s' and '%s'",
type1->tp_name,
type2->tp_name
);
throw PythonException();
}
#endif
NUITKA_MAY_BE_UNUSED static PyObject *BINARY_OPERATION_REMAINDER( PyObject *operand1, PyObject *operand2 )
{
assertObject( operand1 );
assertObject( operand2 );
binaryfunc slot1 = NULL;
binaryfunc slot2 = NULL;
PyTypeObject *type1 = Py_TYPE( operand1 );
PyTypeObject *type2 = Py_TYPE( operand2 );
if ( type1->tp_as_number != NULL && NEW_STYLE_NUMBER( operand1 ) )
{
slot1 = type1->tp_as_number->nb_remainder;
}
if ( type1 != type2 )
{
if ( type2->tp_as_number != NULL && NEW_STYLE_NUMBER( operand2 ) )
{
slot2 = type2->tp_as_number->nb_remainder;
if ( slot1 == slot2 )
{
slot2 = NULL;
}
}
}
if ( slot1 != NULL )
{
if ( slot2 && PyType_IsSubtype( type2, type1 ) )
{
PyObject *x = slot2( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
slot2 = NULL;
}
PyObject *x = slot1( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
}
if ( slot2 != NULL )
{
PyObject *x = slot2( operand1, operand2 );
if ( x != Py_NotImplemented )
{
if ( x == 0 )
{
throw PythonException();
}
return x;
}
Py_DECREF( x );
}
#if PYTHON_VERSION < 300
if ( !NEW_STYLE_NUMBER( operand1 ) || !NEW_STYLE_NUMBER( operand2 ) )
{
int err = PyNumber_CoerceEx( &operand1, &operand2 );
if ( err < 0 )
{
throw PythonException();
}
if ( err == 0 )
{
PyNumberMethods *mv = Py_TYPE( operand1 )->tp_as_number;
if ( mv )
{
binaryfunc slot = mv->nb_remainder;
if ( slot != NULL )
{
PyObject *x = slot( operand1, operand2 );
Py_DECREF( operand1 );
Py_DECREF( operand2 );
if ( x == NULL )
{
throw PythonException();
}
return x;
}
}
// CoerceEx did that
Py_DECREF( operand1 );
Py_DECREF( operand2 );
}
}
#endif
PyErr_Format(
PyExc_TypeError,
"unsupported operand type(s) for %%: '%s' and '%s'",
type1->tp_name,
type2->tp_name
);
throw PythonException();
}
NUITKA_MAY_BE_UNUSED static PyObject *POWER_OPERATION( PyObject *operand1, PyObject *operand2 )
{
PyObject *result = PyNumber_Power( operand1, operand2, Py_None );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *POWER_OPERATION_INPLACE( PyObject *operand1, PyObject *operand2 )
{
PyObject *result = PyNumber_InPlacePower( operand1, operand2, Py_None );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/helper/attributes.hpp 0000644 0001750 0001750 00000037606 12265264105 025045 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_HELPER_ATTRIBUTES_H__
#define __NUITKA_HELPER_ATTRIBUTES_H__
#if PYTHON_VERSION < 300
NUITKA_MAY_BE_UNUSED static PyObject *FIND_ATTRIBUTE_IN_CLASS( PyClassObject *klass, PyObject *attr_name )
{
PyObject *result = GET_STRING_DICT_VALUE( (PyDictObject *)klass->cl_dict, (PyStringObject *)attr_name );
if ( result == NULL )
{
Py_ssize_t base_count = PyTuple_Size( klass->cl_bases );
for ( Py_ssize_t i = 0; i < base_count; i++ )
{
result = FIND_ATTRIBUTE_IN_CLASS( (PyClassObject *)PyTuple_GetItem( klass->cl_bases, i ), attr_name );
if ( result )
{
break;
}
}
}
return result;
}
#endif
#if PYTHON_VERSION < 300
extern PyObject *CALL_FUNCTION_WITH_ARGS2( PyObject *called, PyObject *arg1, PyObject *arg2 );
static PyObject *LOOKUP_INSTANCE( PyObject *source, PyObject *attr_name )
{
assertObject( source );
assertObject( attr_name );
assert( PyInstance_Check( source ) );
assert( PyString_Check( attr_name ) );
PyInstanceObject *source_instance = (PyInstanceObject *)source;
// TODO: The special cases should get their own SET_ATTRIBUTE variant on the
// code generation level as SET_ATTRIBUTE is called with constants only.
assert( attr_name != const_str_plain___dict__ );
assert( attr_name != const_str_plain___class__ );
// Try the instance dict first.
PyObject *result = GET_STRING_DICT_VALUE( (PyDictObject *)source_instance->in_dict, (PyStringObject *)attr_name );
if ( result )
{
return INCREASE_REFCOUNT( result );
}
// Next see if a class has it
result = FIND_ATTRIBUTE_IN_CLASS( source_instance->in_class, attr_name );
if ( result )
{
descrgetfunc func = Py_TYPE( result )->tp_descr_get;
if ( func )
{
result = func( result, source, (PyObject *)source_instance->in_class );
if (unlikely( result == NULL ))
{
throw PythonException();
}
assertObject( result );
return result;
}
else
{
return INCREASE_REFCOUNT( result );
}
}
THROW_IF_ERROR_OCCURED_NOT( PyExc_AttributeError );
// Finally allow a __getattr__ to handle it or else it's an error.
if (unlikely( source_instance->in_class->cl_getattr == NULL ))
{
PyErr_Format( PyExc_AttributeError, "%s instance has no attribute '%s'", PyString_AS_STRING( source_instance->in_class->cl_name ), PyString_AS_STRING( attr_name ) );
throw PythonException();
}
else
{
return CALL_FUNCTION_WITH_ARGS2(
source_instance->in_class->cl_getattr,
source,
attr_name
);
}
}
#endif
NUITKA_MAY_BE_UNUSED static PyObject *LOOKUP_ATTRIBUTE( PyObject *source, PyObject *attr_name )
{
assertObject( source );
assertObject( attr_name );
#if PYTHON_VERSION < 300
if ( PyInstance_Check( source ) )
{
PyObject *result = LOOKUP_INSTANCE( source, attr_name );
assertObject( result );
return result;
}
else
#endif
{
PyTypeObject *type = Py_TYPE( source );
if ( type->tp_getattro != NULL )
{
PyObject *result = (*type->tp_getattro)( source, attr_name );
if (unlikely( result == NULL ))
{
throw PythonException();
}
assertObject( result );
return result;
}
else if ( type->tp_getattr != NULL )
{
PyObject *result = (*type->tp_getattr)( source, Nuitka_String_AsString_Unchecked( attr_name ) );
if (unlikely( result == NULL ))
{
throw PythonException();
}
assertObject( result );
return result;
}
else
{
PyErr_Format( PyExc_AttributeError, "'%s' object has no attribute '%s'", type->tp_name, Nuitka_String_AsString_Unchecked( attr_name ) );
throw PythonException();
}
}
}
NUITKA_MAY_BE_UNUSED static PyObject *LOOKUP_ATTRIBUTE_DICT_SLOT( PyObject *source )
{
assertObject( source );
#if PYTHON_VERSION < 300
if (likely( PyInstance_Check( source ) ))
{
PyInstanceObject *source_instance = (PyInstanceObject *)source;
return INCREASE_REFCOUNT( source_instance->in_dict );
}
else
#endif
{
PyTypeObject *type = Py_TYPE( source );
if ( type->tp_getattro != NULL )
{
PyObject *result = (*type->tp_getattro)( source, const_str_plain___dict__ );
if (unlikely( result == NULL ))
{
throw PythonException();
}
assertObject( result );
return result;
}
else if ( type->tp_getattr != NULL )
{
PyObject *result = (*type->tp_getattr)( source, (char *)"__dict__" );
if (unlikely( result == NULL ))
{
throw PythonException();
}
assertObject( result );
return result;
}
else
{
PyErr_Format( PyExc_AttributeError, "'%s' object has no attribute '__dict__'", type->tp_name );
throw PythonException();
}
}
}
NUITKA_MAY_BE_UNUSED static PyObject *LOOKUP_ATTRIBUTE_CLASS_SLOT( PyObject *source )
{
assertObject( source );
#if PYTHON_VERSION < 300
if (likely( PyInstance_Check( source ) ))
{
PyInstanceObject *source_instance = (PyInstanceObject *)source;
return INCREASE_REFCOUNT( (PyObject *)source_instance->in_class );
}
else
#endif
{
PyTypeObject *type = Py_TYPE( source );
if ( type->tp_getattro != NULL )
{
PyObject *result = (*type->tp_getattro)( source, const_str_plain___class__ );
if (unlikely( result == NULL ))
{
throw PythonException();
}
assertObject( result );
return result;
}
else if ( type->tp_getattr != NULL )
{
PyObject *result = (*type->tp_getattr)( source, (char *)"__class__" );
if (unlikely( result == NULL ))
{
throw PythonException();
}
assertObject( result );
return result;
}
else
{
PyErr_Format( PyExc_AttributeError, "'%s' object has no attribute '__class__'", type->tp_name );
throw PythonException();
}
}
}
NUITKA_MAY_BE_UNUSED static bool HAS_ATTRIBUTE( PyObject *source, PyObject *attr_name )
{
assertObject( source );
assertObject( attr_name );
int res = PyObject_HasAttr( source, attr_name );
if (unlikely( res == -1 ))
{
throw PythonException();
}
return res == 1;
}
#if PYTHON_VERSION < 300
extern PyObject *CALL_FUNCTION_WITH_ARGS3( PyObject *called, PyObject *arg1, PyObject *arg2, PyObject *arg3 );
static void SET_INSTANCE( PyObject *target, PyObject *attr_name, PyObject *value )
{
assertObject( target );
assertObject( attr_name );
assertObject( value );
assert( PyInstance_Check( target ) );
assert( PyString_Check( attr_name ) );
PyInstanceObject *target_instance = (PyInstanceObject *)target;
// The special cases should get their own SET_ATTRIBUTE_xxxx_SLOT variants
// on the code generation level as SET_ATTRIBUTE is called with constants
// only.
assert( attr_name != const_str_plain___dict__ );
assert( attr_name != const_str_plain___class__ );
if ( target_instance->in_class->cl_setattr != NULL )
{
PyObject *result = CALL_FUNCTION_WITH_ARGS3(
target_instance->in_class->cl_setattr,
target, attr_name, value
);
Py_DECREF( result );
}
else
{
int status = PyDict_SetItem(
target_instance->in_dict,
attr_name,
value
);
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
}
#endif
NUITKA_MAY_BE_UNUSED static void SET_ATTRIBUTE( PyObject *value, PyObject *target, PyObject *attr_name )
{
assertObject( target );
assertObject( attr_name );
assertObject( value );
#if PYTHON_VERSION < 300
if ( PyInstance_Check( target ) )
{
SET_INSTANCE( target, attr_name, value );
}
else
#endif
{
PyTypeObject *type = Py_TYPE( target );
if ( type->tp_setattro != NULL )
{
int status = (*type->tp_setattro)( target, attr_name, value );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
else if ( type->tp_setattr != NULL )
{
int status = (*type->tp_setattr)( target, Nuitka_String_AsString_Unchecked( attr_name ), value );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
else if ( type->tp_getattr == NULL && type->tp_getattro == NULL )
{
PyErr_Format(
PyExc_TypeError,
"'%s' object has no attributes (assign to %s)",
type->tp_name,
Nuitka_String_AsString_Unchecked( attr_name )
);
throw PythonException();
}
else
{
PyErr_Format(
PyExc_TypeError,
"'%s' object has only read-only attributes (assign to %s)",
type->tp_name,
Nuitka_String_AsString_Unchecked( attr_name )
);
throw PythonException();
}
}
}
NUITKA_MAY_BE_UNUSED static void SET_ATTRIBUTE_DICT_SLOT( PyObject *value, PyObject *target )
{
assertObject( target );
assertObject( value );
#if PYTHON_VERSION < 300
if ( likely( PyInstance_Check( target ) ))
{
PyInstanceObject *target_instance = (PyInstanceObject *)target;
if (unlikely( !PyDict_Check( value ) ))
{
PyErr_SetString( PyExc_TypeError, "__dict__ must be set to a dictionary" );
throw PythonException();
}
PyObject *old = target_instance->in_dict;
target_instance->in_dict = INCREASE_REFCOUNT( value );
Py_DECREF( old );
}
else
#endif
{
PyTypeObject *type = Py_TYPE( target );
if ( type->tp_setattro != NULL )
{
int status = (*type->tp_setattro)( target, const_str_plain___dict__, value );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
else if ( type->tp_setattr != NULL )
{
int status = (*type->tp_setattr)( target, (char *)"__dict__", value );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
else if ( type->tp_getattr == NULL && type->tp_getattro == NULL )
{
PyErr_Format(
PyExc_TypeError,
"'%s' object has no attributes (assign to __dict__)",
type->tp_name
);
throw PythonException();
}
else
{
PyErr_Format(
PyExc_TypeError,
"'%s' object has only read-only attributes (assign to __dict__)",
type->tp_name
);
throw PythonException();
}
}
}
NUITKA_MAY_BE_UNUSED static void SET_ATTRIBUTE_CLASS_SLOT( PyObject *value, PyObject *target )
{
assertObject( target );
assertObject( value );
#if PYTHON_VERSION < 300
if (likely( PyInstance_Check( target ) ))
{
PyInstanceObject *target_instance = (PyInstanceObject *)target;
if (unlikely( !PyClass_Check( value ) ))
{
PyErr_SetString( PyExc_TypeError, "__class__ must be set to a class" );
throw PythonException();
}
PyObject *old = (PyObject *)( target_instance->in_class );
target_instance->in_class = (PyClassObject *)INCREASE_REFCOUNT( value );
Py_DECREF( old );
}
else
#endif
{
PyTypeObject *type = Py_TYPE( target );
if ( type->tp_setattro != NULL )
{
int status = (*type->tp_setattro)( target, const_str_plain___class__, value );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
else if ( type->tp_setattr != NULL )
{
int status = (*type->tp_setattr)( target, (char *)"__class__", value );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
else if ( type->tp_getattr == NULL && type->tp_getattro == NULL )
{
PyErr_Format(
PyExc_TypeError,
"'%s' object has no attributes (assign to __class__)",
type->tp_name
);
throw PythonException();
}
else
{
PyErr_Format(
PyExc_TypeError,
"'%s' object has only read-only attributes (assign to __class__)",
type->tp_name
);
throw PythonException();
}
}
}
NUITKA_MAY_BE_UNUSED static void DEL_ATTRIBUTE( PyObject *target, PyObject *attr_name )
{
assertObject( target );
assertObject( attr_name );
int status = PyObject_DelAttr( target, attr_name );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
NUITKA_MAY_BE_UNUSED static PyObject *LOOKUP_SPECIAL( PyObject *source, PyObject *attr_name )
{
#if PYTHON_VERSION < 300
if ( PyInstance_Check( source ) )
{
return LOOKUP_INSTANCE( source, attr_name );
}
#endif
// TODO: There is heavy optimization in CPython to avoid it. Potentially that's worth
// it to imitate that.
PyObject *result = _PyType_Lookup( Py_TYPE( source ), attr_name );
if (likely( result ))
{
descrgetfunc func = Py_TYPE( result )->tp_descr_get;
if ( func == NULL )
{
return INCREASE_REFCOUNT( result );
}
else
{
PyObject *func_result = func( result, source, (PyObject *)( Py_TYPE( source ) ) );
if (unlikely( func_result == NULL ))
{
throw PythonException();
}
return func_result;
}
}
PyErr_SetObject( PyExc_AttributeError, attr_name );
throw PythonException();
}
// Necessary to abstract the with statement lookup difference between
// pre-Python2.7 and others. Since Python 2.7 the code does no full attribute
// lookup anymore, but instead treats enter and exit as specials.
NUITKA_MAY_BE_UNUSED static inline PyObject *LOOKUP_WITH_ENTER( PyObject *source )
{
#if PYTHON_VERSION < 270
return LOOKUP_ATTRIBUTE( source, const_str_plain___enter__ );
#else
return LOOKUP_SPECIAL( source, const_str_plain___enter__ );
#endif
}
NUITKA_MAY_BE_UNUSED static inline PyObject *LOOKUP_WITH_EXIT( PyObject *source )
{
#if PYTHON_VERSION < 270
return LOOKUP_ATTRIBUTE( source, const_str_plain___exit__ );
#else
return LOOKUP_SPECIAL( source, const_str_plain___exit__ );
#endif
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/helper/raising.hpp 0000644 0001750 0001750 00000031144 12265264105 024302 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_HELPER_RAISING_H__
#define __NUITKA_HELPER_RAISING_H__
#if PYTHON_VERSION < 300
#define WRONG_EXCEPTION_TYPE_ERROR_MESSAGE "exceptions must be old-style classes or derived from BaseException, not %s"
#else
#define WRONG_EXCEPTION_TYPE_ERROR_MESSAGE "exceptions must derive from BaseException"
#endif
#if PYTHON_VERSION >= 300
static void CHAIN_EXCEPTION( PyObject *exception_type, PyObject *exception_value )
{
// Implicit chain of exception already existing.
PyThreadState *thread_state = PyThreadState_GET();
// Normalize existing exception first.
PyErr_NormalizeException( &thread_state->exc_type, &thread_state->exc_value, &thread_state->exc_traceback );
PyObject *old_exc_value = thread_state->exc_value;
if ( old_exc_value != NULL && old_exc_value != Py_None && old_exc_value != exception_value )
{
Py_INCREF( old_exc_value );
PyObject *o = old_exc_value, *context;
while (( context = PyException_GetContext(o) ))
{
Py_DECREF( context );
if ( context == exception_value )
{
PyException_SetContext( o, NULL );
break;
}
o = context;
}
PyException_SetContext( exception_value, old_exc_value );
PyException_SetTraceback( old_exc_value, thread_state->exc_traceback );
}
}
#endif
NUITKA_NO_RETURN NUITKA_MAY_BE_UNUSED static void RAISE_EXCEPTION_WITH_TYPE( PyObject *exception_type, PyObject *exception_tb )
{
PyTracebackObject *traceback = (PyTracebackObject *)exception_tb;
assertObject( traceback );
assert( PyTraceBack_Check( traceback ) );
assertObject( exception_type );
if ( PyExceptionClass_Check( exception_type ) )
{
PyObject *value = NULL;
Py_INCREF( exception_type );
Py_XINCREF( traceback );
NORMALIZE_EXCEPTION( &exception_type, &value, &traceback );
#if PYTHON_VERSION >= 270
if (unlikely( !PyExceptionInstance_Check( value ) ))
{
Py_DECREF( exception_type );
Py_DECREF( value );
Py_XDECREF( traceback );
PyErr_Format(
PyExc_TypeError,
"calling %s() should have returned an instance of BaseException, not '%s'",
((PyTypeObject *)exception_type)->tp_name,
Py_TYPE( value )->tp_name
);
throw PythonException();
}
#endif
#if PYTHON_VERSION >= 300
CHAIN_EXCEPTION( exception_type, value );
#endif
throw PythonException(
exception_type,
value,
traceback
);
}
else if ( PyExceptionInstance_Check( exception_type ) )
{
PyObject *value = exception_type;
exception_type = PyExceptionInstance_Class( exception_type );
#if PYTHON_VERSION >= 300
CHAIN_EXCEPTION( exception_type, value );
PyTracebackObject *prev = (PyTracebackObject *)PyException_GetTraceback( value );
if ( prev != NULL )
{
assert( traceback->tb_next == NULL );
traceback->tb_next = prev;
}
PyException_SetTraceback( value, (PyObject *)traceback );
#endif
throw PythonException(
INCREASE_REFCOUNT( exception_type ),
INCREASE_REFCOUNT( value ),
INCREASE_REFCOUNT( traceback )
);
}
else
{
PyErr_Format( PyExc_TypeError, WRONG_EXCEPTION_TYPE_ERROR_MESSAGE, Py_TYPE( exception_type )->tp_name );
PythonException to_throw;
to_throw.setTraceback( INCREASE_REFCOUNT( traceback ) );
throw to_throw;
}
}
#if PYTHON_VERSION >= 300
NUITKA_NO_RETURN NUITKA_MAY_BE_UNUSED static void RAISE_EXCEPTION_WITH_CAUSE( PyObject *exception_type, PyObject *exception_cause, PyObject *exception_tb )
{
PyTracebackObject *traceback = (PyTracebackObject *)exception_tb;
assertObject( exception_type );
assertObject( exception_cause );
#if PYTHON_VERSION >= 330
// None is not a cause.
if ( exception_cause == Py_None )
{
exception_cause = NULL;
}
else
#endif
if ( PyExceptionClass_Check( exception_cause ) )
{
exception_cause = PyObject_CallObject( exception_cause, NULL );
if (unlikely( exception_cause == NULL ))
{
throw PythonException();
}
}
else
{
Py_INCREF( exception_cause );
}
#if PYTHON_VERSION >= 330
if (unlikely( exception_cause != NULL && !PyExceptionInstance_Check( exception_cause ) ))
#else
if (unlikely( !PyExceptionInstance_Check( exception_cause ) ))
#endif
{
Py_XDECREF( exception_cause );
PyErr_Format( PyExc_TypeError, "exception causes must derive from BaseException" );
throw PythonException();
}
if ( PyExceptionClass_Check( exception_type ) )
{
PyObject *value = NULL;
Py_INCREF( exception_type );
Py_INCREF( traceback );
NORMALIZE_EXCEPTION( &exception_type, &value, &traceback );
if (unlikely( !PyExceptionInstance_Check( value ) ))
{
Py_DECREF( exception_type );
Py_XDECREF( value );
Py_DECREF( traceback );
Py_XDECREF( exception_cause );
PyErr_Format(
PyExc_TypeError,
"calling %s() should have returned an instance of BaseException, not '%s'",
((PyTypeObject *)exception_type)->tp_name,
Py_TYPE( value )->tp_name
);
throw PythonException();
}
#if PYTHON_VERSION >= 300
CHAIN_EXCEPTION( exception_type, value );
#endif
PythonException to_throw( exception_type, value, traceback );
to_throw.setCause( exception_cause );
throw to_throw;
}
else if ( PyExceptionInstance_Check( exception_type ) )
{
PyObject *value = exception_type;
exception_type = PyExceptionInstance_Class( value );
#if PYTHON_VERSION >= 300
CHAIN_EXCEPTION( exception_type, value );
#endif
PythonException to_throw(
INCREASE_REFCOUNT( exception_type ),
INCREASE_REFCOUNT( value ),
INCREASE_REFCOUNT( traceback )
);
to_throw.setCause( exception_cause );
throw to_throw;
}
else
{
Py_XDECREF( exception_cause );
PyErr_Format( PyExc_TypeError, WRONG_EXCEPTION_TYPE_ERROR_MESSAGE, Py_TYPE( exception_type )->tp_name );
PythonException to_throw;
to_throw.setTraceback( INCREASE_REFCOUNT( traceback ) );
throw to_throw;
}
}
#endif
NUITKA_NO_RETURN NUITKA_MAY_BE_UNUSED static void RAISE_EXCEPTION_WITH_VALUE( PyObject *exception_type, PyObject *value, PyObject *exception_tb )
{
assertObject( exception_type );
PyTracebackObject *traceback = (PyTracebackObject *)exception_tb;
// Non-empty tuple exceptions are the first element.
while (unlikely( PyTuple_Check( exception_type ) && PyTuple_Size( exception_type ) > 0 ))
{
exception_type = PyTuple_GET_ITEM( exception_type, 0 );
}
if ( PyExceptionClass_Check( exception_type ) )
{
Py_INCREF( exception_type );
Py_XINCREF( value );
Py_XINCREF( traceback );
NORMALIZE_EXCEPTION( &exception_type, &value, &traceback );
#if PYTHON_VERSION >= 270
if (unlikely( !PyExceptionInstance_Check( value ) ))
{
PyErr_Format(
PyExc_TypeError,
"calling %s() should have returned an instance of BaseException, not '%s'",
((PyTypeObject *)exception_type)->tp_name,
Py_TYPE( value )->tp_name
);
Py_DECREF( exception_type );
Py_XDECREF( value );
Py_XDECREF( traceback );
throw PythonException();
}
#endif
throw PythonException(
exception_type,
value,
traceback
);
}
else if ( PyExceptionInstance_Check( exception_type ) )
{
if (unlikely( value != NULL && value != Py_None ))
{
PyErr_Format(
PyExc_TypeError,
"instance exception may not have a separate value"
);
throw PythonException();
}
// The type is rather a value, so we are overriding it here.
value = exception_type;
exception_type = PyExceptionInstance_Class( exception_type );
throw PythonException(
INCREASE_REFCOUNT( exception_type ),
INCREASE_REFCOUNT( value ),
INCREASE_REFCOUNT_X( traceback )
);
}
else
{
PyErr_Format( PyExc_TypeError, WRONG_EXCEPTION_TYPE_ERROR_MESSAGE, Py_TYPE( exception_type )->tp_name );
throw PythonException();
}
}
NUITKA_NO_RETURN NUITKA_MAY_BE_UNUSED static void RAISE_EXCEPTION_WITH_VALUE_NO_NORMALIZE( PyObject *exception_type, PyObject *value, PyObject *tb )
{
PyTracebackObject *traceback = (PyTracebackObject *)tb;
assertObject( exception_type );
assert( !PyTuple_Check( exception_type ) );
if ( PyExceptionClass_Check( exception_type ) )
{
throw PythonException(
INCREASE_REFCOUNT( exception_type ),
INCREASE_REFCOUNT( value ),
INCREASE_REFCOUNT_X( traceback )
);
}
else if ( PyExceptionInstance_Check( exception_type ) )
{
assert ( value == NULL || value == Py_None );
// The type is rather a value, so we are overriding it here.
value = exception_type;
exception_type = PyExceptionInstance_Class( exception_type );
throw PythonException(
INCREASE_REFCOUNT( exception_type ),
INCREASE_REFCOUNT( value ),
INCREASE_REFCOUNT_X( traceback )
);
}
else
{
assert( false );
PyErr_Format( PyExc_TypeError, WRONG_EXCEPTION_TYPE_ERROR_MESSAGE, Py_TYPE( exception_type )->tp_name );
throw PythonException();
}
}
NUITKA_NO_RETURN NUITKA_MAY_BE_UNUSED static inline void RAISE_EXCEPTION_WITH_TRACEBACK( PyObject *exception_type, PyObject *value, PyObject *traceback )
{
if ( traceback == Py_None )
{
traceback = NULL;
}
// Check traceback
if( traceback != NULL && !PyTraceBack_Check( traceback ) )
{
PyErr_Format( PyExc_TypeError, "raise: arg 3 must be a traceback or None" );
throw PythonException();
}
RAISE_EXCEPTION_WITH_VALUE( exception_type, value, traceback );
}
NUITKA_NO_RETURN NUITKA_MAY_BE_UNUSED static void RERAISE_EXCEPTION( void )
{
PyThreadState *tstate = PyThreadState_GET();
assert( tstate );
PyObject *type = tstate->exc_type != NULL ? tstate->exc_type : Py_None;
PyObject *value = tstate->exc_value;
PyObject *tb = tstate->exc_traceback;
assertObject( type );
#if PYTHON_VERSION >= 300
if ( type == Py_None )
{
PyErr_Format( PyExc_RuntimeError, "No active exception to reraise" );
throw PythonException();
}
#endif
RAISE_EXCEPTION_WITH_TRACEBACK( type, value, tb );
}
// Throw an exception from within an expression, this is without normalization. Note:
// There is also a form for use as a statement, and also doesn't do it, seeing this used
// normally means, the implicit exception was not propagated.
NUITKA_NO_RETURN NUITKA_MAY_BE_UNUSED static PyObject *THROW_EXCEPTION( PyObject *exception_type, PyObject *exception_value, PyObject *traceback )
{
RAISE_EXCEPTION_WITH_VALUE_NO_NORMALIZE(
exception_type,
exception_value,
traceback
);
}
NUITKA_MAY_BE_UNUSED static void THROW_IF_ERROR_OCCURED( void )
{
if ( ERROR_OCCURED() )
{
throw PythonException();
}
}
NUITKA_MAY_BE_UNUSED static void THROW_IF_ERROR_OCCURED_NOT( PyObject *ignored )
{
if ( ERROR_OCCURED() )
{
if ( PyErr_ExceptionMatches( ignored ))
{
PyErr_Clear();
}
else
{
throw PythonException();
}
}
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/helper/sequences.hpp 0000644 0001750 0001750 00000007206 12265264105 024643 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_HELPER_SEQUENCES_H__
#define __NUITKA_HELPER_SEQUENCES_H__
NUITKA_MAY_BE_UNUSED static PyObject *TO_LIST( PyObject *seq_obj )
{
PyObject *result = PySequence_List( seq_obj );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *TO_TUPLE( PyObject *seq_obj )
{
PyObject *result = PySequence_Tuple( seq_obj );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *TO_SET( PyObject *seq_obj )
{
PyObject *result = PySet_New( seq_obj );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *SEQUENCE_CONTAINS( PyObject *element, PyObject *sequence )
{
int result = PySequence_Contains( sequence, element );
if (unlikely( result == -1 ))
{
throw PythonException();
}
return BOOL_FROM( result == 1 );
}
NUITKA_MAY_BE_UNUSED static PyObject *SEQUENCE_CONTAINS_NOT( PyObject *element, PyObject *sequence )
{
int result = PySequence_Contains( sequence, element );
if (unlikely( result == -1 ))
{
throw PythonException();
}
return BOOL_FROM( result == 0 );
}
NUITKA_MAY_BE_UNUSED static bool SEQUENCE_CONTAINS_BOOL( PyObject *element, PyObject *sequence )
{
int result = PySequence_Contains( sequence, element );
if (unlikely( result == -1 ))
{
throw PythonException();
}
return result == 1;
}
NUITKA_MAY_BE_UNUSED static bool SEQUENCE_CONTAINS_NOT_BOOL( PyObject *element, PyObject *sequence )
{
int result = PySequence_Contains( sequence, element );
if (unlikely( result == -1 ))
{
throw PythonException();
}
return result == 0;
}
NUITKA_MAY_BE_UNUSED static void SEQUENCE_SETITEM( PyObject *sequence, Py_ssize_t index, PyObject *value )
{
assertObject( sequence );
assertObject( value );
PySequenceMethods *sequence_methods = Py_TYPE( sequence )->tp_as_sequence;
if ( sequence_methods != NULL && sequence_methods->sq_ass_item )
{
if ( index < 0 )
{
if ( sequence_methods->sq_length )
{
Py_ssize_t length = (*sequence_methods->sq_length)( sequence );
if ( length < 0 )
{
throw PythonException();
}
index += length;
}
}
int res = sequence_methods->sq_ass_item( sequence, index, value );
if (unlikely( res == -1 ))
{
throw PythonException();
}
}
else
{
PyErr_Format(
PyExc_TypeError,
"'%s' object does not support item assignment",
Py_TYPE( sequence )->tp_name
);
throw PythonException();
}
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/compiled_frame.hpp 0000644 0001750 0001750 00000003040 12265264105 024327 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_COMPILED_FRAME_H__
#define __NUITKA_COMPILED_FRAME_H__
// Create a frame object for the given code object and module
extern PyFrameObject *MAKE_FRAME( PyCodeObject *code, PyObject *module );
// Create a code object for the given filename and function name
#if PYTHON_VERSION < 300
extern PyCodeObject *MAKE_CODEOBJ( PyObject *filename, PyObject *function_name, int line, PyObject *argnames, int arg_count, int flags );
#else
extern PyCodeObject *MAKE_CODEOBJ( PyObject *filename, PyObject *function_name, int line, PyObject *argnames, int arg_count, int kw_only_count, int flags );
#endif
extern PyTypeObject Nuitka_Frame_Type;
static inline bool Nuitka_Frame_Check( PyObject *object )
{
return Py_TYPE( object ) == &Nuitka_Frame_Type;
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/unfreezing.hpp 0000644 0001750 0001750 00000003272 12265264105 023544 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_UNFREEZING_H__
#define __NUITKA_UNFREEZING_H__
// This define guards these definitions from being used without the unfreezing
// mode actually being active at all.
#define NUITKA_COMPILED_MODULE 0
#define NUITKA_SHLIB_MODULE 1
struct Nuitka_MetaPathBasedLoaderEntry
{
// Full module name, including package
char *name;
// Entry function if compiled module, otherwise NULL.
#if PYTHON_VERSION < 300
void (*python_initfunc)( void );
#else
PyObject * (*python_initfunc)( void );
#endif
// Flags: NUITKA_COMPILED_MODULE or NUITKA_SHLIB_MODULE
int flags;
};
// For embedded modules, to be unpacked. Used by main program/package only
extern void registerMetaPathBasedUnfreezer( struct Nuitka_MetaPathBasedLoaderEntry *loader_entries );
// For use as the "__loader__" attribute of compiled modules.
#if PYTHON_VERSION >= 330
extern PyObject *metapath_based_loader;
#endif
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/frame_guards.hpp 0000644 0001750 0001750 00000041240 12265264105 024024 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_FRAME_GUARDS_H__
#define __NUITKA_FRAME_GUARDS_H__
inline static void assertCodeObject( PyCodeObject *code_object )
{
assertObject( (PyObject *)code_object );
}
inline static void assertFrameObject( PyFrameObject *frame_object )
{
assertObject( (PyObject *)frame_object );
assertCodeObject( frame_object->f_code );
}
NUITKA_MAY_BE_UNUSED static PyFrameObject *INCREASE_REFCOUNT( PyFrameObject *frame_object )
{
assertFrameObject( frame_object );
Py_INCREF( frame_object );
return frame_object;
}
NUITKA_MAY_BE_UNUSED static PyFrameObject *INCREASE_REFCOUNT_X( PyFrameObject *frame_object )
{
Py_XINCREF( frame_object );
return frame_object;
}
NUITKA_MAY_BE_UNUSED static bool isFrameUnusable( PyFrameObject *frame_object )
{
return
// Never used.
frame_object == NULL ||
// Still in use
Py_REFCNT( frame_object ) > 1 ||
// Last used by another thread (TODO: Could just set it when re-using)
frame_object->f_tstate != PyThreadState_GET() ||
// Was detached from (TODO: When detaching, can't we just have another
// frame guard instead)
frame_object->f_back != NULL;
}
inline static void popFrameStack( void )
{
PyThreadState *tstate = PyThreadState_GET();
PyFrameObject *old = tstate->frame;
#if _DEBUG_FRAME
printf( "Taking off frame %s %s\n", PyString_AsString( PyObject_Str( (PyObject *)old ) ), PyString_AsString( PyObject_Repr( (PyObject *)old->f_code ) ) );
#endif
tstate->frame = old->f_back;
old->f_back = NULL;
// We might be very top level, e.g. in a thread, and therefore do not insist on value.
Py_XDECREF( tstate->frame );
#if _DEBUG_FRAME
printf( "Now at top frame %s %s\n", PyString_AsString( PyObject_Str( (PyObject *)tstate->frame ) ), PyString_AsString( PyObject_Repr( (PyObject *)tstate->frame->f_code ) ) );
#endif
}
inline static void pushFrameStack( PyFrameObject *frame_object )
{
assertFrameObject( frame_object );
PyThreadState *tstate = PyThreadState_GET();
// Look at current frame.
PyFrameObject *old = tstate->frame;
#if _DEBUG_FRAME
if ( old )
{
assertCodeObject( old->f_code );
printf( "Upstacking to frame %s %s\n", PyString_AsString( PyObject_Str( (PyObject *)old ) ), PyString_AsString( PyObject_Repr( (PyObject *)old->f_code ) ) );
}
#endif
// No recursion allowed of course, assert against it.
assert( old != frame_object );
// Push the new frame as the currently active one.
tstate->frame = frame_object;
// We don't allow touching cached frame objects where this is not true.
assert( frame_object->f_back == NULL );
if ( old != NULL )
{
assertFrameObject( old );
frame_object->f_back = INCREASE_REFCOUNT( old );
}
#if _DEBUG_FRAME
printf( "Now at top frame %s %s\n", PyString_AsString( PyObject_Str( (PyObject *)tstate->frame ) ), PyString_AsString( PyObject_Repr( (PyObject *)tstate->frame->f_code ) ) );
#endif
}
#if _DEBUG_REFRAME
static inline void dumpFrameStack( void )
{
PyFrameObject *current = PyThreadState_GET()->frame;
int total = 0;
while( current )
{
total++;
current = current->f_back;
}
current = PyThreadState_GET()->frame;
puts( ">--------->" );
while( current )
{
printf( "Frame stack %d: %s %s\n", total--, PyString_AsString( PyObject_Str( (PyObject *)current ) ), PyString_AsString( PyObject_Str( (PyObject *)current->f_code ) ) );
current = current->f_back;
}
puts( ">---------<" );
}
#endif
// Make a replacement for the current top frame, that we again own exclusively
// enough so that the line numbers are detached.
extern PyFrameObject *detachCurrentFrame();
class FrameGuard
{
public:
explicit FrameGuard( PyFrameObject *frame_object )
{
assertFrameObject( frame_object );
// Remember it.
this->frame_object = frame_object;
// Push the new frame as the currently active one.
pushFrameStack( frame_object );
// Keep the frame object alive for this C++ objects live time.
Py_INCREF( frame_object );
#if _DEBUG_REFRAME
// dumpFrameStack();
#endif
}
~FrameGuard()
{
// Our frame should be on top.
assert( PyThreadState_GET()->frame == this->frame_object );
// Put the previous frame on top instead.
popFrameStack();
assert( PyThreadState_GET()->frame != this->frame_object );
// Should still be good.
assertFrameObject( this->frame_object );
// Now release our frame object reference.
Py_DECREF( this->frame_object );
}
inline PyFrameObject *getFrame() const
{
return INCREASE_REFCOUNT( this->frame_object );
}
inline PyFrameObject *getFrame0() const
{
return this->frame_object;
}
// Use this to set the current line of the frame
inline void setLineNumber( int lineno ) const
{
assertFrameObject( this->frame_object );
assert( lineno >= 1 );
// Make sure f_lineno is the actually used information.
assert( this->frame_object->f_trace == Py_None );
this->frame_object->f_lineno = lineno;
}
inline int getLineNumber() const
{
assertFrameObject( this->frame_object );
return this->frame_object->f_lineno;
}
void check() const
{
assertFrameObject( this->frame_object );
// Make sure f_lineno is the actually used information.
assert( this->frame_object->f_trace == Py_None );
}
// Replace the frame object by a newer one.
void detachFrame( void )
{
// Our old frame should be on top.
assert( PyThreadState_GET()->frame == this->frame_object );
this->frame_object = detachCurrentFrame();
// Our new frame should be on top.
assert( PyThreadState_GET()->frame == this->frame_object );
}
private:
PyFrameObject *frame_object;
};
class FrameGuardWithExceptionPreservation
{
public:
explicit FrameGuardWithExceptionPreservation( PyFrameObject *frame_object )
{
assertFrameObject( frame_object );
// Remember it.
this->frame_object = frame_object;
// Push the new frame as the currently active one.
pushFrameStack( frame_object );
// Keep the frame object alive for this C++ objects live time.
Py_INCREF( frame_object );
this->preserving = false;
#if _DEBUG_REFRAME
// dumpFrameStack();
#endif
}
~FrameGuardWithExceptionPreservation()
{
// Our frame should be on top.
assert( PyThreadState_GET()->frame == this->frame_object );
// Put the previous frame on top instead.
popFrameStack();
assert( PyThreadState_GET()->frame != this->frame_object );
// Should still be good.
assertFrameObject( this->frame_object );
if ( this->preserving )
{
_SET_CURRENT_EXCEPTION( this->frame_object->f_exc_type, this->frame_object->f_exc_value, (PyTracebackObject *)this->frame_object->f_exc_traceback );
Py_XDECREF( this->frame_object->f_exc_type );
Py_XDECREF( this->frame_object->f_exc_value );
Py_XDECREF( this->frame_object->f_exc_traceback );
this->frame_object->f_exc_type = NULL;
this->frame_object->f_exc_value = NULL;
this->frame_object->f_exc_traceback = NULL;
}
// Now release our frame object reference.
Py_DECREF( this->frame_object );
}
inline PyFrameObject *getFrame() const
{
return INCREASE_REFCOUNT( this->frame_object );
}
inline PyFrameObject *getFrame0() const
{
return this->frame_object;
}
// Use this to set the current line of the frame
inline void setLineNumber( int lineno ) const
{
assertFrameObject( this->frame_object );
assert( lineno >= 1 );
// Make sure f_lineno is the actually used information.
assert( this->frame_object->f_trace == Py_None );
this->frame_object->f_lineno = lineno;
}
inline int getLineNumber() const
{
assertFrameObject( this->frame_object );
return this->frame_object->f_lineno;
}
void check() const
{
assertFrameObject( this->frame_object );
// Make sure f_lineno is the actually used information.
assert( this->frame_object->f_trace == Py_None );
}
// Replace the frame object by a newer one.
void detachFrame( void )
{
// Our old frame should be on top.
assert( PyThreadState_GET()->frame == this->frame_object );
this->frame_object = detachCurrentFrame();
// Our new frame should be on top.
assert( PyThreadState_GET()->frame == this->frame_object );
}
void preserveExistingException()
{
if ( this->preserving == false )
{
PyThreadState *thread_state = PyThreadState_GET();
if ( thread_state->exc_type != NULL && thread_state->exc_type != Py_None )
{
this->frame_object->f_exc_type = INCREASE_REFCOUNT( thread_state->exc_type );
this->frame_object->f_exc_value = INCREASE_REFCOUNT_X( thread_state->exc_value );
this->frame_object->f_exc_traceback = INCREASE_REFCOUNT_X( thread_state->exc_traceback );
}
else
{
this->frame_object->f_exc_type = NULL;
this->frame_object->f_exc_value = NULL;
this->frame_object->f_exc_traceback = NULL;
}
this->preserving = true;
}
}
#if PYTHON_VERSION >= 300
void restoreExistingException()
{
if ( this->preserving == true )
{
_SET_CURRENT_EXCEPTION( this->frame_object->f_exc_type, this->frame_object->f_exc_value, (PyTracebackObject *)this->frame_object->f_exc_traceback );
Py_XDECREF( this->frame_object->f_exc_type );
Py_XDECREF( this->frame_object->f_exc_value );
Py_XDECREF( this->frame_object->f_exc_traceback );
this->frame_object->f_exc_type = NULL;
this->frame_object->f_exc_value = NULL;
this->frame_object->f_exc_traceback = NULL;
this->preserving = false;
}
}
#endif
private:
bool preserving;
PyFrameObject *frame_object;
};
class FrameGuardLight
{
public:
explicit FrameGuardLight( PyFrameObject **frame_ptr )
{
assertFrameObject( *frame_ptr );
// Remember it.
this->frame_ptr = frame_ptr;
#if PYTHON_VERSION >= 300
preserving = false;
#endif
}
~FrameGuardLight()
{
// Should still be good.
assertFrameObject( *this->frame_ptr );
#if PYTHON_VERSION >= 300
if ( this->preserving )
{
PyFrameObject *frame_object = *this->frame_ptr;
_SET_CURRENT_EXCEPTION( frame_object->f_exc_type, frame_object->f_exc_value, (PyTracebackObject *)frame_object->f_exc_traceback );
Py_XDECREF( frame_object->f_exc_type );
Py_XDECREF( frame_object->f_exc_value );
Py_XDECREF( frame_object->f_exc_traceback );
frame_object->f_exc_type = NULL;
frame_object->f_exc_value = NULL;
frame_object->f_exc_traceback = NULL;
}
#endif
}
PyFrameObject *getFrame() const
{
return INCREASE_REFCOUNT( *this->frame_ptr );
}
PyFrameObject *getFrame0() const
{
return *this->frame_ptr;
}
inline int getLineNumber() const
{
return (*this->frame_ptr)->f_lineno;
}
// Use this to set the current line of the frame
void setLineNumber( int lineno ) const
{
assertFrameObject( *this->frame_ptr );
assert( lineno >= 1 );
// Make sure f_lineno is the actually used information.
assert( (*this->frame_ptr)->f_trace == Py_None );
(*this->frame_ptr)->f_lineno = lineno;
}
// Replace the frame object by a newer one.
void detachFrame( void )
{
// Our old frame should be on top.
assert( PyThreadState_GET()->frame == *this->frame_ptr );
*this->frame_ptr = detachCurrentFrame();
// Our new frame should be on top.
assert( PyThreadState_GET()->frame == *this->frame_ptr );
}
void preserveExistingException()
{
#if PYTHON_VERSION >= 300
if ( this->preserving == false )
{
PyThreadState *thread_state = PyThreadState_GET();
PyFrameObject *frame_object = *this->frame_ptr;
if ( thread_state->exc_type != NULL && thread_state->exc_type != Py_None )
{
frame_object->f_exc_type = INCREASE_REFCOUNT( thread_state->exc_type );
frame_object->f_exc_value = INCREASE_REFCOUNT_X( thread_state->exc_value );
frame_object->f_exc_traceback = INCREASE_REFCOUNT_X( thread_state->exc_traceback );
}
else
{
frame_object->f_exc_type = NULL;
frame_object->f_exc_value = NULL;
frame_object->f_exc_traceback = NULL;
}
this->preserving = true;
}
#endif
}
#if PYTHON_VERSION >= 300
void restoreExistingException()
{
if ( this->preserving == true )
{
PyFrameObject *frame_object = *this->frame_ptr;
_SET_CURRENT_EXCEPTION( frame_object->f_exc_type, frame_object->f_exc_value, (PyTracebackObject *)frame_object->f_exc_traceback );
Py_XDECREF( frame_object->f_exc_type );
Py_XDECREF( frame_object->f_exc_value );
Py_XDECREF( frame_object->f_exc_traceback );
frame_object->f_exc_type = NULL;
frame_object->f_exc_value = NULL;
frame_object->f_exc_traceback = NULL;
this->preserving = false;
}
}
#endif
private:
PyFrameObject **frame_ptr;
#if PYTHON_VERSION >= 300
bool preserving;
#endif
};
class FrameGuardVeryLight
{
public:
explicit FrameGuardVeryLight() {}
inline int getLineNumber() const
{
PyFrameObject *frame_object = PyThreadState_GET()->frame;
return frame_object->f_lineno;
}
inline void setLineNumber( int lineno ) const
{
PyFrameObject *frame_object = PyThreadState_GET()->frame;
assertFrameObject( frame_object );
assert( lineno >= 1 );
// Make sure f_lineno is the actually used information.
assert( frame_object->f_trace == Py_None );
frame_object->f_lineno = lineno;
}
PyFrameObject *getFrame() const
{
return INCREASE_REFCOUNT( this->getFrame0() );
}
PyFrameObject *getFrame0() const
{
return PyThreadState_GET()->frame;
}
void preserveExistingException()
{
}
void detachFrame( void )
{
}
#if PYTHON_VERSION >= 300
void restoreExistingException()
{
}
#endif
};
#if PYTHON_VERSION >= 300
class ExceptionRestorerFrameGuard
{
public:
explicit ExceptionRestorerFrameGuard( FrameGuardWithExceptionPreservation *frame_guard )
{
this->frame_guard = frame_guard;
}
~ExceptionRestorerFrameGuard()
{
this->frame_guard->restoreExistingException();
}
private:
FrameGuardWithExceptionPreservation *frame_guard;
};
class ExceptionRestorerFrameGuardLight
{
public:
explicit ExceptionRestorerFrameGuardLight( FrameGuardLight *frame_guard )
{
this->frame_guard = frame_guard;
}
~ExceptionRestorerFrameGuardLight()
{
this->frame_guard->restoreExistingException();
}
private:
FrameGuardLight *frame_guard;
};
class ExceptionRestorerFrameGuardVeryLight
{
public:
explicit ExceptionRestorerFrameGuardVeryLight( FrameGuardVeryLight *frame_guard )
{
this->frame_guard = frame_guard;
}
~ExceptionRestorerFrameGuardVeryLight()
{
this->frame_guard->restoreExistingException();
}
private:
FrameGuardVeryLight *frame_guard;
};
#endif
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/calling.hpp 0000644 0001750 0001750 00000005207 12265264105 023001 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_CALLING_H__
#define __NUITKA_CALLING_H__
#include "__helpers.hpp"
extern PyObject *const_tuple_empty;
NUITKA_MAY_BE_UNUSED static PyObject *CALL_FUNCTION( PyObject *function_object, PyObject *positional_args, PyObject *named_args )
{
assertObject( function_object );
assertObject( positional_args );
assert( named_args == NULL || Py_REFCNT( named_args ) > 0 );
ternaryfunc call_slot = Py_TYPE( function_object )->tp_call;
if (unlikely( call_slot == NULL ))
{
PyErr_Format(
PyExc_TypeError,
"'%s' object is not callable",
function_object->ob_type->tp_name
);
throw PythonException();
}
if (unlikely( Py_EnterRecursiveCall( (char *)" while calling a Python object") ))
{
throw PythonException();
}
PyObject *result = (*call_slot)( function_object, positional_args, named_args );
Py_LeaveRecursiveCall();
if ( result == NULL )
{
if (unlikely( !ERROR_OCCURED() ))
{
PyErr_Format(
PyExc_SystemError,
"NULL result without error in PyObject_Call"
);
}
throw PythonException();
}
else
{
return result;
}
}
// Function call variant with no arguments provided at all.
extern PyObject *CALL_FUNCTION_NO_ARGS( PyObject *called );
// Function call variants with positional arguments tuple.
NUITKA_MAY_BE_UNUSED static PyObject *CALL_FUNCTION_WITH_POSARGS( PyObject *function_object, PyObject *positional_args )
{
return CALL_FUNCTION(
function_object,
positional_args,
NULL
);
}
NUITKA_MAY_BE_UNUSED static PyObject *CALL_FUNCTION_WITH_KEYARGS( PyObject *function_object, PyObject *named_args )
{
return CALL_FUNCTION(
function_object,
const_tuple_empty,
named_args
);
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/compiled_generator.hpp 0000644 0001750 0001750 00000023324 12265264105 025232 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_COMPILED_GENERATOR_H__
#define __NUITKA_COMPILED_GENERATOR_H__
#include "Python.h"
#include "methodobject.h"
#include "frameobject.h"
// Compiled generator function type.
// Another cornerstone of the integration into CPython. Try to behave as well as
// normal generator function objects do or even better.
// *** Nuitka_Generator type begin
#include "fibers.hpp"
// Status of the generator object.
enum Generator_Status {
status_Unused, // Not used so far
status_Running, // Running, used but didn't stop yet
status_Finished // Stoped, no more values to come
};
// The Nuitka_GeneratorObject is the storage associated with a compiled
// generator object instance of which there can be many for each code.
typedef struct {
PyObject_HEAD
PyObject *m_name;
Fiber m_yielder_context;
Fiber m_caller_context;
void *m_context;
releaser m_cleanup;
// Weakrefs are supported for generator objects in CPython.
PyObject *m_weakrefs;
int m_running;
void *m_code;
PyObject *m_yielded;
PyObject *m_exception_type, *m_exception_value;
PyTracebackObject *m_exception_tb;
PyFrameObject *m_frame;
PyCodeObject *m_code_object;
// Was it ever used, is it still running, or already finished.
Generator_Status m_status;
} Nuitka_GeneratorObject;
extern PyTypeObject Nuitka_Generator_Type;
typedef void (*yielder_func)( Nuitka_GeneratorObject * );
extern PyObject *Nuitka_Generator_New( yielder_func code, PyObject *name, PyCodeObject *code_object, void *context, releaser cleanup );
extern PyObject *Nuitka_Generator_New( yielder_func code, PyObject *name, PyCodeObject *code_object );
static inline bool Nuitka_Generator_Check( PyObject *object )
{
return Py_TYPE( object ) == &Nuitka_Generator_Type;
}
static inline PyObject *Nuitka_Generator_GetName( PyObject *object )
{
return ((Nuitka_GeneratorObject *)object)->m_name;
}
static void RAISE_GENERATOR_EXCEPTION( Nuitka_GeneratorObject *generator )
{
assertObject( generator->m_exception_type );
Py_INCREF( generator->m_exception_type );
Py_XINCREF( generator->m_exception_value );
Py_XINCREF( generator->m_exception_tb );
PyErr_Restore(
generator->m_exception_type,
generator->m_exception_value,
(PyObject *)generator->m_exception_tb
);
generator->m_exception_type = NULL;
generator->m_exception_value = NULL;
generator->m_exception_tb = NULL;
throw PythonException();
}
static inline void CHECK_EXCEPTION( Nuitka_GeneratorObject *generator )
{
if ( generator->m_exception_type )
{
RAISE_GENERATOR_EXCEPTION( generator );
}
}
static inline PyObject *YIELD( Nuitka_GeneratorObject *generator, PyObject *value )
{
assertObject( value );
generator->m_yielded = value;
// Return to the calling context.
swapFiber( &generator->m_yielder_context, &generator->m_caller_context );
CHECK_EXCEPTION( generator );
return generator->m_yielded;
}
static inline PyObject *YIELD_IN_HANDLER( Nuitka_GeneratorObject *generator, PyObject *value )
{
assertObject( value );
generator->m_yielded = value;
#if PYTHON_VERSION >= 300
// When yielding from an exception handler in Python3, the exception
// preserved to the frame is restore, while the current one is put there.
PyThreadState *thread_state = PyThreadState_GET();
PyObject *saved_exception_type = thread_state->exc_type;
PyObject *saved_exception_value = thread_state->exc_value;
PyObject *saved_exception_traceback = thread_state->exc_traceback;
thread_state->exc_type = thread_state->frame->f_exc_type;
thread_state->exc_value = thread_state->frame->f_exc_value;
thread_state->exc_traceback = thread_state->frame->f_exc_traceback;
thread_state->frame->f_exc_type = saved_exception_type;
thread_state->frame->f_exc_value = saved_exception_value;
thread_state->frame->f_exc_traceback = saved_exception_traceback;
#endif
// Return to the calling context.
swapFiber( &generator->m_yielder_context, &generator->m_caller_context );
// When yielding, the exception preserved to the frame is restore, while the
// current one is put there.
#if PYTHON_VERSION >= 300
// When returning from yield, the exception of the frame is preserved, and
// the one that enters should be there.
thread_state = PyThreadState_GET();
saved_exception_type = thread_state->exc_type;
saved_exception_value = thread_state->exc_value;
saved_exception_traceback = thread_state->exc_traceback;
thread_state->exc_type = thread_state->frame->f_exc_type;
thread_state->exc_value = thread_state->frame->f_exc_value;
thread_state->exc_traceback = thread_state->frame->f_exc_traceback;
thread_state->frame->f_exc_type = saved_exception_type;
thread_state->frame->f_exc_value = saved_exception_value;
thread_state->frame->f_exc_traceback = saved_exception_traceback;
#endif
CHECK_EXCEPTION( generator );
return generator->m_yielded;
}
#if PYTHON_VERSION >= 330
extern PyObject *ERROR_GET_STOP_ITERATION_VALUE();
extern PyObject *PyGen_Send( PyGenObject *gen, PyObject *arg );
static inline PyObject *YIELD_FROM( Nuitka_GeneratorObject *generator, PyObject *value )
{
// This is the value, propagated back and forth the sub-generator and the
// yield from consumer.
PyObject *send_value = Py_None;
while( 1 )
{
// Send iteration value to the sub-generator, which may be a CPython
// generator object, something with an iterator next, or a send method,
// where the later is only required if values other than "None" need to
// be passed in.
PyObject *retval;
// Exception, was thrown into us, need to send that to sub-generator.
if ( generator->m_exception_type )
{
// The yielding generator is being closed, but we also are tasked to
// immediately close the currently running sub-generator.
if ( PyErr_GivenExceptionMatches( generator->m_exception_type, PyExc_GeneratorExit ) )
{
PyObject *close_method = PyObject_GetAttrString( value, (char *)"close" );
if ( close_method )
{
PyObject *close_value = PyObject_Call( close_method, const_tuple_empty, NULL );
Py_DECREF( close_method );
if (unlikely( close_value == NULL ))
{
throw PythonException();
}
Py_DECREF( close_value );
}
RAISE_GENERATOR_EXCEPTION( generator );
}
PyObject *throw_method = PyObject_GetAttrString( value, (char *)"throw" );
if ( throw_method )
{
retval = PyObject_CallFunctionObjArgs( throw_method, generator->m_exception_type, generator->m_exception_value, generator->m_exception_tb, NULL );
Py_DECREF( throw_method );
if (unlikely( send_value == NULL ))
{
if ( PyErr_ExceptionMatches( PyExc_StopIteration ) )
{
return ERROR_GET_STOP_ITERATION_VALUE();
}
throw PythonException();
}
generator->m_exception_type = NULL;
generator->m_exception_value = NULL;
generator->m_exception_tb = NULL;
}
else if ( PyErr_ExceptionMatches( PyExc_AttributeError ) )
{
PyErr_Clear();
RAISE_GENERATOR_EXCEPTION( generator );
}
else
{
throw PythonException();
}
}
else if ( PyGen_CheckExact( value ) )
{
retval = PyGen_Send( (PyGenObject *)value, Py_None );
}
else if ( send_value == Py_None )
{
retval = Py_TYPE( value )->tp_iternext( value );
}
else
{
retval = PyObject_CallMethod( value, (char *)"send", (char *)"O", send_value );
}
// Check the sub-generator result
if ( retval == NULL )
{
assert( ERROR_OCCURED() );
// The sub-generator has given an exception. In case of
// StopIteration, we need to check the value, as it is going to be
// the expression value of this "yield from", and we are done. All
// other errors, we need to raise.
if (likely( PyErr_ExceptionMatches( PyExc_StopIteration ) ))
{
return ERROR_GET_STOP_ITERATION_VALUE();
}
throw PythonException();
}
else
{
generator->m_yielded = retval;
// Return to the calling context.
swapFiber( &generator->m_yielder_context, &generator->m_caller_context );
send_value = generator->m_yielded;
assertObject( send_value );
}
}
}
#endif
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/helpers.hpp 0000644 0001750 0001750 00000067670 12265270763 023055 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_HELPERS_H__
#define __NUITKA_HELPERS_H__
#define _DEBUG_FRAME 0
#define _DEBUG_REFRAME 0
extern PyObject *const_tuple_empty;
extern PyObject *const_str_plain___dict__;
extern PyObject *const_str_plain___class__;
extern PyObject *const_str_plain___enter__;
extern PyObject *const_str_plain___exit__;
// From CPython, to allow us quick access to the dictionary of an module, the
// structure is normally private, but we need it for quick access to the module
// dictionary.
typedef struct {
PyObject_HEAD
PyObject *md_dict;
} PyModuleObject;
extern void PRINT_ITEM_TO( PyObject *file, PyObject *object );
static PyObject *INCREASE_REFCOUNT( PyObject *object );
static PyObject *INCREASE_REFCOUNT_X( PyObject *object );
// Helper to check that an object is valid and has positive reference count.
static inline void assertObject( PyObject *value )
{
assert( value != NULL );
assert( Py_REFCNT( value ) > 0 );
}
static inline void assertObject( PyTracebackObject *value )
{
assertObject( (PyObject *)value );
}
// Due to ABI issues, it seems that on Windows the symbols used by
// _PyObject_GC_TRACK are not exported and we need to use a function that does
// it instead.
#if defined( _WIN32 )
#define Nuitka_GC_Track PyObject_GC_Track
#define Nuitka_GC_UnTrack PyObject_GC_UnTrack
#else
#define Nuitka_GC_Track _PyObject_GC_TRACK
#define Nuitka_GC_UnTrack _PyObject_GC_UNTRACK
#endif
#include "nuitka/variables_temporary.hpp"
#include "nuitka/exceptions.hpp"
// For the MAKE_TUPLE macros.
#include "__helpers.hpp"
// Helper functions for reference count handling in the fly.
NUITKA_MAY_BE_UNUSED static PyObject *INCREASE_REFCOUNT( PyObject *object )
{
assertObject( object );
Py_INCREF( object );
return object;
}
NUITKA_MAY_BE_UNUSED static PyObject *INCREASE_REFCOUNT_X( PyObject *object )
{
Py_XINCREF( object );
return object;
}
NUITKA_MAY_BE_UNUSED static PyObject *DECREASE_REFCOUNT( PyObject *object )
{
assertObject( object );
Py_DECREF( object );
return object;
}
#include "printing.hpp"
#include "nuitka/helper/boolean.hpp"
#include "nuitka/helper/dictionaries.hpp"
#if PYTHON_VERSION >= 300
static char *_PyUnicode_AS_STRING( PyObject *unicode )
{
#if PYTHON_VERSION < 330
PyObject *bytes = _PyUnicode_AsDefaultEncodedString( unicode, NULL );
if (unlikely( bytes == NULL ))
{
throw PythonException();
}
return PyBytes_AS_STRING( bytes );
#else
return PyUnicode_AsUTF8( unicode );
#endif
}
#endif
#include "nuitka/helper/raising.hpp"
#include "helper/operations.hpp"
#include "nuitka/helper/richcomparisons.hpp"
#include "nuitka/helper/sequences.hpp"
static inline bool Nuitka_Function_Check( PyObject *object );
static inline PyObject *Nuitka_Function_GetName( PyObject *object );
static inline bool Nuitka_Generator_Check( PyObject *object );
static inline PyObject *Nuitka_Generator_GetName( PyObject *object );
#include "nuitka/calling.hpp"
NUITKA_MAY_BE_UNUSED static long FROM_LONG( PyObject *value )
{
long result = PyInt_AsLong( value );
if (unlikely( result == -1 ))
{
THROW_IF_ERROR_OCCURED();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *TO_FLOAT( PyObject *value )
{
PyObject *result;
#if PYTHON_VERSION < 300
if ( PyString_CheckExact( value ) )
{
result = PyFloat_FromString( value, NULL );
}
#else
if ( PyUnicode_CheckExact( value ) )
{
result = PyFloat_FromString( value );
}
#endif
else
{
result = PyNumber_Float( value );
}
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *TO_INT( PyObject *value )
{
PyObject *result = PyNumber_Int( value );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *TO_INT2( PyObject *value, PyObject *base )
{
int base_int = PyInt_AsLong( base );
if (unlikely( base_int == -1 ))
{
THROW_IF_ERROR_OCCURED();
}
#if PYTHON_VERSION < 300
if (unlikely( !Nuitka_String_Check( value ) ))
{
PyErr_Format( PyExc_TypeError, "int() can't convert non-string with explicit base" );
throw PythonException();
}
char *value_str = Nuitka_String_AsString( value );
if (unlikely( value_str == NULL ))
{
throw PythonException();
}
PyObject *result = PyInt_FromString( value_str, NULL, base_int );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
#else
if ( PyUnicode_Check( value ) )
{
#if PYTHON_VERSION < 330
char *value_str = Nuitka_String_AsString( value );
if (unlikely( value_str == NULL ))
{
throw PythonException();
}
PyObject *result = PyInt_FromString( value_str, NULL, base_int );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
#else
return PyLong_FromUnicodeObject( value, base_int );
#endif
}
else if ( PyBytes_Check( value ) || PyByteArray_Check( value ) )
{
// Check for "NUL" as PyLong_FromString has no length parameter,
Py_ssize_t size = Py_SIZE( value );
char *value_str;
if ( PyByteArray_Check( value ) )
{
value_str = PyByteArray_AS_STRING( value );
}
else
{
value_str = PyBytes_AS_STRING( value );
}
if ( strlen( value_str ) != (size_t)size || size == 0 )
{
PyErr_Format(
PyExc_ValueError,
"invalid literal for int() with base %d: %R",
base_int,
value
);
throw PythonException();
}
return PyLong_FromString( value_str, NULL, base_int );
}
else
{
PyErr_Format(
PyExc_TypeError,
"int() can't convert non-string with explicit base"
);
throw PythonException();
}
#endif
}
NUITKA_MAY_BE_UNUSED static PyObject *TO_LONG( PyObject *value )
{
PyObject *result = PyNumber_Long( value );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *TO_LONG2( PyObject *value, PyObject *base )
{
int base_int = PyInt_AsLong( base );
if (unlikely( base_int == -1 ))
{
THROW_IF_ERROR_OCCURED();
}
#if PYTHON_VERSION < 300
if (unlikely( !Nuitka_String_Check( value ) ))
{
PyErr_Format( PyExc_TypeError, "long() can't convert non-string with explicit base" );
throw PythonException();
}
#endif
char *value_str = Nuitka_String_AsString( value );
if (unlikely( value_str == NULL ))
{
throw PythonException();
}
PyObject *result = PyLong_FromString( value_str, NULL, base_int );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *TO_BOOL( PyObject *value )
{
return BOOL_FROM( CHECK_IF_TRUE( value ) );
}
NUITKA_MAY_BE_UNUSED static PyObject *TO_STR( PyObject *value )
{
PyObject *result = PyObject_Str( value );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *TO_UNICODE( PyObject *value )
{
PyObject *result = PyObject_Unicode( value );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *TO_UNICODE3( PyObject *value, PyObject *encoding, PyObject *errors )
{
assertObject( encoding );
char *encoding_str;
if ( encoding == NULL )
{
encoding_str = NULL;
}
else if ( Nuitka_String_Check( encoding ) )
{
encoding_str = Nuitka_String_AsString_Unchecked( encoding );
}
#if PYTHON_VERSION < 300
else if ( PyUnicode_Check( encoding ) )
{
PyObject *uarg2 = _PyUnicode_AsDefaultEncodedString( encoding, NULL );
assertObject( uarg2 );
encoding_str = Nuitka_String_AsString_Unchecked( uarg2 );
}
#endif
else
{
PyErr_Format( PyExc_TypeError, "unicode() argument 2 must be string, not %s", Py_TYPE( encoding )->tp_name );
throw PythonException();
}
char *errors_str;
if ( errors == NULL )
{
errors_str = NULL;
}
else if ( Nuitka_String_Check( errors ) )
{
errors_str = Nuitka_String_AsString_Unchecked( errors );
}
#if PYTHON_VERSION < 300
else if ( PyUnicode_Check( errors ) )
{
PyObject *uarg3 = _PyUnicode_AsDefaultEncodedString( errors, NULL );
assertObject( uarg3 );
errors_str = Nuitka_String_AsString_Unchecked( uarg3 );
}
#endif
else
{
PyErr_Format( PyExc_TypeError, "unicode() argument 3 must be string, not %s", Py_TYPE( errors )->tp_name );
throw PythonException();
}
PyObject *result = PyUnicode_FromEncodedObject( value, encoding_str, errors_str );
if (unlikely( result == NULL ))
{
throw PythonException();
}
assert( PyUnicode_Check( result ) );
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *MAKE_STATIC_METHOD( PyObject *method )
{
assertObject( method );
PyObject *attempt = PyStaticMethod_New( method );
if ( attempt )
{
return attempt;
}
else
{
PyErr_Clear();
return method;
}
}
// Stolen from CPython implementation, so we can access it.
typedef struct {
PyObject_HEAD
long it_index;
PyObject *it_seq;
} seqiterobject;
NUITKA_MAY_BE_UNUSED static PyObject *MAKE_ITERATOR( PyObject *iterated )
{
getiterfunc tp_iter = NULL;
#if PYTHON_VERSION < 300
if ( PyType_HasFeature( Py_TYPE( iterated ), Py_TPFLAGS_HAVE_ITER ))
{
#endif
tp_iter = Py_TYPE( iterated )->tp_iter;
#if PYTHON_VERSION < 300
}
#endif
if ( tp_iter )
{
PyObject *result = (*Py_TYPE( iterated )->tp_iter)( iterated );
if (likely( result != NULL ))
{
if (unlikely( !PyIter_Check( result )) )
{
PyErr_Format( PyExc_TypeError, "iter() returned non-iterator of type '%s'", Py_TYPE( result )->tp_name );
Py_DECREF( result );
throw PythonException();
}
return result;
}
else
{
throw PythonException();
}
}
else if ( PySequence_Check( iterated ) )
{
seqiterobject *result = PyObject_GC_New( seqiterobject, &PySeqIter_Type );
assert( result );
result->it_index = 0;
result->it_seq = INCREASE_REFCOUNT( iterated );
Nuitka_GC_Track( result );
return (PyObject *)result;
}
else
{
PyErr_Format( PyExc_TypeError, "'%s' object is not iterable", Py_TYPE( iterated )->tp_name );
throw PythonException();
}
}
// Return the next item of an iterator. Avoiding any exception for end of
// iteration, callers must deal with NULL return as end of iteration, but will
// know it wasn't an Python exception, that will show as a thrown exception.
NUITKA_MAY_BE_UNUSED static PyObject *ITERATOR_NEXT( PyObject *iterator )
{
assertObject( iterator );
assert( Py_TYPE( iterator )->tp_iternext );
PyObject *result = (*Py_TYPE( iterator )->tp_iternext)( iterator );
if (unlikely( result == NULL ))
{
THROW_IF_ERROR_OCCURED_NOT( PyExc_StopIteration );
}
else
{
assertObject( result );
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *BUILTIN_NEXT1( PyObject *iterator )
{
assertObject( iterator );
assert( Py_TYPE( iterator )->tp_iternext );
PyObject *result = (*Py_TYPE( iterator )->tp_iternext)( iterator );
if (unlikely( result == NULL ))
{
// The iteration can return NULL with no error, which means StopIteration.
if ( !ERROR_OCCURED() )
{
throw PythonException( PyExc_StopIteration );
}
throw PythonException();
}
else
{
assertObject( result );
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *BUILTIN_NEXT2( PyObject *iterator, PyObject *default_value )
{
assertObject( iterator );
assertObject( default_value );
PyObject *result = (*Py_TYPE( iterator )->tp_iternext)( iterator );
if (unlikely( result == NULL ))
{
if ( ERROR_OCCURED() )
{
if ( PyErr_ExceptionMatches( PyExc_StopIteration ))
{
PyErr_Clear();
return INCREASE_REFCOUNT( default_value );
}
else
{
throw PythonException();
}
}
else
{
return INCREASE_REFCOUNT( default_value );
}
}
else
{
assertObject( result );
}
return result;
}
NUITKA_MAY_BE_UNUSED static inline PyObject *UNPACK_NEXT( PyObject *iterator, int seq_size_so_far )
{
assertObject( iterator );
assert( PyIter_Check( iterator ) );
PyObject *result = (*Py_TYPE( iterator )->tp_iternext)( iterator );
if (unlikely( result == NULL ))
{
#if PYTHON_VERSION < 300
if (unlikely( !ERROR_OCCURED() ))
#else
if (unlikely( !ERROR_OCCURED() || PyErr_ExceptionMatches( PyExc_StopIteration ) ))
#endif
{
if ( seq_size_so_far == 1 )
{
PyErr_Format( PyExc_ValueError, "need more than 1 value to unpack" );
}
else
{
PyErr_Format( PyExc_ValueError, "need more than %d values to unpack", seq_size_so_far );
}
}
throw PythonException();
}
assertObject( result );
return result;
}
NUITKA_MAY_BE_UNUSED static inline PyObject *UNPACK_PARAMETER_NEXT( PyObject *iterator, int seq_size_so_far )
{
assertObject( iterator );
assert( PyIter_Check( iterator ) );
PyObject *result = (*Py_TYPE( iterator )->tp_iternext)( iterator );
if (unlikely( result == NULL ))
{
#if PYTHON_VERSION < 300
if (unlikely( !ERROR_OCCURED() ))
#else
if (unlikely( !ERROR_OCCURED() || PyErr_ExceptionMatches( PyExc_StopIteration ) ))
#endif
{
if ( seq_size_so_far == 1 )
{
PyErr_Format( PyExc_ValueError, "need more than 1 value to unpack" );
}
else
{
PyErr_Format( PyExc_ValueError, "need more than %d values to unpack", seq_size_so_far );
}
}
return NULL;
}
assertObject( result );
return result;
}
#if PYTHON_VERSION < 300
#define UNPACK_ITERATOR_CHECK( iterator, count ) _UNPACK_ITERATOR_CHECK( iterator )
NUITKA_MAY_BE_UNUSED static inline void _UNPACK_ITERATOR_CHECK( PyObject *iterator )
#else
NUITKA_MAY_BE_UNUSED static inline void UNPACK_ITERATOR_CHECK( PyObject *iterator, int count )
#endif
{
assertObject( iterator );
assert( PyIter_Check( iterator ) );
PyObject *attempt = (*Py_TYPE( iterator )->tp_iternext)( iterator );
if (likely( attempt == NULL ))
{
THROW_IF_ERROR_OCCURED_NOT( PyExc_StopIteration );
}
else
{
Py_DECREF( attempt );
#if PYTHON_VERSION < 300
PyErr_Format( PyExc_ValueError, "too many values to unpack" );
#else
PyErr_Format( PyExc_ValueError, "too many values to unpack (expected %d)", count );
#endif
throw PythonException();
}
}
NUITKA_MAY_BE_UNUSED static inline bool UNPACK_PARAMETER_ITERATOR_CHECK( PyObject *iterator )
{
assertObject( iterator );
assert( PyIter_Check( iterator ) );
PyObject *attempt = (*Py_TYPE( iterator )->tp_iternext)( iterator );
if (likely( attempt == NULL ))
{
if ( ERROR_OCCURED() )
{
if (likely( PyErr_ExceptionMatches( PyExc_StopIteration ) ))
{
PyErr_Clear();
}
else
{
return false;
}
}
return true;
}
else
{
Py_DECREF( attempt );
PyErr_Format( PyExc_ValueError, "too many values to unpack" );
return false;
}
}
NUITKA_MAY_BE_UNUSED static bool HAS_KEY( PyObject *source, PyObject *key )
{
assertObject( source );
assertObject( key );
assert( PyMapping_Check( source ) );
return PyMapping_HasKey( source, key ) != 0;
}
NUITKA_MAY_BE_UNUSED static PyObject *LOOKUP_VARS( PyObject *source )
{
assertObject( source );
PyObject *result = PyObject_GetAttr( source, const_str_plain___dict__ );
if (unlikely( result == NULL ))
{
PyErr_Format(
PyExc_TypeError,
"vars() argument must have __dict__ attribute"
);
throw PythonException();
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *IMPORT_NAME( PyObject *module, PyObject *import_name )
{
assertObject( module );
assertObject( import_name );
PyObject *result = PyObject_GetAttr( module, import_name );
if (unlikely( result == NULL ))
{
if ( PyErr_ExceptionMatches( PyExc_AttributeError ) )
{
PyErr_Format( PyExc_ImportError, "cannot import name %s", Nuitka_String_AsString( import_name ));
}
throw PythonException();
}
return result;
}
#include "nuitka/helper/indexes.hpp"
#include "nuitka/helper/subscripts.hpp"
#include "nuitka/helper/slices.hpp"
#include "nuitka/helper/attributes.hpp"
NUITKA_MAY_BE_UNUSED static void APPEND_TO_LIST( PyObject *list, PyObject *item )
{
assertObject( list );
assertObject( item );
int status = PyList_Append( list, item );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
NUITKA_MAY_BE_UNUSED static void ADD_TO_SET( PyObject *set, PyObject *item )
{
int status = PySet_Add( set, item );
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
NUITKA_MAY_BE_UNUSED static PyObject *SEQUENCE_CONCAT( PyObject *seq1, PyObject *seq2 )
{
PyObject *result = PySequence_Concat( seq1, seq2 );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
#include "nuitka/builtins.hpp"
#include "nuitka/frame_guards.hpp"
#include "nuitka/variables_parameters.hpp"
#include "nuitka/variables_locals.hpp"
#include "nuitka/variables_shared.hpp"
NUITKA_MAY_BE_UNUSED static PyObject *TUPLE_COPY( PyObject *tuple )
{
assertObject( tuple );
assert( PyTuple_CheckExact( tuple ) );
Py_ssize_t size = PyTuple_GET_SIZE( tuple );
PyObject *result = PyTuple_New( size );
if (unlikely( result == NULL ))
{
throw PythonException();
}
for ( Py_ssize_t i = 0; i < size; i++ )
{
PyTuple_SET_ITEM( result, i, INCREASE_REFCOUNT( PyTuple_GET_ITEM( tuple, i ) ) );
}
return result;
}
NUITKA_MAY_BE_UNUSED static PyObject *LIST_COPY( PyObject *list )
{
assertObject( list );
assert( PyList_CheckExact( list ) );
Py_ssize_t size = PyList_GET_SIZE( list );
PyObject *result = PyList_New( size );
if (unlikely( result == NULL ))
{
throw PythonException();
}
for ( Py_ssize_t i = 0; i < size; i++ )
{
PyList_SET_ITEM( result, i, INCREASE_REFCOUNT( PyList_GET_ITEM( list, i ) ) );
}
return result;
}
// Compile source code given, pretending the file name was given.
extern PyObject *COMPILE_CODE( PyObject *source_code, PyObject *file_name, PyObject *mode, int flags );
// For quicker builtin open() functionality.
extern PyObject *OPEN_FILE( PyObject *file_name, PyObject *mode, PyObject *buffering );
// For quicker builtin chr() functionality.
extern PyObject *BUILTIN_CHR( PyObject *value );
// For quicker builtin ord() functionality.
extern PyObject *BUILTIN_ORD( PyObject *value );
// For quicker builtin bin() functionality.
extern PyObject *BUILTIN_BIN( PyObject *value );
// For quicker builtin oct() functionality.
extern PyObject *BUILTIN_OCT( PyObject *value );
// For quicker builtin hex() functionality.
extern PyObject *BUILTIN_HEX( PyObject *value );
// For quicker callable() functionality.
extern PyObject *BUILTIN_CALLABLE( PyObject *value );
// For quicker iter() functionality if 2 arguments arg given.
extern PyObject *BUILTIN_ITER2( PyObject *callable, PyObject *sentinel );
// For quicker type() functionality if 1 argument is given.
extern PyObject *BUILTIN_TYPE1( PyObject *arg );
// For quicker type() functionality if 3 arguments are given (to build a new
// type).
extern PyObject *BUILTIN_TYPE3( PyObject *module_name, PyObject *name, PyObject *bases, PyObject *dict );
// For quicker builtin range() functionality.
extern PyObject *BUILTIN_RANGE3( PyObject *low, PyObject *high, PyObject *step );
extern PyObject *BUILTIN_RANGE2( PyObject *low, PyObject *high );
extern PyObject *BUILTIN_RANGE( PyObject *boundary );
extern PyObject *BUILTIN_XRANGE( PyObject *low, PyObject *high, PyObject *step );
// For quicker builtin len() functionality.
extern PyObject *BUILTIN_LEN( PyObject *boundary );
// For quicker builtin dir(arg) functionality.
extern PyObject *BUILTIN_DIR1( PyObject *arg );
// For quicker builtin super() functionality.
extern PyObject *BUILTIN_SUPER( PyObject *type, PyObject *object );
// For quicker isinstance() functionality.
extern PyObject *BUILTIN_ISINSTANCE( PyObject *inst, PyObject *cls );
extern bool BUILTIN_ISINSTANCE_BOOL( PyObject *inst, PyObject *cls );
// For quicker getattr() functionality.
extern PyObject *BUILTIN_GETATTR( PyObject *object, PyObject *attribute, PyObject *default_value );
// For quicker setattr() functionality.
extern void BUILTIN_SETATTR( PyObject *object, PyObject *attribute, PyObject *value );
extern PyObject *const_str_plain___builtins__;
NUITKA_MAY_BE_UNUSED static PyObject *EVAL_CODE( PyObject *code, PyObject *globals, PyObject *locals )
{
if ( PyDict_Check( globals ) == 0 )
{
PyErr_Format( PyExc_TypeError, "exec: arg 2 must be a dictionary or None" );
throw PythonException();
}
if ( locals == NULL || locals == Py_None )
{
locals = globals;
}
if ( PyMapping_Check( locals ) == 0 )
{
PyErr_Format( PyExc_TypeError, "exec: arg 3 must be a mapping or None" );
throw PythonException();
}
// Set the __builtins__ in globals, it is expected to be present.
if ( PyDict_GetItem( globals, const_str_plain___builtins__ ) == NULL )
{
if ( PyDict_SetItem( globals, const_str_plain___builtins__, (PyObject *)module_builtin ) == -1 )
{
throw PythonException();
}
}
#if PYTHON_VERSION < 300
PyObject *result = PyEval_EvalCode( (PyCodeObject *)code, globals, locals );
#else
PyObject *result = PyEval_EvalCode( code, globals, locals );
#endif
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
#include "nuitka/importing.hpp"
// For the constant loading:
extern void UNSTREAM_INIT( void );
extern PyObject *UNSTREAM_CONSTANT( unsigned char const *buffer, Py_ssize_t size );
extern PyObject *UNSTREAM_STRING( unsigned char const *buffer, Py_ssize_t size, bool intern );
extern PyObject *UNSTREAM_CHAR( unsigned char value, bool intern );
#if PYTHON_VERSION < 300
extern PyObject *UNSTREAM_UNICODE( unsigned char const *buffer, Py_ssize_t size );
#else
extern PyObject *UNSTREAM_BYTES( unsigned char const *buffer, Py_ssize_t size );
#endif
extern PyObject *UNSTREAM_FLOAT( unsigned char const *buffer );
extern void enhancePythonTypes( void );
// Parse the command line parameters and provide it to sys module.
extern void setCommandLineParameters( int argc, char *argv[], bool initial );
// Replace builtin functions with ones that accept compiled types too.
extern void patchBuiltinModule( void );
// Replace type comparison with one that accepts compiled types too, will work
// for "==" and "!=", but not for "is" checks.
extern void patchTypeComparison( void );
#if PYTHON_VERSION < 300
// Initialize value for tp_compare default.
extern cmpfunc DefaultSlotCompare;
extern void initSlotCompare( void );
#endif
#if PYTHON_VERSION >= 300
NUITKA_MAY_BE_UNUSED static PyObject *SELECT_METACLASS( PyObject *metaclass, PyObject *bases )
{
assertObject( metaclass );
assertObject( bases );
if (likely( PyType_Check( metaclass ) ))
{
// Determine the proper metatype
Py_ssize_t nbases = PyTuple_GET_SIZE( bases );
PyTypeObject *winner = (PyTypeObject *)metaclass;
for ( int i = 0; i < nbases; i++ )
{
PyObject *base = PyTuple_GET_ITEM( bases, i );
PyTypeObject *base_type = Py_TYPE( base );
if ( PyType_IsSubtype( winner, base_type ) )
{
// Ignore if current winner is already a subtype.
continue;
}
else if ( PyType_IsSubtype( base_type, winner ) )
{
// Use if, if it's a subtype of the current winner.
winner = base_type;
continue;
}
else
{
PyErr_Format(
PyExc_TypeError,
"metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases"
);
throw PythonException();
}
}
if (unlikely( winner == NULL ))
{
throw PythonException();
}
return INCREASE_REFCOUNT( (PyObject *)winner );
}
else
{
return INCREASE_REFCOUNT( metaclass );
}
}
#else
NUITKA_MAY_BE_UNUSED static PyObject *SELECT_METACLASS( PyObject *bases, PyObject *metaclass_global )
{
assertObject( bases );
PyObject *metaclass;
assert( bases != Py_None );
if ( PyTuple_GET_SIZE( bases ) > 0 )
{
PyObject *base = PyTuple_GET_ITEM( bases, 0 );
metaclass = PyObject_GetAttr( base, const_str_plain___class__ );
if ( metaclass == NULL )
{
PyErr_Clear();
metaclass = INCREASE_REFCOUNT( (PyObject *)Py_TYPE( base ) );
}
}
else if ( metaclass_global != NULL )
{
metaclass = INCREASE_REFCOUNT( metaclass_global );
}
else
{
// Default to old style class.
metaclass = INCREASE_REFCOUNT( (PyObject *)&PyClass_Type );
}
return metaclass;
}
#endif
NUITKA_MAY_BE_UNUSED static PyObject *MODULE_NAME( PyObject *module )
{
char const *module_name = PyModule_GetName( module );
#if PYTHON_VERSION < 300
PyObject *result = PyString_FromString( module_name );
PyString_InternInPlace( &result );
return result;
#else
PyObject *result = PyUnicode_FromString( module_name );
PyUnicode_InternInPlace( &result );
return result;
#endif
}
#if defined(_NUITKA_STANDALONE) || _NUITKA_FROZEN > 0
extern void prepareStandaloneEnvironment();
extern char *getBinaryDirectory();
#endif
#if _NUITKA_STANDALONE
extern void setEarlyFrozenModulesFileAttribute( void );
#endif
#include
NUITKA_MAY_BE_UNUSED static PyObject *MAKE_TUPLE( PyObject **elements, Py_ssize_t size )
{
PyObject *result = PyTuple_New( size );
for( Py_ssize_t i = 0; i < size; i++ )
{
PyTuple_SET_ITEM( result, i, INCREASE_REFCOUNT( elements[i] ) );
}
return result;
}
// Make a deep copy of an object.
extern PyObject *DEEP_COPY( PyObject *value );
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/exceptions.hpp 0000644 0001750 0001750 00000032020 12265264105 023542 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_EXCEPTIONS_H__
#define __NUITKA_EXCEPTIONS_H__
static bool ERROR_OCCURED( void )
{
PyThreadState *tstate = PyThreadState_GET();
return tstate->curexc_type != NULL;
}
NUITKA_MAY_BE_UNUSED static PyObject *GET_ERROR_OCCURED( void )
{
PyThreadState *tstate = PyThreadState_GET();
return tstate->curexc_type;
}
#if PYTHON_VERSION < 300
NUITKA_MAY_BE_UNUSED static void dumpTraceback( PyTracebackObject *traceback )
{
puts( "Dumping traceback:" );
if ( traceback == NULL ) puts( "" );
while( traceback )
{
puts( " Frame object chain:" );
PyFrameObject *frame = traceback->tb_frame;
while ( frame )
{
printf( " Frame at %s\n", PyString_AsString( PyObject_Str( (PyObject *)frame->f_code )));
frame = frame->f_back;
}
traceback = traceback->tb_next;
}
puts( "End of Dump." );
}
#endif
NUITKA_MAY_BE_UNUSED static PyTracebackObject *INCREASE_REFCOUNT( PyTracebackObject *traceback_object )
{
Py_INCREF( traceback_object );
return traceback_object;
}
NUITKA_MAY_BE_UNUSED static PyTracebackObject *INCREASE_REFCOUNT_X( PyTracebackObject *traceback_object )
{
Py_XINCREF( traceback_object );
return traceback_object;
}
NUITKA_MAY_BE_UNUSED static PyObject *MAKE_TRACEBACK( PyFrameObject *frame )
{
// assertFrameObject( frame );
PyTracebackObject *result = PyObject_GC_New( PyTracebackObject, &PyTraceBack_Type );
result->tb_next = NULL;
result->tb_frame = frame;
result->tb_lasti = 0;
result->tb_lineno = frame->f_lineno;
Nuitka_GC_Track( result );
return (PyObject *)result;
}
#if PYTHON_VERSION < 300
extern PyObject *const_str_plain_exc_type, *const_str_plain_exc_value, *const_str_plain_exc_traceback;
#endif
// Helper that sets the current thread exception, releasing the current one, for
// use in this file only.
inline void _SET_CURRENT_EXCEPTION( PyObject *exception_type, PyObject *exception_value, PyTracebackObject *exception_tb )
{
PyThreadState *thread_state = PyThreadState_GET();
PyObject *old_type = thread_state->exc_type;
PyObject *old_value = thread_state->exc_value;
PyObject *old_tb = thread_state->exc_traceback;
thread_state->exc_type = INCREASE_REFCOUNT_X( exception_type );
thread_state->exc_value = INCREASE_REFCOUNT_X( exception_value );
thread_state->exc_traceback = (PyObject *)INCREASE_REFCOUNT_X( exception_tb );
Py_XDECREF( old_type );
Py_XDECREF( old_value );
Py_XDECREF( old_tb );
#if PYTHON_VERSION < 300
// Set sys attributes in the fastest possible way.
PyObject *sys_dict = thread_state->interp->sysdict;
assertObject( sys_dict );
PyDict_SetItem( sys_dict, const_str_plain_exc_type, exception_type ? exception_type : Py_None );
PyDict_SetItem( sys_dict, const_str_plain_exc_value, exception_value ? exception_value : Py_None );
PyDict_SetItem( sys_dict, const_str_plain_exc_traceback, exception_tb ? (PyObject *)exception_tb : Py_None );
#endif
}
inline void NORMALIZE_EXCEPTION( PyObject **exception_type, PyObject **exception_value, PyTracebackObject **exception_tb )
{
PyErr_NormalizeException( exception_type, exception_value, (PyObject **)exception_tb );
}
class PythonException
{
public:
PythonException()
{
this->_importFromPython();
}
PythonException( PyObject *exception )
{
assertObject( exception );
Py_INCREF( exception );
this->exception_type = exception;
this->exception_value = NULL;
this->exception_tb = NULL;
}
PythonException( PyObject *exception, PyTracebackObject *traceback )
{
assertObject( exception );
assertObject( traceback );
this->exception_type = exception;
this->exception_value = NULL;
this->exception_tb = traceback;
}
PythonException( PyObject *exception, PyObject *value, PyTracebackObject *traceback )
{
assertObject( exception );
assert( value == NULL || Py_REFCNT( value ) > 0 );
assert( traceback == NULL || Py_REFCNT( traceback ) > 0 );
this->exception_type = exception;
this->exception_value = value;
this->exception_tb = traceback;
}
PythonException( const PythonException &other )
{
this->exception_type = other.exception_type;
this->exception_value = other.exception_value;
this->exception_tb = other.exception_tb;
Py_XINCREF( this->exception_type );
Py_XINCREF( this->exception_value );
Py_XINCREF( this->exception_tb );
}
void operator=( const PythonException &other )
{
Py_XINCREF( other.exception_type );
Py_XINCREF( other.exception_value );
Py_XINCREF( other.exception_tb );
Py_XDECREF( this->exception_type );
Py_XDECREF( this->exception_value );
Py_XDECREF( this->exception_tb );
this->exception_type = other.exception_type;
this->exception_value = other.exception_value;
this->exception_tb = other.exception_tb;
}
~PythonException()
{
Py_XDECREF( this->exception_type );
Py_XDECREF( this->exception_value );
Py_XDECREF( this->exception_tb );
}
inline void _importFromPython()
{
PyErr_Fetch( &this->exception_type, &this->exception_value, (PyObject **)&this->exception_tb );
assertObject( this->exception_type );
}
inline void normalize()
{
NORMALIZE_EXCEPTION( &this->exception_type, &this->exception_value, &this->exception_tb );
#if PYTHON_VERSION >= 300
PyException_SetTraceback( this->exception_value, (PyObject *)this->exception_tb );
#endif
}
inline bool matches( PyObject *exception ) const
{
#if PYTHON_VERSION >= 300
if ( PyTuple_Check( exception ))
{
Py_ssize_t length = PyTuple_Size( exception );
for ( Py_ssize_t i = 0; i < length; i += 1 )
{
PyObject *element = PyTuple_GET_ITEM( exception, i );
if (unlikely( !PyExceptionClass_Check( element ) ))
{
PyErr_Format( PyExc_TypeError, "catching classes that do not inherit from BaseException is not allowed" );
throw PythonException();
}
}
}
else if (unlikely( !PyExceptionClass_Check( exception ) ))
{
PyErr_Format( PyExc_TypeError, "catching classes that do not inherit from BaseException is not allowed" );
throw PythonException();
}
#endif
return
PyErr_GivenExceptionMatches( this->exception_type, exception ) ||
PyErr_GivenExceptionMatches( this->exception_value, exception );
}
inline void toPython()
{
PyErr_Restore( this->exception_type, this->exception_value, (PyObject *)this->exception_tb );
assert( this->exception_type );
#ifndef __NUITKA_NO_ASSERT__
PyThreadState *thread_state = PyThreadState_GET();
#endif
assert( this->exception_type == thread_state->curexc_type );
assert( thread_state->curexc_type );
this->exception_type = NULL;
this->exception_value = NULL;
this->exception_tb = NULL;
}
inline void toExceptionHandler()
{
this->normalize();
_SET_CURRENT_EXCEPTION( this->exception_type, this->exception_value, this->exception_tb );
}
inline PyObject *getType()
{
if ( this->exception_value == NULL )
{
this->normalize();
}
return this->exception_type;
}
inline PyObject *getValue()
{
if ( this->exception_value == NULL )
{
this->normalize();
}
return this->exception_value;
}
inline PyTracebackObject *getTraceback() const
{
return (PyTracebackObject *)this->exception_tb;
}
inline void addTraceback( PyFrameObject *frame )
{
assert( this->exception_tb );
#if 0
printf( "addTraceback %p %s %p %s %d %d\n", exception_tb->tb_frame, PyString_AsString( exception_tb->tb_frame->f_code->co_name ), frame, PyString_AsString( frame->f_code->co_name ), this->exception_tb->tb_lineno, frame->f_lineno );
#endif
if ( this->exception_tb->tb_frame != frame || this->exception_tb->tb_lineno != frame->f_lineno )
{
Py_INCREF( frame );
PyTracebackObject *traceback_new = (PyTracebackObject *)MAKE_TRACEBACK( frame );
traceback_new->tb_next = this->exception_tb;
this->exception_tb = traceback_new;
}
}
inline void setTraceback( PyTracebackObject *traceback )
{
assert( traceback == NULL || Py_REFCNT( traceback ) > 0 );
PyTracebackObject *old = this->exception_tb;
this->exception_tb = traceback;
Py_XDECREF( old );
}
inline void setTraceback( PyObject *traceback )
{
assert( PyTraceBack_Check( traceback ) );
return this->setTraceback( (PyTracebackObject *)traceback );
}
inline bool hasTraceback() const
{
return this->exception_tb != NULL;
}
void setType( PyObject *exception_type )
{
Py_XDECREF( this->exception_type );
this->exception_type = exception_type;
}
#if PYTHON_VERSION >= 300
void setCause( PyObject *exception_cause )
{
PyException_SetCause( this->exception_value, exception_cause );
}
#endif
void dump() const
{
PRINT_ITEM_TO( NULL, this->exception_type );
}
private:
friend class PythonExceptionKeeper;
// For the restore of saved ones.
PythonException( PyObject *exception, PyObject *value, PyObject *traceback )
{
this->exception_type = exception;
this->exception_value = value;
this->exception_tb = (PyTracebackObject *)traceback;
}
PyObject *exception_type, *exception_value;
PyTracebackObject *exception_tb;
};
class PythonExceptionKeeper
{
public:
PythonExceptionKeeper()
{
this->keeping = false;
#ifndef __NUITKA_NO_ASSERT__
this->exception_type = NULL;
this->exception_value = NULL;
this->exception_tb = NULL;
#endif
}
~PythonExceptionKeeper()
{
if ( this->keeping )
{
Py_XDECREF( this->exception_type );
Py_XDECREF( this->exception_value );
Py_XDECREF( this->exception_tb );
}
}
void save( const PythonException &e )
{
this->exception_type = INCREASE_REFCOUNT_X( e.exception_type );
this->exception_value = INCREASE_REFCOUNT_X( e.exception_value );
this->exception_tb = INCREASE_REFCOUNT_X( e.exception_tb );
this->keeping = true;
}
void rethrow()
{
if ( this->keeping )
{
Py_XINCREF( this->exception_type );
Py_XINCREF( this->exception_value );
Py_XINCREF( this->exception_tb );
// Restore the frame line number from the traceback, if
// present. Otherwise it will changed already.
if ( this->exception_tb )
{
this->exception_tb->tb_frame->f_lineno = this->exception_tb->tb_lineno;
}
throw PythonException( this->exception_type, this->exception_value, this->exception_tb );
}
}
bool isEmpty() const
{
return !this->keeping;
}
private:
PythonExceptionKeeper( const PythonExceptionKeeper &other )
{
assert( false );
}
bool keeping;
PyObject *exception_type, *exception_value;
PyTracebackObject *exception_tb;
};
class ContinueException
{
};
class BreakException
{
};
class ReturnValueException
{
public:
explicit ReturnValueException( PyObject *value )
{
// Always called with extra reference, whose ownership is transfered to
// us here.
assertObject( value );
this->value = value;
}
ReturnValueException( const ReturnValueException &other )
{
this->value = other.getValue1();
}
~ReturnValueException()
{
assertObject( value );
Py_DECREF( this->value );
}
PyObject *getValue0() const
{
return this->value;
}
PyObject *getValue1() const
{
return INCREASE_REFCOUNT( this->value );
}
private:
PyObject *value;
};
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/variables_shared.hpp 0000644 0001750 0001750 00000025266 12265264105 024675 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_VARIABLES_SHARED_H__
#define __NUITKA_VARIABLES_SHARED_H__
class PyObjectSharedStorage
{
public:
explicit PyObjectSharedStorage( PyObject *var_name, PyObject *object )
{
assert( object == NULL || Py_REFCNT( object ) > 0 );
this->var_name = var_name;
this->object = object;
this->ref_count = 1;
}
~PyObjectSharedStorage()
{
Py_XDECREF( this->object );
}
void assign0( PyObject *object )
{
assertObject( object );
PyObject *old_object = this->object;
this->object = INCREASE_REFCOUNT( object );
// Free old value if any available and owned.
Py_XDECREF( old_object );
}
void assign1( PyObject *object )
{
assertObject( object );
PyObject *old_object = this->object;
this->object = object;
// Free old value if any available and owned.
Py_XDECREF( old_object );
}
#if PYTHON_VERSION >= 300
void del( bool tolerant )
{
if ( this->object )
{
Py_DECREF( this->object );
}
else if ( !tolerant )
{
PyErr_Format( PyExc_NameError, "free variable '%s' referenced before assignment in enclosing scope", Nuitka_String_AsString( this->var_name ) );
throw PythonException();
}
this->object = NULL;
}
#endif
inline PyObject *getVarName() const
{
return this->var_name;
}
PyObject *var_name;
PyObject *object;
int ref_count;
private:
PyObjectSharedStorage( const PyObjectSharedStorage & ) { assert( false ); };
};
class PyObjectSharedLocalVariable
{
public:
explicit PyObjectSharedLocalVariable( PyObject *var_name, PyObject *object = NULL )
{
this->storage = new PyObjectSharedStorage( var_name, object );
}
explicit PyObjectSharedLocalVariable() {
this->storage = NULL;
};
~PyObjectSharedLocalVariable()
{
if ( this->storage )
{
assert( this->storage->ref_count > 0 );
this->storage->ref_count -= 1;
if ( this->storage->ref_count == 0 )
{
delete this->storage;
}
}
}
void setVariableNameAndValue( PyObject *var_name, PyObject *object )
{
this->setVariableName( var_name );
this->assign1( object );
}
void setVariableName( PyObject *var_name )
{
assert( this->storage == NULL );
this->storage = new PyObjectSharedStorage( var_name, NULL );
}
void shareWith( const PyObjectSharedLocalVariable &other )
{
assert( this->storage == NULL );
assert( other.storage != NULL );
this->storage = other.storage;
this->storage->ref_count += 1;
}
void assign0( PyObject *object ) const
{
this->storage->assign0( object );
}
void assign1( PyObject *object ) const
{
this->storage->assign1( object );
}
#if PYTHON_VERSION >= 300
void del( bool tolerant ) const
{
this->storage->del( tolerant );
}
#endif
PyObject *asObject0() const
{
assert( this->storage );
if ( this->storage->object == NULL )
{
PyErr_Format(
PyExc_UnboundLocalError,
"free variable '%s' referenced before assignment in enclosing scope",
Nuitka_String_AsString( this->storage->getVarName() )
);
throw PythonException();
}
if ( Py_REFCNT( this->storage->object ) == 0 )
{
PyErr_Format(
PyExc_UnboundLocalError,
"free variable '%s' referenced after its finalization in enclosing scope",
Nuitka_String_AsString( this->storage->getVarName() )
);
throw PythonException();
}
return this->storage->object;
}
PyObject *asObject1() const
{
return INCREASE_REFCOUNT( this->asObject0() );
}
bool isInitialized() const
{
return this->storage->object != NULL;
}
PyObject *getVariableName() const
{
return this->storage->var_name;
}
PyObject *updateLocalsDict( PyObject *locals_dict ) const
{
if ( this->isInitialized() )
{
#if PYTHON_VERSION < 300
int status = PyDict_SetItem(
#else
int status = PyObject_SetItem(
#endif
locals_dict,
this->getVariableName(),
this->asObject0()
);
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
return locals_dict;
}
PyObject *updateLocalsDir( PyObject *locals_list ) const
{
assert( PyList_Check( locals_list ) );
if ( this->isInitialized() )
{
int status = PyList_Append(
locals_list,
this->getVariableName()
);
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
return locals_list;
}
protected:
PyObjectSharedStorage *storage;
private:
PyObjectSharedLocalVariable( const PyObjectSharedLocalVariable & ) { assert( false ); };
};
class PyObjectClosureVariable : public PyObjectSharedLocalVariable
{
public:
explicit PyObjectClosureVariable()
{
this->storage = NULL;
}
PyObject *asObject0() const
{
assert( this->storage );
if ( this->storage->object == NULL )
{
PyErr_Format(
PyExc_NameError,
"free variable '%s' referenced before assignment in enclosing scope",
Nuitka_String_AsString( this->storage->getVarName() )
);
throw PythonException();
}
if ( Py_REFCNT( this->storage->object ) == 0 )
{
PyErr_Format(
PyExc_UnboundLocalError,
"free variable '%s' referenced after its finalization in enclosing scope",
Nuitka_String_AsString( this->storage->getVarName() )
);
throw PythonException();
}
return this->storage->object;
}
protected:
PyObjectClosureVariable( const PyObjectClosureVariable & ) { assert( false ); }
};
class PyObjectSharedTempStorage
{
public:
explicit PyObjectSharedTempStorage( PyObject *object )
{
assert( object == NULL || Py_REFCNT( object ) > 0 );
this->object = object;
this->ref_count = 1;
}
~PyObjectSharedTempStorage()
{
Py_XDECREF( this->object );
}
void assign0( PyObject *object )
{
assertObject( object );
PyObject *old_object = this->object;
this->object = INCREASE_REFCOUNT( object );
// Free old value.
Py_XDECREF( old_object );
}
void assign1( PyObject *object )
{
assertObject( object );
PyObject *old_object = this->object;
this->object = object;
// Free old value.
Py_XDECREF( old_object );
}
void del( bool tolerant )
{
// TODO: Tolerance, what would it mean here.
Py_XDECREF( this->object );
this->object = NULL;
}
PyObject *object;
int ref_count;
private:
PyObjectSharedTempStorage( const PyObjectSharedTempStorage & ) { assert( false ); };
};
class PyObjectSharedTempVariable
{
public:
explicit PyObjectSharedTempVariable( PyObject *object )
{
this->storage = new PyObjectSharedTempStorage( object );
}
explicit PyObjectSharedTempVariable()
{
this->storage = NULL;
};
~PyObjectSharedTempVariable()
{
if ( this->storage )
{
assert( this->storage->ref_count > 0 );
this->storage->ref_count -= 1;
if ( this->storage->ref_count == 0 )
{
delete this->storage;
}
}
}
void shareWith( const PyObjectSharedTempVariable &other )
{
assert( this->storage == NULL );
assert( other.storage != NULL );
this->storage = other.storage;
this->storage->ref_count += 1;
}
void assign0( PyObject *object ) const
{
this->storage->assign0( object );
}
void assign1( PyObject *object ) const
{
this->storage->assign1( object );
}
void del( bool tolerant ) const
{
this->storage->del( tolerant );
}
PyObject *asObject0() const
{
assert( this->storage );
assertObject( this->storage->object );
return this->storage->object;
}
PyObject *asObject1() const
{
return INCREASE_REFCOUNT( this->asObject0() );
}
bool isInitialized() const
{
return this->storage->object != NULL;
}
#if PYTHON_VERSION >= 340
PyObject *updateLocalsDict( PyObject *var_name, PyObject *locals_dict ) const
{
if ( this->isInitialized() )
{
#if PYTHON_VERSION < 300
int status = PyDict_SetItem(
#else
int status = PyObject_SetItem(
#endif
locals_dict,
var_name,
this->asObject0()
);
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
return locals_dict;
}
PyObject *updateLocalsDir( PyObject *var_name, PyObject *locals_list ) const
{
assert( PyList_Check( locals_list ) );
if ( this->isInitialized() )
{
int status = PyList_Append(
locals_list,
var_name
);
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
return locals_list;
}
#endif
protected:
PyObjectSharedTempStorage *storage;
private:
PyObjectSharedTempVariable( const PyObjectSharedTempVariable & ) { assert( false ); };
};
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/threading.hpp 0000644 0001750 0001750 00000003772 12265264105 023342 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_THREADING_H__
#define __NUITKA_THREADING_H__
#if PYTHON_VERSION < 300
// We share this with CPython bytecode main loop.
PyAPI_DATA(volatile int) _Py_Ticker;
#else
extern volatile int _Py_Ticker;
#define _Py_CheckInterval 20
#endif
NUITKA_MAY_BE_UNUSED static void CONSIDER_THREADING( void )
{
// Decrease ticker
if ( --_Py_Ticker < 0 )
{
_Py_Ticker = _Py_CheckInterval;
int res = Py_MakePendingCalls();
if (unlikely( res < 0 ))
{
throw PythonException();
}
PyThreadState *tstate = PyThreadState_GET();
assert( tstate );
if ( PyEval_ThreadsInitialized() )
{
// Release and acquire the GIL, it's very inefficient, because we
// don't even know if it makes sense to do it. A controlling thread
// should be used to determine if it's needed at all.
PyEval_SaveThread();
PyEval_AcquireThread( tstate );
}
if (unlikely( tstate->async_exc != NULL ))
{
PyObjectTemporary tmp_async_exc( tstate->async_exc );
tstate->async_exc = NULL;
throw PythonException( tmp_async_exc.asObject0() );
}
}
}
#endif
Nuitka-0.5.0.1/nuitka/build/include/nuitka/variables_parameters.hpp 0000644 0001750 0001750 00000015776 12265264105 025577 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#ifndef __NUITKA_VARIABLES_PARAMETERS_H__
#define __NUITKA_VARIABLES_PARAMETERS_H__
class PyObjectLocalParameterVariableWithDel
{
public:
explicit PyObjectLocalParameterVariableWithDel( PyObject *var_name, PyObject *object )
{
assertObject( var_name );
assertObject( object );
this->var_name = var_name;
this->object = object;
}
explicit PyObjectLocalParameterVariableWithDel()
{
this->var_name = NULL;
this->object = NULL;
}
void setVariableNameAndValue( PyObject *var_name, PyObject *object )
{
assertObject( var_name );
assert( this->var_name == NULL);
this->var_name = var_name;
assertObject( object );
assert( this->object == NULL);
this->object = object;
}
~PyObjectLocalParameterVariableWithDel()
{
assertObject( this->var_name );
Py_XDECREF( this->object );
}
void assign0( PyObject *object )
{
assertObject( object );
PyObject *old_object = this->object;
this->object = INCREASE_REFCOUNT( object );
// Free old value if any available and owned.
Py_XDECREF( old_object );
}
void assign1( PyObject *object )
{
assertObject( object );
PyObject *old_object = this->object;
this->object = object;
// Free old value if any available and owned.
Py_XDECREF( old_object );
}
PyObject *asObject0() const
{
if ( this->object == NULL )
{
PyErr_Format( PyExc_UnboundLocalError, "local variable '%s' referenced before assignment", Nuitka_String_AsString( this->var_name ) );
throw PythonException();
}
assertObject( this->object );
return this->object;
}
PyObject *asObject1() const
{
return INCREASE_REFCOUNT( this->asObject0() );
}
bool isInitialized() const
{
return this->object != NULL;
}
void del( bool tolerant )
{
if (unlikely( this->object == NULL ))
{
if ( tolerant == false )
{
PyErr_Format( PyExc_UnboundLocalError, "local variable '%s' referenced before assignment", Nuitka_String_AsString( this->var_name ) );
throw PythonException();
}
}
else
{
assertObject( this->object );
Py_DECREF( this->object );
this->object = NULL;
}
}
PyObject *getVariableName() const
{
assertObject( this->var_name );
return this->var_name;
}
PyObject *updateLocalsDict( PyObject *locals_dict ) const
{
if ( this->isInitialized() )
{
int status = PyDict_SetItem(
locals_dict,
this->getVariableName(),
this->asObject0()
);
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
return locals_dict;
}
PyObject *updateLocalsDir( PyObject *locals_list ) const
{
assert( PyList_Check( locals_list ) );
if ( this->isInitialized() )
{
int status = PyList_Append(
locals_list,
this->getVariableName()
);
if (unlikely( status == -1 ))
{
throw PythonException();
}
}
return locals_list;
}
private:
PyObjectLocalParameterVariableWithDel( const PyObjectLocalParameterVariableWithDel &other ) { assert( false ); }
PyObject *var_name;
PyObject *object;
};
class PyObjectLocalParameterVariableNoDel
{
public:
explicit PyObjectLocalParameterVariableNoDel( PyObject *var_name, PyObject *object )
{
assertObject( object );
assertObject( var_name );
this->var_name = var_name;
this->object = object;
}
explicit PyObjectLocalParameterVariableNoDel()
{
this->var_name = NULL;
this->object = NULL;
}
void setVariableNameAndValue( PyObject *var_name, PyObject *object )
{
assertObject( var_name );
assert( this->var_name == NULL);
this->var_name = var_name;
assertObject( object );
assert( this->object == NULL);
this->object = object;
}
~PyObjectLocalParameterVariableNoDel()
{
assertObject( this->object );
assertObject( this->var_name );
Py_DECREF( this->object );
}
void assign0( PyObject *object )
{
assertObject( object );
assertObject( this->object );
PyObject *old_object = this->object;
this->object = INCREASE_REFCOUNT( object );
// Free old value if any available and owned.
Py_DECREF( old_object );
}
void assign1( PyObject *object )
{
assertObject( object );
assertObject( this->object );
PyObject *old_object = this->object;
this->object = object;
// Free old value if any available and owned.
Py_DECREF( old_object );
}
PyObject *asObject0() const
{
assertObject( this->object );
return this->object;
}
PyObject *asObject1() const
{
return INCREASE_REFCOUNT( this->asObject0() );
}
PyObject *getVariableName() const
{
assertObject( this->var_name );
return this->var_name;
}
PyObject *updateLocalsDict( PyObject *locals_dict ) const
{
int status = PyDict_SetItem(
locals_dict,
this->getVariableName(),
this->asObject0()
);
if (unlikely( status == -1 ))
{
throw PythonException();
}
return locals_dict;
}
PyObject *updateLocalsDir( PyObject *locals_list ) const
{
assert( PyList_Check( locals_list ) );
int status = PyList_Append(
locals_list,
this->getVariableName()
);
if (unlikely( status == -1 ))
{
throw PythonException();
}
return locals_list;
}
private:
PyObjectLocalParameterVariableNoDel( const PyObjectLocalParameterVariableNoDel &other ) { assert( false ); }
PyObject *var_name;
PyObject *object;
};
#endif
Nuitka-0.5.0.1/nuitka/build/SconsInterface.py 0000644 0001750 0001750 00000011113 12265264105 021212 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Scons interface.
Interaction with scons. Find the binary, and run it with a set of given
options.
"""
import os
import subprocess
import sys
from nuitka import Options, Tracing, Utils
def getSconsDataPath():
return Utils.dirname(__file__)
def getSconsInlinePath():
return Utils.joinpath(getSconsDataPath(), "inline_copy")
def getSconsBinaryCall():
""" Return a way to execute Scons.
Using potentially inline copy if no system Scons is available
or if we are on Windows.
"""
if Utils.isFile("/usr/bin/scons"):
return ["/usr/bin/scons"]
else:
return [
getPython2ExePath(),
Utils.joinpath(getSconsInlinePath(), "bin", "scons.py")
]
def _getPython2ExePathWindows():
# Shortcuts for the default installation directories, to avoid going to
# registry at all.
if os.path.isfile(r"c:\Python27\python.exe"):
return r"c:\Python27\python.exe"
elif os.path.isfile(r"c:\Python26\python.exe"):
return r"c:\Python26\python.exe"
# Windows only code, pylint: disable=E0602,F0401
try:
import _winreg as winreg
except ImportError:
import winreg # lint:ok
for search in ("2.7", "2.6"):
for arch_key in 0, winreg.KEY_WOW64_32KEY, winreg.KEY_WOW64_64KEY:
try:
key = winreg.OpenKey(
winreg.HKEY_LOCAL_MACHINE,
r"SOFTWARE\Python\PythonCore\%s\InstallPath" % search,
0,
winreg.KEY_READ | arch_key
)
return Utils.joinpath(
winreg.QueryValue(key, ''),
"python.exe"
)
except WindowsError: # lint:ok
pass
def getPython2ExePath():
""" Find a way to call Python2. Scons needs it."""
if Utils.python_version < 300:
return sys.executable
elif Utils.getOS() == "Windows":
python_exe = _getPython2ExePathWindows()
if python_exe is not None:
return python_exe
else:
sys.exit("""\
Error, need to find Python2 executable under C:\\Python26 or \
C:\\Python27 to execute scons which is not Python3 compatible.""")
elif os.path.exists("/usr/bin/python2"):
return "python2"
else:
return "python"
def runScons(options, quiet):
# For the scons file to find the static C++ files and include path. The
# scons file is unable to use __file__ for the task.
os.environ["NUITKA_SCONS"] = getSconsDataPath()
if Utils.getOS() == "Windows":
# On Windows this Scons variable must be set by us.
os.environ["SCONS_LIB_DIR"] = Utils.joinpath(
getSconsInlinePath(),
"lib",
"scons-2.3.0"
)
# Also, for MinGW we can avoid the user having to add the path if he
# used the default path or installed it on the same drive by appending
# to the PATH variable before executing scons.
os.environ["PATH"] += r";\MinGW\bin;C:\MinGW\bin"
scons_command = getSconsBinaryCall()
if quiet:
scons_command.append("--quiet")
scons_command += [
# The scons file
"-f",
Utils.joinpath(getSconsDataPath(), "SingleExe.scons"),
# Parallel compilation.
"--jobs",
str(Options.getJobLimit()),
# Do not warn about deprecations of Scons
"--warn=no-deprecated",
# Don't load "site_scons" at all.
"--no-site-dir",
]
if Options.isShowScons():
scons_command.append("--debug=explain")
# Option values to provide to scons.
for key, value in options.items():
scons_command += [key + "=" + value]
if Options.isShowScons():
Tracing.printLine("Scons command:", " ".join(scons_command))
return 0 == subprocess.call(scons_command)
Nuitka-0.5.0.1/nuitka/build/static_src/ 0000755 0001750 0001750 00000000000 12265271051 020071 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/build/static_src/InspectPatcher.cpp 0000644 0001750 0001750 00000015155 12265264105 023522 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// This is responsible for updating parts of CPython to better work with Nuitka
// by replacing CPython implementations with enhanced versions.
#include "nuitka/prelude.hpp"
#if PYTHON_VERSION >= 300
extern PyObject *const_str_plain_inspect;
extern PyObject *const_str_plain_site;
extern PyObject *const_int_0;
static PyObject *module_inspect;
static char *kwlist[] = { (char *)"object", NULL };
static PyObject *old_getgeneratorstate = NULL;
static PyObject *_inspect_getgeneratorstate_replacement( PyObject *self, PyObject *args, PyObject *kwds )
{
PyObject *object;
if ( !PyArg_ParseTupleAndKeywords( args, kwds, "O:getgeneratorstate", kwlist, &object, NULL ))
{
return NULL;
}
if ( Nuitka_Generator_Check( object ) )
{
Nuitka_GeneratorObject *generator = (Nuitka_GeneratorObject *)object;
if ( generator->m_running )
{
return PyObject_GetAttrString( module_inspect, "GEN_RUNNING" );
}
else if ( generator->m_status == status_Finished )
{
return PyObject_GetAttrString( module_inspect, "GEN_CLOSED" );
}
else if ( generator->m_status == status_Unused )
{
return PyObject_GetAttrString( module_inspect, "GEN_CREATED" );
}
else
{
return PyObject_GetAttrString( module_inspect, "GEN_SUSPENDED" );
}
}
else
{
return old_getgeneratorstate->ob_type->tp_call( old_getgeneratorstate, args, kwds );
}
}
#endif
#if PYTHON_VERSION >= 300
static PyMethodDef _method_def_inspect_getgeneratorstate_replacement =
{
"isframe",
(PyCFunction)_inspect_getgeneratorstate_replacement,
METH_VARARGS | METH_KEYWORDS,
NULL
};
// Replace inspect functions with ones that accept compiled types too.
static void patchInspectModule( void )
{
#if PYTHON_VERSION >= 300
#ifdef _NUITKA_EXE
// May need to import the "site" module, because otherwise the patching can
// fail with it being unable to load it.
if ( Py_NoSiteFlag == 0 )
{
try
{
IMPORT_MODULE( const_str_plain_site, Py_None, Py_None, const_tuple_empty, const_int_0 );
}
catch( PythonException & )
{
PyErr_Clear();
// Ignore ImportError, site is not a must.
}
}
#endif
try
{
module_inspect = IMPORT_MODULE( const_str_plain_inspect, Py_None, Py_None, const_tuple_empty, const_int_0 );
}
catch( PythonException &e )
{
e.toPython();
PyErr_PrintEx( 0 );
Py_Exit( 1 );
}
assertObject( module_inspect );
// Patch "inspect.getgeneratorstate" unless it is already patched.
old_getgeneratorstate = PyObject_GetAttrString( module_inspect, "getgeneratorstate" );
assertObject( old_getgeneratorstate );
if ( PyFunction_Check( old_getgeneratorstate ) )
{
PyObject *inspect_getgeneratorstate_replacement = PyCFunction_New( &_method_def_inspect_getgeneratorstate_replacement, NULL );
assertObject( inspect_getgeneratorstate_replacement );
PyObject_SetAttrString( module_inspect, "getgeneratorstate", inspect_getgeneratorstate_replacement );
}
#endif
}
#endif
extern int Nuitka_IsInstance( PyObject *inst, PyObject *cls );
static PyObject *_builtin_isinstance_replacement( PyObject *self, PyObject *args )
{
PyObject *inst, *cls;
if (unlikely( PyArg_UnpackTuple(args, "isinstance", 2, 2, &inst, &cls) == 0 ))
{
return NULL;
}
int res = Nuitka_IsInstance( inst, cls );
if (unlikely( res < 0 ))
{
return NULL;
}
return PyBool_FromLong( res );
}
static PyMethodDef _method_def_builtin_isinstance_replacement =
{
"isinstance",
(PyCFunction)_builtin_isinstance_replacement,
METH_VARARGS,
NULL
};
extern PyModuleObject *module_builtin;
void patchBuiltinModule()
{
assertObject( (PyObject *)module_builtin );
// Patch "inspect.isfunction" unless it is already patched.
PyObject *old_isinstance = PyObject_GetAttrString( (PyObject *)module_builtin, "isinstance" );
assertObject( old_isinstance );
// TODO: Find safe criterion, these was a C method before
if ( true || PyFunction_Check( old_isinstance ))
{
PyObject *builtin_isinstance_replacement = PyCFunction_New( &_method_def_builtin_isinstance_replacement, NULL );
assertObject( builtin_isinstance_replacement );
PyObject_SetAttrString( (PyObject *)module_builtin, "isinstance", builtin_isinstance_replacement );
}
Py_DECREF( old_isinstance );
#if PYTHON_VERSION >= 300
patchInspectModule();
#endif
}
static richcmpfunc original_PyType_tp_richcompare = NULL;
static PyObject *Nuitka_type_tp_richcompare( PyObject *a, PyObject *b, int op )
{
if (likely( op == Py_EQ || op == Py_NE ))
{
if ( a == (PyObject *)&Nuitka_Function_Type )
{
a = (PyObject *)&PyFunction_Type;
}
else if ( a == (PyObject *)&Nuitka_Method_Type )
{
a = (PyObject *)&PyMethod_Type;
}
else if ( a == (PyObject *)&Nuitka_Generator_Type )
{
a = (PyObject *)&PyGen_Type;
}
if ( b == (PyObject *)&Nuitka_Function_Type )
{
b = (PyObject *)&PyFunction_Type;
}
else if ( b == (PyObject *)&Nuitka_Method_Type )
{
b = (PyObject *)&PyMethod_Type;
}
else if ( b == (PyObject *)&Nuitka_Generator_Type )
{
b = (PyObject *)&PyGen_Type;
}
}
assertObject( a );
assertObject( b );
assert( original_PyType_tp_richcompare );
return original_PyType_tp_richcompare( a, b, op );
}
void patchTypeComparison()
{
if ( original_PyType_tp_richcompare == NULL )
{
original_PyType_tp_richcompare = PyType_Type.tp_richcompare;
PyType_Type.tp_richcompare = Nuitka_type_tp_richcompare;
}
}
Nuitka-0.5.0.1/nuitka/build/static_src/x64_ucontext_src/ 0000755 0001750 0001750 00000000000 12265271051 023312 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/build/static_src/x64_ucontext_src/swapfiber.S 0000644 0001750 0001750 00000004521 12265264105 025424 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// Implementation of process context switch "swapFiber" for x64
#define oRBP 120
#define oRSP 160
#define oRBX 128
#define oR8 40
#define oR9 48
#define oR12 72
#define oR13 80
#define oR14 88
#define oR15 96
#define oRDI 104
#define oRSI 112
#define oRDX 136
#define oRCX 152
#define oRIP 168
#define oFPREGS 224
#define oFPREGSMEM 424
.globl swapFiber
swapFiber:
// Save the preserved used for passing args and the return address.
movq %rbx, oRBX(%rdi)
movq %rbp, oRBP(%rdi)
movq %r12, oR12(%rdi)
movq %r13, oR13(%rdi)
movq %r14, oR14(%rdi)
movq %r15, oR15(%rdi)
movq %rdi, oRDI(%rdi)
movq %rsi, oRSI(%rdi)
movq %rdx, oRDX(%rdi)
movq %rcx, oRCX(%rdi)
movq %r8, oR8(%rdi)
movq %r9, oR9(%rdi)
movq (%rsp), %rcx
movq %rcx, oRIP(%rdi)
// Make room for return address.
leaq 8(%rsp), %rcx
movq %rcx, oRSP(%rdi)
// Separate floating-point register content memory
leaq oFPREGSMEM(%rdi), %rcx
movq %rcx, oFPREGS(%rdi)
// Load the new stack pointer and the preserved registers.
movq oRSP(%rsi), %rsp
movq oRBX(%rsi), %rbx
movq oRBP(%rsi), %rbp
movq oR12(%rsi), %r12
movq oR13(%rsi), %r13
movq oR14(%rsi), %r14
movq oR15(%rsi), %r15
// We should return to the address provided, put it on top of stack.
movq oRIP(%rsi), %rcx
pushq %rcx
// Setup registers for arguments.
movq oRDI(%rsi), %rdi
movq oRDX(%rsi), %rdx
movq oRCX(%rsi), %rcx
movq oR8(%rsi), %r8
movq oR9(%rsi), %r9
// Setup %rsi.
movq oRSI(%rsi), %rsi
ret
Nuitka-0.5.0.1/nuitka/build/static_src/x64_ucontext_src/fibers_x64.cpp 0000644 0001750 0001750 00000003442 12265264105 025776 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// Implementation of process context switch for x64.
#include "nuitka/prelude.hpp"
#define STACK_SIZE (1024*1024)
// Keep one stack around to avoid the overhead of repeated malloc/free in
// case of frequent instantiations in a loop.
static void *last_stack = NULL;
void initFiber( Fiber *to )
{
to->f_context.uc_stack.ss_sp = NULL;
to->f_context.uc_link = NULL;
to->start_stack = NULL;
}
void prepareFiber( Fiber *to, void *code, unsigned long arg )
{
int res = getcontext( &to->f_context );
assert( res == 0 );
to->f_context.uc_stack.ss_size = STACK_SIZE;
to->f_context.uc_stack.ss_sp = last_stack ? last_stack : malloc( STACK_SIZE );
to->start_stack = to->f_context.uc_stack.ss_sp;
to->f_context.uc_link = NULL;
last_stack = NULL;
makecontext( &to->f_context, (void (*)())code, 1, (unsigned long)arg );
}
void releaseFiber( Fiber *to )
{
if ( last_stack == NULL )
{
last_stack = to->start_stack;
}
else
{
free( to->start_stack );
}
}
Nuitka-0.5.0.1/nuitka/build/static_src/CompiledFunctionType.cpp 0000644 0001750 0001750 00000075061 12265264105 024714 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#include "nuitka/prelude.hpp"
#include "nuitka/compiled_method.hpp"
// Needed for offsetof
#include
// tp_descr_get slot, bind a function to an object.
static PyObject *Nuitka_Function_descr_get( PyObject *function, PyObject *object, PyObject *klass )
{
assert( Nuitka_Function_Check( function ) );
#if PYTHON_VERSION >= 300
if ( object == NULL || object == Py_None )
{
return INCREASE_REFCOUNT( function );
}
#endif
return Nuitka_Method_New(
(Nuitka_FunctionObject *)function,
object == Py_None ? NULL : object,
klass
);
}
// tp_repr slot, decide how a function shall be output
static PyObject *Nuitka_Function_tp_repr( Nuitka_FunctionObject *function )
{
#if PYTHON_VERSION < 300
return PyString_FromFormat(
#else
return PyUnicode_FromFormat(
#endif
"",
#if PYTHON_VERSION < 330
Nuitka_String_AsString( function->m_name ),
#else
Nuitka_String_AsString( function->m_qualname ),
#endif
function
);
}
static PyObject *Nuitka_Function_tp_call( Nuitka_FunctionObject *function, PyObject *args, PyObject *kw )
{
assertObject( args );
assert( PyTuple_Check( args ) );
if ( kw || function->m_direct_arg_parser == NULL )
{
return function->m_code(
function,
&PyTuple_GET_ITEM( args, 0 ),
PyTuple_GET_SIZE( args ),
kw
);
}
else
{
return function->m_direct_arg_parser(
function,
&PyTuple_GET_ITEM( args, 0 ),
int( PyTuple_GET_SIZE( args ) )
);
}
}
static long Nuitka_Function_tp_traverse( PyObject *function, visitproc visit, void *arg )
{
// TODO: Identify the impact of not visiting owned objects and/or if it
// could be NULL instead. The methodobject visits its self and module. I
// understand this is probably so that back references of this function to
// its upper do not make it stay in the memory. A specific test if that
// works might be needed.
return 0;
}
static long Nuitka_Function_tp_hash( Nuitka_FunctionObject *function )
{
return function->m_counter;
}
static PyObject *Nuitka_Function_get_name( Nuitka_FunctionObject *object )
{
return INCREASE_REFCOUNT( object->m_name );
}
static int Nuitka_Function_set_name( Nuitka_FunctionObject *object, PyObject *value )
{
#if PYTHON_VERSION < 300
if (unlikely( value == NULL || PyString_Check( value ) == 0 ))
#else
if (unlikely( value == NULL || PyUnicode_Check( value ) == 0 ))
#endif
{
PyErr_Format( PyExc_TypeError, "__name__ must be set to a string object" );
return -1;
}
PyObject *old = object->m_name;
object->m_name = INCREASE_REFCOUNT( value );
Py_DECREF( old );
return 0;
}
#if PYTHON_VERSION >= 330
static PyObject *Nuitka_Function_get_qualname( Nuitka_FunctionObject *object )
{
return INCREASE_REFCOUNT( object->m_qualname );
}
static int Nuitka_Function_set_qualname( Nuitka_FunctionObject *object, PyObject *value )
{
if (unlikely( value == NULL || PyUnicode_Check( value ) == 0 ))
{
PyErr_Format( PyExc_TypeError, "__qualname__ must be set to a string object" );
return -1;
}
PyObject *old = object->m_qualname;
object->m_qualname = INCREASE_REFCOUNT( value );
Py_DECREF( old );
return 0;
}
#endif
static PyObject *Nuitka_Function_get_doc( Nuitka_FunctionObject *object )
{
return INCREASE_REFCOUNT( object->m_doc );
}
static int Nuitka_Function_set_doc( Nuitka_FunctionObject *object, PyObject *value )
{
if ( value == Py_None || value == NULL )
{
object->m_doc = Py_None;
}
#if PYTHON_VERSION < 300
else if (unlikely( PyString_Check( value ) == 0 ))
{
PyErr_Format( PyExc_TypeError, "__name__ must be set to a string object" );
return -1;
}
#else
else if (unlikely( PyUnicode_Check( value ) == 0 ))
{
PyErr_Format( PyExc_TypeError, "__name__ must be set to a string object" );
return -1;
}
#endif
else
{
object->m_doc = INCREASE_REFCOUNT( value );
}
return 0;
}
static PyObject *Nuitka_Function_get_dict( Nuitka_FunctionObject *object )
{
if ( object->m_dict == NULL )
{
object->m_dict = PyDict_New();
}
return INCREASE_REFCOUNT( object->m_dict );
}
static int Nuitka_Function_set_dict( Nuitka_FunctionObject *object, PyObject *value )
{
if (unlikely( value == NULL ))
{
PyErr_Format( PyExc_TypeError, "function's dictionary may not be deleted");
return -1;
}
if (likely( PyDict_Check( value ) ))
{
PyObject *old = object->m_dict;
object->m_dict = INCREASE_REFCOUNT( value );
Py_XDECREF( old );
return 0;
}
else
{
PyErr_SetString( PyExc_TypeError, "setting function's dictionary to a non-dict" );
return -1;
}
}
static PyObject *Nuitka_Function_get_code( Nuitka_FunctionObject *object )
{
return INCREASE_REFCOUNT( (PyObject *)object->m_code_object );
}
static int Nuitka_Function_set_code( Nuitka_FunctionObject *object, PyObject *value )
{
PyErr_Format( PyExc_RuntimeError, "__code__ is not writable in Nuitka" );
return -1;
}
static PyObject *Nuitka_Function_get_defaults( Nuitka_FunctionObject *object )
{
return INCREASE_REFCOUNT( (PyObject *)object->m_defaults );
}
static int Nuitka_Function_set_defaults( Nuitka_FunctionObject *object, PyObject *value )
{
if ( value == NULL )
{
value = Py_None;
}
if (unlikely( value != Py_None && PyTuple_Check( value ) == false ))
{
PyErr_Format( PyExc_TypeError, "__defaults__ must be set to a tuple object" );
return -1;
}
if ( object->m_defaults == Py_None && value != Py_None )
{
PyErr_Format( PyExc_TypeError, "Nuitka doesn't support __defaults__ size changes" );
return -1;
}
if ( object->m_defaults != Py_None && ( value == Py_None || PyTuple_Size( object->m_defaults ) != PyTuple_Size( value ) ) )
{
PyErr_Format( PyExc_TypeError, "Nuitka doesn't support __defaults__ size changes" );
return -1;
}
PyObject *old = object->m_defaults;
object->m_defaults = INCREASE_REFCOUNT( value );
Py_DECREF( old );
return 0;
}
#if PYTHON_VERSION >= 300
static PyObject *Nuitka_Function_get_kwdefaults( Nuitka_FunctionObject *object )
{
return INCREASE_REFCOUNT( (PyObject *)object->m_kwdefaults );
}
static int Nuitka_Function_set_kwdefaults( Nuitka_FunctionObject *object, PyObject *value )
{
if ( value == NULL )
{
value = Py_None;
}
if (unlikely( value != Py_None && PyDict_Check( value ) == false ))
{
PyErr_Format( PyExc_TypeError, "__kwdefaults__ must be set to a dict object" );
return -1;
}
PyObject *old = object->m_kwdefaults;
object->m_kwdefaults = INCREASE_REFCOUNT( value );
Py_DECREF( old );
return 0;
}
static PyObject *Nuitka_Function_get_annotations( Nuitka_FunctionObject *object )
{
if ( object->m_annotations == NULL )
{
object->m_annotations = PyDict_New();
}
return INCREASE_REFCOUNT( (PyObject *)object->m_annotations );
}
static int Nuitka_Function_set_annotations( Nuitka_FunctionObject *object, PyObject *value )
{
// CPython silently converts None to empty dictionary.
if ( value == Py_None || value == NULL )
{
value = PyDict_New();
}
if (unlikely( PyDict_Check( value ) == false ))
{
PyErr_Format( PyExc_TypeError, "__annotations__ must be set to a dict object" );
return -1;
}
PyObject *old = object->m_annotations;
object->m_annotations = INCREASE_REFCOUNT( value );
Py_XDECREF( old );
return 0;
}
#endif
static int Nuitka_Function_set_globals( Nuitka_FunctionObject *object, PyObject *value )
{
PyErr_Format( PyExc_TypeError, "readonly attribute" );
return -1;
}
static PyObject *Nuitka_Function_get_globals( Nuitka_FunctionObject *object )
{
return INCREASE_REFCOUNT( PyModule_GetDict( object->m_module ) );
}
extern PyObject *const_str_plain___module__;
static int Nuitka_Function_set_module( Nuitka_FunctionObject *object, PyObject *value )
{
if ( object->m_dict == NULL )
{
object->m_dict = PyDict_New();
}
if ( value == NULL )
{
value = Py_None;
}
return PyDict_SetItem( object->m_dict, const_str_plain___module__, value );
}
static PyObject *Nuitka_Function_get_module( Nuitka_FunctionObject *object )
{
if ( object->m_dict )
{
PyObject *result = PyDict_GetItem( object->m_dict, const_str_plain___module__ );
if ( result != NULL )
{
return INCREASE_REFCOUNT( result );
}
}
return MODULE_NAME( object->m_module );
}
static PyGetSetDef Nuitka_Function_getset[] =
{
#if PYTHON_VERSION >= 330
{ (char *)"__qualname__", (getter)Nuitka_Function_get_qualname, (setter)Nuitka_Function_set_qualname},
#endif
#if PYTHON_VERSION < 300
{ (char *)"func_name", (getter)Nuitka_Function_get_name, (setter)Nuitka_Function_set_name, NULL },
#endif
{ (char *)"__name__" , (getter)Nuitka_Function_get_name, (setter)Nuitka_Function_set_name, NULL },
#if PYTHON_VERSION < 300
{ (char *)"func_doc", (getter)Nuitka_Function_get_doc, (setter)Nuitka_Function_set_doc, NULL },
#endif
{ (char *)"__doc__" , (getter)Nuitka_Function_get_doc, (setter)Nuitka_Function_set_doc, NULL },
#if PYTHON_VERSION < 300
{ (char *)"func_dict", (getter)Nuitka_Function_get_dict, (setter)Nuitka_Function_set_dict, NULL },
#endif
{ (char *)"__dict__", (getter)Nuitka_Function_get_dict, (setter)Nuitka_Function_set_dict, NULL },
#if PYTHON_VERSION < 300
{ (char *)"func_code", (getter)Nuitka_Function_get_code, (setter)Nuitka_Function_set_code, NULL },
#endif
{ (char *)"__code__", (getter)Nuitka_Function_get_code, (setter)Nuitka_Function_set_code, NULL },
#if PYTHON_VERSION < 300
{ (char *)"func_defaults", (getter)Nuitka_Function_get_defaults, (setter)Nuitka_Function_set_defaults, NULL },
#endif
{ (char *)"__defaults__", (getter)Nuitka_Function_get_defaults, (setter)Nuitka_Function_set_defaults, NULL },
#if PYTHON_VERSION < 300
{ (char *)"func_globals", (getter)Nuitka_Function_get_globals, (setter)Nuitka_Function_set_globals, NULL },
#endif
{ (char *)"__globals__", (getter)Nuitka_Function_get_globals, (setter)Nuitka_Function_set_globals, NULL },
{ (char *)"__module__", (getter)Nuitka_Function_get_module, (setter)Nuitka_Function_set_module, NULL },
#if PYTHON_VERSION >= 300
{ (char *)"__kwdefaults__", (getter)Nuitka_Function_get_kwdefaults, (setter)Nuitka_Function_set_kwdefaults},
{ (char *)"__annotations__", (getter)Nuitka_Function_get_annotations, (setter)Nuitka_Function_set_annotations},
#endif
{ NULL }
};
static PyObject *Nuitka_Function_reduce( Nuitka_FunctionObject *function )
{
return INCREASE_REFCOUNT( function->m_name );
}
static PyMethodDef Nuitka_Generator_methods[] =
{
{ "__reduce__", (PyCFunction)Nuitka_Function_reduce, METH_NOARGS, NULL },
{ NULL }
};
static void Nuitka_Function_tp_dealloc( Nuitka_FunctionObject *function )
{
Nuitka_GC_UnTrack( function );
if ( function->m_weakrefs != NULL )
{
PyObject_ClearWeakRefs( (PyObject *)function );
}
Py_DECREF( function->m_name );
#if PYTHON_VERSION >= 330
Py_DECREF( function->m_qualname );
#endif
Py_XDECREF( function->m_dict );
Py_DECREF( function->m_defaults );
#if PYTHON_VERSION >= 300
Py_DECREF( function->m_kwdefaults );
Py_XDECREF( function->m_annotations );
#endif
if ( function->m_context )
{
function->m_cleanup( function->m_context );
}
PyObject_GC_Del( function );
}
static const long tp_flags =
Py_TPFLAGS_DEFAULT |
#if PYTHON_VERSION < 300
Py_TPFLAGS_HAVE_WEAKREFS |
#endif
Py_TPFLAGS_HAVE_GC;
PyTypeObject Nuitka_Function_Type =
{
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"compiled_function", // tp_name
sizeof(Nuitka_FunctionObject), // tp_basicsize
0, // tp_itemsize
(destructor)Nuitka_Function_tp_dealloc, // tp_dealloc
0, // tp_print
0, // tp_getattr
0, // tp_setattr
0, // tp_compare
(reprfunc)Nuitka_Function_tp_repr, // tp_repr
0, // tp_as_number
0, // tp_as_sequence
0, // tp_as_mapping
(hashfunc)Nuitka_Function_tp_hash, // tp_hash
(ternaryfunc)Nuitka_Function_tp_call, // tp_call
0, // tp_str
PyObject_GenericGetAttr, // tp_getattro
0, // tp_setattro
0, // tp_as_buffer
tp_flags, // tp_flags
0, // tp_doc
(traverseproc)Nuitka_Function_tp_traverse, // tp_traverse
0, // tp_clear
0, // tp_richcompare
offsetof( Nuitka_FunctionObject, m_weakrefs ), // tp_weaklistoffset
0, // tp_iter
0, // tp_iternext
Nuitka_Generator_methods, // tp_methods
0, // tp_members
Nuitka_Function_getset, // tp_getset
0, // tp_base
0, // tp_dict
Nuitka_Function_descr_get, // tp_descr_get
0, // tp_descr_set
offsetof( Nuitka_FunctionObject, m_dict ), // tp_dictoffset
0, // tp_init
0, // tp_alloc
0, // tp_new
0, // tp_free
0, // tp_is_gc
0, // tp_bases
0, // tp_mro
0, // tp_cache
0, // tp_subclasses
0, // tp_weaklist
0, // tp_del
};
#if PYTHON_VERSION < 300
static inline PyObject *make_kfunction( function_arg_parser code, direct_arg_parser dparse, PyObject *name, PyCodeObject *code_object, PyObject *defaults, PyObject *module, PyObject *doc, void *context, releaser cleanup )
#elif PYTHON_VERSION < 330
static inline PyObject *make_kfunction( function_arg_parser code, direct_arg_parser dparse, PyObject *name, PyCodeObject *code_object, PyObject *defaults, PyObject *kwdefaults, PyObject *annotations, PyObject *module, PyObject *doc, void *context, releaser cleanup )
#else
static inline PyObject *make_kfunction( function_arg_parser code, direct_arg_parser dparse, PyObject *name, PyObject *qualname, PyCodeObject *code_object, PyObject *defaults, PyObject *kwdefaults, PyObject *annotations, PyObject *module, PyObject *doc, void *context, releaser cleanup )
#endif
{
Nuitka_FunctionObject *result = PyObject_GC_New( Nuitka_FunctionObject, &Nuitka_Function_Type );
if (unlikely( result == NULL ))
{
PyErr_Format( PyExc_RuntimeError, "cannot create function %s", Nuitka_String_AsString( name ) );
throw PythonException();
}
result->m_code = code;
result->m_direct_arg_parser = dparse;
result->m_name = INCREASE_REFCOUNT( name );
#if PYTHON_VERSION >= 330
result->m_qualname = INCREASE_REFCOUNT( qualname ? qualname : name );
#endif
result->m_context = context;
result->m_cleanup = cleanup;
assertObject( defaults );
assert( defaults == Py_None || ( PyTuple_Check( defaults ) && PyTuple_Size( defaults ) > 0 ) );
result->m_defaults = defaults;
result->m_defaults_given = defaults == Py_None ? 0 : PyTuple_GET_SIZE( defaults );
#if PYTHON_VERSION >= 300
assert( kwdefaults );
assert( kwdefaults == Py_None || ( PyDict_Check( kwdefaults ) && PyDict_Size( kwdefaults ) > 0 ) );
result->m_kwdefaults = kwdefaults;
assert( annotations == NULL || PyDict_Check( annotations ) );
result->m_annotations = annotations;
#endif
result->m_code_object = code_object;
result->m_module = module;
result->m_doc = doc;
result->m_dict = NULL;
result->m_weakrefs = NULL;
static long Nuitka_Function_counter = 0;
result->m_counter = Nuitka_Function_counter++;
Nuitka_GC_Track( result );
return (PyObject *)result;
}
// Make a function without context.
#if PYTHON_VERSION < 300
PyObject *Nuitka_Function_New( function_arg_parser fparse, direct_arg_parser dparse, PyObject *name, PyCodeObject *code_object, PyObject *defaults, PyObject *module, PyObject *doc )
{
return make_kfunction( fparse, dparse, name, code_object, defaults, module, doc, NULL, NULL );
}
#elif PYTHON_VERSION < 330
PyObject *Nuitka_Function_New( function_arg_parser fparse, direct_arg_parser dparse, PyObject *name, PyCodeObject *code_object, PyObject *defaults, PyObject *kwdefaults, PyObject *annotations, PyObject *module, PyObject *doc )
{
return make_kfunction( fparse, dparse, name, code_object, defaults, kwdefaults, annotations, module, doc, NULL, NULL );
}
#else
PyObject *Nuitka_Function_New( function_arg_parser fparse, direct_arg_parser dparse, PyObject *name, PyObject *qualname, PyCodeObject *code_object, PyObject *defaults, PyObject *kwdefaults, PyObject *annotations, PyObject *module, PyObject *doc )
{
return make_kfunction( fparse, dparse, name, qualname, code_object, defaults, kwdefaults, annotations, module, doc, NULL, NULL );
}
#endif
// Make a function with context.
#if PYTHON_VERSION < 300
PyObject *Nuitka_Function_New( function_arg_parser fparse, direct_arg_parser dparse, PyObject *name, PyCodeObject *code_object, PyObject *defaults, PyObject *module, PyObject *doc, void *context, releaser cleanup )
{
return make_kfunction( fparse, dparse, name, code_object, defaults, module, doc, context, cleanup );
}
#elif PYTHON_VERSION < 330
PyObject *Nuitka_Function_New( function_arg_parser fparse, direct_arg_parser dparse, PyObject *name, PyCodeObject *code_object, PyObject *defaults, PyObject *kwdefaults, PyObject *annotations, PyObject *module, PyObject *doc, void *context, releaser cleanup )
{
return make_kfunction( fparse, dparse, name, code_object, defaults, kwdefaults, annotations, module, doc, context, cleanup );
}
#else
PyObject *Nuitka_Function_New( function_arg_parser fparse, direct_arg_parser dparse, PyObject *name, PyObject *qualname, PyCodeObject *code_object, PyObject *defaults, PyObject *kwdefaults, PyObject *annotations, PyObject *module, PyObject *doc, void *context, releaser cleanup )
{
return make_kfunction( fparse, dparse, name, qualname, code_object, defaults, kwdefaults, annotations, module, doc, context, cleanup );
}
#endif
void ERROR_NO_ARGUMENTS_ALLOWED( Nuitka_FunctionObject *function,
#if PYTHON_VERSION >= 330
PyObject *kw,
#endif
Py_ssize_t given )
{
char const *function_name =
Nuitka_String_AsString( function->m_name );
#if PYTHON_VERSION < 330
PyErr_Format(
PyExc_TypeError,
"%s() takes no arguments (%zd given)",
function_name,
given );
#else
if ( kw == NULL )
{
PyErr_Format(
PyExc_TypeError,
"%s() takes 0 positional arguments but %zd was given",
function_name,
given
);
}
else
{
PyObject *tmp_iter = PyObject_GetIter( kw );
PyObject *tmp_arg_name = PyIter_Next( tmp_iter );
Py_DECREF( tmp_iter );
PyErr_Format( PyExc_TypeError,
"%s() got an unexpected keyword argument '%s'",
function_name,
Nuitka_String_AsString( tmp_arg_name )
);
Py_DECREF( tmp_arg_name );
}
#endif
}
void ERROR_MULTIPLE_VALUES( Nuitka_FunctionObject *function, Py_ssize_t index )
{
char const *function_name =
Nuitka_String_AsString( function->m_name );
PyErr_Format(
PyExc_TypeError,
#if PYTHON_VERSION < 330
"%s() got multiple values for keyword argument '%s'",
#else
"%s() got multiple values for argument '%s'",
#endif
function_name,
Nuitka_String_AsString(
PyTuple_GET_ITEM(
function->m_code_object->co_varnames,
index
)
)
);
}
void ERROR_TOO_FEW_ARGUMENTS( Nuitka_FunctionObject *function,
#if PYTHON_VERSION < 270
Py_ssize_t kw_size,
#endif
Py_ssize_t given )
{
Py_ssize_t required_parameter_count =
function->m_code_object->co_argcount;
if ( function->m_defaults != Py_None )
{
required_parameter_count -= PyTuple_GET_SIZE( function->m_defaults );
}
char const *function_name =
Nuitka_String_AsString( function->m_name );
char const *violation =
( function->m_defaults != Py_None || function->m_code_object->co_flags & CO_VARARGS ) ? "at least" : "exactly";
char const *plural =
required_parameter_count == 1 ? "" : "s";
#if PYTHON_VERSION < 270
if ( kw_size > 0 )
{
PyErr_Format(
PyExc_TypeError,
"%s() takes %s %zd non-keyword argument%s (%zd given)",
function_name,
violation,
required_parameter_count,
plural,
given - function->m_defaults_given
);
}
else
{
PyErr_Format(
PyExc_TypeError,
"%s() takes %s %zd argument%s (%zd given)",
function_name,
violation,
required_parameter_count,
plural,
given
);
}
#else
PyErr_Format(
PyExc_TypeError,
"%s() takes %s %zd argument%s (%zd given)",
function_name,
violation,
required_parameter_count,
plural,
given
);
#endif
}
void ERROR_TOO_MANY_ARGUMENTS( Nuitka_FunctionObject *function,
Py_ssize_t given
#if PYTHON_VERSION < 270
, Py_ssize_t kw_size
#endif
#if PYTHON_VERSION >= 330
, Py_ssize_t kw_only
#endif
)
{
Py_ssize_t top_level_parameter_count = function->m_code_object->co_argcount;
char const *function_name =
Nuitka_String_AsString( function->m_name );
#if PYTHON_VERSION < 330
char const *violation =
function->m_defaults != Py_None ? "at most" : "exactly";
#endif
char const *plural =
top_level_parameter_count == 1 ? "" : "s";
#if PYTHON_VERSION < 270
PyErr_Format(
PyExc_TypeError,
"%s() takes %s %zd %sargument%s (%zd given)",
function_name,
violation,
top_level_parameter_count,
kw_size > 0 ? "non-keyword " : "",
plural,
given
);
#elif PYTHON_VERSION < 300
PyErr_Format(
PyExc_TypeError,
"%s() takes %s %zd argument%s (%zd given)",
function_name,
violation,
top_level_parameter_count,
plural,
given
);
#elif PYTHON_VERSION < 330
PyErr_Format(
PyExc_TypeError,
"%s() takes %s %zd positional argument%s (%zd given)",
function_name,
violation,
top_level_parameter_count,
plural,
given
);
#else
char keyword_only_part[100];
if ( kw_only > 0 )
{
sprintf(
keyword_only_part,
" positional argument%s (and %" PY_FORMAT_SIZE_T "d keyword-only argument%s)",
given != 1 ? "s" : "",
kw_only,
kw_only != 1 ? "s" : ""
);
}
else
{
keyword_only_part[0] = 0;
}
if ( function->m_defaults_given == 0 )
{
PyErr_Format(
PyExc_TypeError,
"%s() takes %zd positional argument%s but %zd%s were given",
function_name,
top_level_parameter_count,
plural,
given,
keyword_only_part
);
}
else
{
PyErr_Format(
PyExc_TypeError,
"%s() takes from %zd to %zd positional argument%s but %zd%s were given",
function_name,
top_level_parameter_count - function->m_defaults_given,
top_level_parameter_count,
plural,
given,
keyword_only_part
);
}
#endif
}
#if PYTHON_VERSION >= 330
void ERROR_TOO_FEW_ARGUMENTS( Nuitka_FunctionObject *function,
PyObject **values )
{
char const *function_name =
Nuitka_String_AsString( function->m_name );
PyCodeObject *code_object = function->m_code_object;
Py_ssize_t max_missing = 0;
for( Py_ssize_t i = code_object->co_argcount-1; i >= 0; --i )
{
if ( values[ i ] == NULL )
{
max_missing += 1;
}
}
PyObject *list_str = PyUnicode_FromString( "" );
PyObject *comma_str = PyUnicode_FromString( ", " );
PyObject *and_str = PyUnicode_FromString(
max_missing == 2 ? " and " : ", and "
);
Py_ssize_t missing = 0;
for( Py_ssize_t i = code_object->co_argcount-1; i >= 0; --i )
{
if ( values[ i ] == NULL )
{
PyObject *current_str = PyTuple_GET_ITEM(
code_object->co_varnames,
i
);
PyObjectTemporary current( PyObject_Repr( current_str ) );
if ( missing == 0 )
{
PyObjectTemporary old( list_str );
list_str = PyUnicode_Concat(
list_str,
current.asObject0()
);
}
else if ( missing == 1 )
{
PyObjectTemporary old1( list_str );
list_str = PyUnicode_Concat(
and_str,
list_str
);
PyObjectTemporary old2( list_str );
list_str = PyUnicode_Concat(
current.asObject0(),
list_str
);
}
else
{
PyObjectTemporary old1( list_str );
list_str = PyUnicode_Concat(
comma_str,
list_str
);
PyObjectTemporary old2( list_str );
list_str = PyUnicode_Concat(
current.asObject0(),
list_str
);
}
missing += 1;
}
}
Py_DECREF( comma_str );
Py_DECREF( and_str );
PyErr_Format(
PyExc_TypeError,
"%s() missing %zd required positional argument%s: %s",
function_name,
max_missing,
max_missing > 1 ? "s" : "",
Nuitka_String_AsString( list_str )
);
Py_DECREF( list_str );
}
void ERROR_TOO_FEW_KWONLY( struct Nuitka_FunctionObject *function,
PyObject **kw_vars )
{
char const *function_name =
Nuitka_String_AsString( function->m_name );
PyCodeObject *code_object = function->m_code_object;
Py_ssize_t max_missing = 0;
for( Py_ssize_t i = code_object->co_kwonlyargcount-1; i >= 0; --i )
{
if ( kw_vars[ i ] == NULL )
{
max_missing += 1;
}
}
PyObject *list_str = PyUnicode_FromString( "" );
PyObject *comma_str = PyUnicode_FromString( ", " );
PyObject *and_str = PyUnicode_FromString(
max_missing == 2 ? " and " : ", and "
);
Py_ssize_t missing = 0;
for( Py_ssize_t i = code_object->co_kwonlyargcount-1; i >= 0; --i )
{
if ( kw_vars[ i ] == NULL )
{
PyObject *current_str = PyTuple_GET_ITEM(
code_object->co_varnames,
code_object->co_argcount + i
);
PyObjectTemporary current( PyObject_Repr( current_str ) );
if ( missing == 0 )
{
PyObjectTemporary old( list_str );
list_str = PyUnicode_Concat(
list_str,
current.asObject0()
);
}
else if ( missing == 1 )
{
PyObjectTemporary old1( list_str );
list_str = PyUnicode_Concat(
and_str,
list_str
);
PyObjectTemporary old2( list_str );
list_str = PyUnicode_Concat(
current.asObject0(),
list_str
);
}
else
{
PyObjectTemporary old1( list_str );
list_str = PyUnicode_Concat(
comma_str,
list_str
);
PyObjectTemporary old2( list_str );
list_str = PyUnicode_Concat(
current.asObject0(),
list_str
);
}
missing += 1;
}
}
Py_DECREF( comma_str );
Py_DECREF( and_str );
PyErr_Format(
PyExc_TypeError,
"%s() missing %zd required keyword-only argument%s: %s",
function_name,
max_missing,
max_missing > 1 ? "s" : "",
Nuitka_String_AsString( list_str )
);
Py_DECREF( list_str );
}
#endif
Nuitka-0.5.0.1/nuitka/build/static_src/MetaPathBasedLoader.cpp 0000644 0001750 0001750 00000025062 12265264105 024375 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// This implements the loading of C++ compiled modules and shared library
// extension modules bundled for standalone mode.
// This is achieved mainly by registered a "sys.meta_path" loader, that then
// gets asked for module names, and responds if knows about one. It's fed by
// a table created at compile time.
// The nature and use of these 2 loaded module kinds is very different, but
// having them as distinct loaders would only require to duplicate the search
// and registering of stuff.
#include
#include "nuitka/prelude.hpp"
#include "nuitka/unfreezing.hpp"
// For Python3.3, the loader is a module attribute, so we need to make it
// accessible from this variable.
#if PYTHON_VERSION < 330
static
#endif
PyObject *metapath_based_loader = NULL;
static Nuitka_MetaPathBasedLoaderEntry *loader_entries = NULL;
static char *_kwlist[] = {
(char *)"fullname",
(char *)"unused",
NULL
};
static PyObject *_path_unfreezer_find_module( PyObject *self, PyObject *args, PyObject *kwds )
{
PyObject *module_name;
PyObject *unused;
int res = PyArg_ParseTupleAndKeywords(
args,
kwds,
"O|O:find_module",
_kwlist,
&module_name,
&unused
);
if (unlikely( res == 0 ))
{
return NULL;
}
char const *name = Nuitka_String_AsString( module_name );
if ( Py_VerboseFlag )
{
PySys_WriteStderr( "import %s # considering responsibility\n", name );
}
struct Nuitka_MetaPathBasedLoaderEntry *current = loader_entries;
while ( current->name != NULL )
{
if ( strcmp( name, current->name ) == 0 )
{
if ( Py_VerboseFlag )
{
PySys_WriteStderr( "import %s # claimed responsibility\n", name );
}
return INCREASE_REFCOUNT( metapath_based_loader );
}
current++;
}
if ( Py_VerboseFlag )
{
PySys_WriteStderr( "import %s # denied responsibility\n", name );
}
return INCREASE_REFCOUNT( Py_None );
}
#ifdef _NUITKA_STANDALONE
#if PYTHON_VERSION < 300
typedef void (*entrypoint_t)( void );
#else
typedef PyObject * (*entrypoint_t)( void );
#endif
#ifndef _WIN32
// Shared libraries loading.
#include
#endif
PyObject *callIntoShlibModule( const char *full_name, const char *filename )
{
// Determine the package name and basename of the module to load.
char const *dot = strrchr( full_name, '.' );
char const *name;
char const *package;
if ( dot == NULL )
{
package = NULL;
name = full_name;
}
else {
package = (char *)full_name;
name = dot+1;
}
char entry_function_name[1024];
snprintf(
entry_function_name, sizeof( entry_function_name ),
#if PYTHON_VERSION < 300
"init%s",
#else
"PyInit_%s",
#endif
name
);
#ifdef _WIN32
unsigned int old_mode = SetErrorMode( SEM_FAILCRITICALERRORS );
char abs_filename[ 4096 ];
LPTSTR unused = NULL;
// Need to use absolute filename for Win9x to work correctly, however
// important that is.
GetFullPathName( filename, sizeof( abs_filename ), abs_filename, &unused );
if ( Py_VerboseFlag )
{
PySys_WriteStderr(
"import %s # LoadLibraryEx(\"%s\");\n",
filename
);
}
HINSTANCE hDLL = LoadLibraryEx( abs_filename, NULL, LOAD_WITH_ALTERED_SEARCH_PATH );
if (unlikely( hDLL == NULL ))
{
PyErr_Format( PyExc_ImportError, "LoadLibraryEx '%s' failed", abs_filename );
return NULL;
}
entrypoint_t entrypoint = (entrypoint_t)GetProcAddress( hDLL, entry_function_name );
SetErrorMode( old_mode );
#else
int dlopenflags = PyThreadState_GET()->interp->dlopenflags;
if ( Py_VerboseFlag )
{
PySys_WriteStderr(
"import %s # dlopen(\"%s\", %x);\n",
full_name,
filename,
dlopenflags
);
}
void *handle = dlopen( filename, dlopenflags );
if (unlikely( handle == NULL ))
{
const char *error = dlerror();
if (unlikely( error == NULL ))
{
error = "unknown dlopen() error";
}
PyErr_SetString( PyExc_ImportError, error );
return NULL;
}
entrypoint_t entrypoint = (entrypoint_t)dlsym(
handle,
entry_function_name
);
#endif
assert( entrypoint );
char *old_context = _Py_PackageContext;
_Py_PackageContext = (char *)package;
// Finally call into the DLL.
#if PYTHON_VERSION < 300
(*entrypoint)();
#else
PyObject *module = (*entrypoint)();
#endif
_Py_PackageContext = old_context;
#if PYTHON_VERSION < 300
PyObject *module = PyDict_GetItemString(
PyImport_GetModuleDict(),
full_name
);
#endif
if (unlikely( module == NULL ))
{
PyErr_Format(
PyExc_SystemError,
"dynamic module not initialized properly"
);
return NULL;
}
#if PYTHON_VERSION >= 300
struct PyModuleDef *def = PyModule_GetDef( module );
def->m_base.m_init = entrypoint;
#endif
// Set filename attribute
int res = PyModule_AddStringConstant( module, "__file__", filename );
if (unlikely( res < 0 ))
{
// Might be refuted, which wouldn't be harmful.
PyErr_Clear();
}
#if PYTHON_VERSION >= 300
PyDict_SetItemString( PyImport_GetModuleDict(), full_name, module );
#endif
return module;
}
#endif
static PyObject *_path_unfreezer_load_module( PyObject *self, PyObject *args, PyObject *kwds )
{
PyObject *module_name;
PyObject *unused;
int res = PyArg_ParseTupleAndKeywords(
args,
kwds,
"O|O:load_module",
_kwlist,
&module_name,
&unused
);
if (unlikely( res == 0 ))
{
return NULL;
}
assert( module_name );
assert( Nuitka_String_Check( module_name ) );
char *name = Nuitka_String_AsString( module_name );
struct Nuitka_MetaPathBasedLoaderEntry *current = loader_entries;
while ( current->name != NULL )
{
if ( strcmp( name, current->name ) == 0 )
{
if ( Py_VerboseFlag )
{
PySys_WriteStderr( "Loading %s\n", name );
}
#ifdef _NUITKA_STANDALONE
if ( ( current->flags & NUITKA_SHLIB_MODULE ) != 0 )
{
char filename[4096];
strcpy( filename, getBinaryDirectory() );
char *d = filename;
d += strlen( filename );
assert(*d == 0);
*d++ = SEP;
char *s = current->name;
while( *s )
{
if ( *s == '.' )
{
*d++ = SEP;
s++;
}
else
{
*d++ = *s++;
}
}
*d = 0;
#ifdef _WIN32
strcat( filename, ".pyd" );
#else
strcat( filename, ".so" );
#endif
callIntoShlibModule(
current->name,
filename
);
}
else
#endif
{
assert( ( current->flags & NUITKA_SHLIB_MODULE ) == 0 );
current->python_initfunc();
}
if (unlikely( ERROR_OCCURED() ))
{
return NULL;
}
if ( Py_VerboseFlag )
{
PySys_WriteStderr( "Loaded %s\n", name );
}
return LOOKUP_SUBSCRIPT( PyImport_GetModuleDict(), module_name );
}
current++;
}
assert( false );
return INCREASE_REFCOUNT( Py_None );
}
static PyMethodDef _method_def_loader_find_module =
{
"find_module",
(PyCFunction)_path_unfreezer_find_module,
METH_VARARGS | METH_KEYWORDS,
NULL
};
static PyMethodDef _method_def_loader_load_module =
{
"load_module",
(PyCFunction)_path_unfreezer_load_module,
METH_VARARGS | METH_KEYWORDS,
NULL
};
void registerMetaPathBasedUnfreezer( struct Nuitka_MetaPathBasedLoaderEntry *_loader_entries )
{
// Do it only once.
if ( loader_entries )
{
assert( _loader_entries == loader_entries );
return;
}
loader_entries = _loader_entries;
// Build the dictionary of the "loader" object, which needs to have two
// methods "find_module" where we acknowledge that we are capable of loading
// the module, and "load_module" that does the actual thing.
PyObject *method_dict = PyDict_New();
assertObject( method_dict );
PyObject *loader_find_module = PyCFunction_New(
&_method_def_loader_find_module,
NULL
);
assertObject( loader_find_module );
PyDict_SetItemString( method_dict, "find_module", loader_find_module );
PyObject *loader_load_module = PyCFunction_New(
&_method_def_loader_load_module,
NULL
);
assertObject( loader_load_module );
PyDict_SetItemString( method_dict, "load_module", loader_load_module );
// Build the actual class.
metapath_based_loader = PyObject_CallFunctionObjArgs(
(PyObject *)&PyType_Type,
#if PYTHON_VERSION < 300
PyString_FromString( "_nuitka_compiled_modules_loader" ),
#else
PyUnicode_FromString( "_nuitka_compiled_modules_loader" ),
#endif
const_tuple_empty,
method_dict,
NULL
);
assertObject( metapath_based_loader );
if ( Py_VerboseFlag )
{
PySys_WriteStderr( "setup nuitka compiled module/shlib importer\n" );
}
// And also provide it as a meta path loader.
int res = PyList_Insert(
PySys_GetObject( ( char *)"meta_path" ),
#if PYTHON_VERSION < 330
0,
#else
2,
#endif
metapath_based_loader
);
assert( res == 0 );
}
Nuitka-0.5.0.1/nuitka/build/static_src/CompiledFrameType.cpp 0000644 0001750 0001750 00000040045 12265264105 024153 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#include "nuitka/prelude.hpp"
#include "structmember.h"
#define OFF( x ) offsetof( PyFrameObject, x )
struct Nuitka_FrameObject
{
PyFrameObject m_frame;
};
static PyMemberDef Nuitka_Frame_memberlist[] = {
{ (char *)"f_back", T_OBJECT, OFF( f_back ), READONLY | RESTRICTED },
{ (char *)"f_code", T_OBJECT, OFF( f_code ), READONLY | RESTRICTED },
{ (char *)"f_builtins", T_OBJECT, OFF( f_builtins ), READONLY | RESTRICTED },
{ (char *)"f_globals", T_OBJECT, OFF( f_globals ), READONLY | RESTRICTED },
{ (char *)"f_lasti", T_INT, OFF( f_lasti ), READONLY | RESTRICTED },
{ NULL }
};
#if PYTHON_VERSION < 300
static PyObject *Nuitka_Frame_get_exc_traceback( PyFrameObject *frame )
{
if ( frame->f_exc_traceback != NULL )
{
return INCREASE_REFCOUNT( frame->f_exc_traceback );
}
else
{
return INCREASE_REFCOUNT( Py_None );
}
}
static int Nuitka_Frame_set_exc_traceback( PyFrameObject *frame, PyObject *traceback )
{
if ( frame->f_exc_traceback != NULL )
{
Py_DECREF(frame->f_exc_traceback );
}
if ( traceback == Py_None )
{
frame->f_exc_traceback = NULL;
}
else
{
frame->f_exc_traceback = INCREASE_REFCOUNT_X( traceback );
}
return 0;
}
static PyObject *Nuitka_Frame_get_exc_type( PyFrameObject *frame )
{
if ( frame->f_exc_type != NULL )
{
return INCREASE_REFCOUNT( frame->f_exc_type );
}
else
{
return INCREASE_REFCOUNT( Py_None );
}
}
static int Nuitka_Frame_set_exc_type( PyFrameObject *frame, PyObject *exception_type )
{
if ( frame->f_exc_type != NULL )
{
Py_DECREF( frame->f_exc_type );
}
if ( exception_type == Py_None )
{
frame->f_exc_type = NULL;
}
else
{
frame->f_exc_type = INCREASE_REFCOUNT_X( exception_type );
}
return 0;
}
static PyObject *Nuitka_Frame_get_exc_value( PyFrameObject *frame )
{
if ( frame->f_exc_value != NULL )
{
return INCREASE_REFCOUNT( frame->f_exc_value );
}
else
{
return INCREASE_REFCOUNT( Py_None );
}
}
static int Nuitka_Frame_set_exc_value( PyFrameObject *frame, PyObject *exception_value )
{
if ( frame->f_exc_value != NULL )
{
Py_DECREF( frame->f_exc_value );
}
if ( exception_value == Py_None )
{
frame->f_exc_value = NULL;
}
else
{
frame->f_exc_value = INCREASE_REFCOUNT_X( exception_value );
}
return 0;
}
#endif
static PyObject *Nuitka_Frame_getlocals( PyFrameObject *frame, void *closure )
{
// Note: Very important that we correctly support this function to work:
PyFrame_FastToLocals( frame );
return INCREASE_REFCOUNT( frame->f_locals );
}
static PyObject *Nuitka_Frame_getlineno( PyFrameObject *frame, void *closure )
{
return PyInt_FromLong( frame->f_lineno );
}
static PyObject *Nuitka_Frame_gettrace( PyFrameObject *frame, void *closure )
{
return INCREASE_REFCOUNT( frame->f_trace );
}
static int Nuitka_Frame_settrace( PyFrameObject *frame, PyObject* v, void *closure )
{
PyErr_Format( PyExc_RuntimeError, "f_trace is not writable in Nuitka" );
return -1;
}
static PyObject *Nuitka_Frame_get_restricted( PyFrameObject *frame, void *closure )
{
return INCREASE_REFCOUNT( Py_False );
}
static PyGetSetDef Nuitka_Frame_getsetlist[] = {
{ (char *)"f_locals", (getter)Nuitka_Frame_getlocals, NULL, NULL },
{ (char *)"f_lineno", (getter)Nuitka_Frame_getlineno, NULL, NULL },
{ (char *)"f_trace", (getter)Nuitka_Frame_gettrace, (setter)Nuitka_Frame_settrace, NULL },
{ (char *)"f_restricted", (getter)Nuitka_Frame_get_restricted, NULL, NULL },
#if PYTHON_VERSION < 300
{ (char *)"f_exc_traceback", (getter)Nuitka_Frame_get_exc_traceback, (setter)Nuitka_Frame_set_exc_traceback, NULL },
{ (char *)"f_exc_type", (getter)Nuitka_Frame_get_exc_type, (setter)Nuitka_Frame_set_exc_type, NULL },
{ (char *)"f_exc_value", (getter)Nuitka_Frame_get_exc_value, (setter)Nuitka_Frame_set_exc_value, NULL },
#endif
{ NULL }
};
static void Nuitka_Frame_tp_dealloc( Nuitka_FrameObject *nuitka_frame )
{
Nuitka_GC_UnTrack( nuitka_frame );
Py_TRASHCAN_SAFE_BEGIN( nuitka_frame )
PyFrameObject *frame = &nuitka_frame->m_frame;
// locals
PyObject **valuestack = frame->f_valuestack;
for ( PyObject **p = frame->f_localsplus; p < valuestack; p++ )
{
Py_CLEAR( *p );
}
// stack if any
if ( frame->f_stacktop != NULL )
{
for ( PyObject **p = valuestack; p < frame->f_stacktop; p++ )
{
Py_XDECREF( *p );
}
}
Py_XDECREF( frame->f_back );
Py_DECREF( frame->f_builtins );
Py_DECREF( frame->f_globals );
Py_CLEAR( frame->f_locals );
Py_CLEAR( frame->f_trace );
Py_CLEAR( frame->f_exc_type );
Py_CLEAR( frame->f_exc_value );
Py_CLEAR( frame->f_exc_traceback );
PyObject_GC_Del( nuitka_frame );
Py_TRASHCAN_SAFE_END( nuitka_frame )
}
static int Nuitka_Frame_tp_traverse( PyFrameObject *frame, visitproc visit, void *arg )
{
Py_VISIT( frame->f_back );
Py_VISIT( frame->f_code );
Py_VISIT( frame->f_builtins );
Py_VISIT( frame->f_globals );
Py_VISIT( frame->f_locals );
Py_VISIT( frame->f_trace );
Py_VISIT( frame->f_exc_type );
Py_VISIT( frame->f_exc_value );
Py_VISIT( frame->f_exc_traceback );
// locals
Py_ssize_t slots = frame->f_code->co_nlocals + PyTuple_GET_SIZE( frame->f_code->co_cellvars ) + PyTuple_GET_SIZE( frame->f_code->co_freevars );
PyObject **fastlocals = frame->f_localsplus;
for ( Py_ssize_t i = slots; --i >= 0; ++fastlocals )
{
Py_VISIT( *fastlocals );
}
// stack if any
if ( frame->f_stacktop != NULL )
{
for ( PyObject **p = frame->f_valuestack; p < frame->f_stacktop; p++ )
{
Py_VISIT( *p );
}
}
return 0;
}
static void Nuitka_Frame_tp_clear( PyFrameObject *frame )
{
PyObject **oldtop = frame->f_stacktop;
frame->f_stacktop = NULL;
Py_CLEAR( frame->f_exc_type );
Py_CLEAR( frame->f_exc_value );
Py_CLEAR( frame->f_exc_traceback );
Py_CLEAR( frame->f_trace );
// locals
Py_ssize_t slots = frame->f_code->co_nlocals + PyTuple_GET_SIZE( frame->f_code->co_cellvars ) + PyTuple_GET_SIZE( frame->f_code->co_freevars );
PyObject **fastlocals = frame->f_localsplus;
for ( Py_ssize_t i = slots; --i >= 0; ++fastlocals )
{
Py_CLEAR( *fastlocals );
}
// stack if any
if ( oldtop != NULL )
{
for ( PyObject **p = frame->f_valuestack; p < oldtop; p++ )
{
Py_CLEAR( *p );
}
}
}
static PyObject *Nuitka_Frame_sizeof( PyFrameObject *frame )
{
Py_ssize_t slots =
frame->f_code->co_stacksize +
frame->f_code->co_nlocals +
PyTuple_GET_SIZE( frame->f_code->co_cellvars ) +
PyTuple_GET_SIZE( frame->f_code->co_freevars );
return PyInt_FromSsize_t( sizeof( Nuitka_FrameObject ) + slots * sizeof(PyObject *) );
}
static PyMethodDef Nuitka_Frame_methods[] =
{
{ "__sizeof__", (PyCFunction)Nuitka_Frame_sizeof, METH_NOARGS, "F.__sizeof__() -> size of F in memory, in bytes" },
{ NULL, NULL }
};
PyTypeObject Nuitka_Frame_Type =
{
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"compiled_frame",
sizeof(Nuitka_FrameObject),
sizeof(PyObject *),
(destructor)Nuitka_Frame_tp_dealloc, // tp_dealloc
0, // tp_print
0, // tp_getattr
0, // tp_setattr
0, // tp_compare
0, // tp_repr
0, // tp_as_number
0, // tp_as_sequence
0, // tp_as_mapping
0, // tp_hash
0, // tp_call
0, // tp_str
PyObject_GenericGetAttr, // tp_getattro
PyObject_GenericSetAttr, // tp_setattro
0, // tp_as_buffer
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, // tp_flags
0, // tp_doc
(traverseproc)Nuitka_Frame_tp_traverse, // tp_traverse
(inquiry)Nuitka_Frame_tp_clear, // tp_clear
0, // tp_richcompare
0, // tp_weaklistoffset
0, // tp_iter
0, // tp_iternext
Nuitka_Frame_methods, // tp_methods
Nuitka_Frame_memberlist, // tp_members
Nuitka_Frame_getsetlist, // tp_getset
0, // tp_base
0, // tp_dict
};
static void tb_dealloc( PyTracebackObject *tb )
{
// printf( "dealloc TB %ld %lx FR %ld %lx\n", Py_REFCNT( tb ), (long)tb, Py_REFCNT( tb->tb_frame ), (long)tb->tb_frame );
PyObject_GC_UnTrack(tb);
// Py_TRASHCAN_SAFE_BEGIN(tb)
Py_XDECREF( tb->tb_next );
Py_XDECREF( tb->tb_frame );
PyObject_GC_Del( tb );
// Py_TRASHCAN_SAFE_END(tb)
}
extern PyObject *const_str_plain___module__;
PyFrameObject *MAKE_FRAME( PyCodeObject *code, PyObject *module )
{
PyTraceBack_Type.tp_dealloc = (destructor)tb_dealloc;
assertCodeObject( code );
PyObject *globals = ((PyModuleObject *)module)->md_dict;
assert( PyDict_Check( globals ) );
Py_ssize_t ncells = PyTuple_GET_SIZE( code->co_cellvars );
Py_ssize_t nfrees = PyTuple_GET_SIZE( code->co_freevars );
Py_ssize_t extras = code->co_stacksize + code->co_nlocals + ncells + nfrees;
Nuitka_FrameObject *result = PyObject_GC_NewVar( Nuitka_FrameObject, &Nuitka_Frame_Type, extras );
if (unlikely( result == NULL ))
{
throw PythonException();
}
PyFrameObject *frame = &result->m_frame;
frame->f_code = code;
extras = code->co_nlocals + ncells + nfrees;
frame->f_valuestack = frame->f_localsplus + extras;
for ( Py_ssize_t i = 0; i < extras; i++ )
{
frame->f_localsplus[i] = NULL;
}
frame->f_locals = NULL;
frame->f_trace = INCREASE_REFCOUNT( Py_None );
frame->f_exc_type = frame->f_exc_value = frame->f_exc_traceback = NULL;
frame->f_stacktop = frame->f_valuestack;
frame->f_builtins = INCREASE_REFCOUNT( (PyObject *)dict_builtin );
frame->f_back = NULL;
frame->f_globals = INCREASE_REFCOUNT( globals );
if (likely( (code->co_flags & CO_OPTIMIZED ) == CO_OPTIMIZED ))
{
frame->f_locals = NULL;
}
else if ( likely( (code->co_flags & CO_NEWLOCALS ) ) )
{
frame->f_locals = PyDict_New();
if (unlikely( frame->f_locals == NULL ))
{
Py_DECREF( result );
throw PythonException();
}
PyDict_SetItem( frame->f_locals, const_str_plain___module__, MODULE_NAME( module ) );
}
else
{
frame->f_locals = INCREASE_REFCOUNT( globals );
}
frame->f_tstate = PyThreadState_GET();
frame->f_lasti = -1;
frame->f_lineno = code->co_firstlineno;
frame->f_iblock = 0;
Nuitka_GC_Track( result );
return (PyFrameObject *)result;
}
extern PyObject *const_str_empty;
extern PyObject *const_bytes_empty;
#if PYTHON_VERSION < 300
PyCodeObject *MAKE_CODEOBJ( PyObject *filename, PyObject *function_name, int line, PyObject *argnames, int arg_count, int flags )
#else
PyCodeObject *MAKE_CODEOBJ( PyObject *filename, PyObject *function_name, int line, PyObject *argnames, int arg_count, int kw_only_count, int flags )
#endif
{
assertObject( filename );
assert( Nuitka_String_Check( filename ) );
assertObject( function_name );
assert( Nuitka_String_Check( function_name ) );
assertObject( argnames );
assert( PyTuple_Check( argnames ) );
// TODO: Consider using PyCode_NewEmpty
PyCodeObject *result = PyCode_New (
arg_count, // argcount
#if PYTHON_VERSION >= 300
kw_only_count, // kw-only count
#endif
0, // nlocals
0, // stacksize
flags, // flags
#if PYTHON_VERSION < 300
const_str_empty, // code (bytecode)
#else
const_bytes_empty, // code (bytecode)
#endif
const_tuple_empty, // consts (we are not going to be compatible)
const_tuple_empty, // names (we are not going to be compatible)
argnames, // varnames (we are not going to be compatible)
const_tuple_empty, // freevars (we are not going to be compatible)
const_tuple_empty, // cellvars (we are not going to be compatible)
filename, // filename
function_name, // name
line, // firstlineno (offset of the code object)
#if PYTHON_VERSION < 300
const_str_empty // lnotab (table to translate code object)
#else
const_bytes_empty // lnotab (table to translate code object)
#endif
);
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
static PyFrameObject *duplicateFrame( PyFrameObject *old_frame )
{
PyFrameObject *new_frame = PyObject_GC_NewVar( PyFrameObject, &PyFrame_Type, 0 );
// Allow only to detach only our tracing frames.
assert( old_frame->f_trace == Py_None );
new_frame->f_trace = INCREASE_REFCOUNT( Py_None );
// Copy the back reference if any.
new_frame->f_back = old_frame->f_back;
Py_XINCREF( new_frame->f_back );
// Take a code reference as well.
new_frame->f_code = old_frame->f_code;
Py_XINCREF( new_frame->f_code );
// Copy attributes.
new_frame->f_globals = INCREASE_REFCOUNT( old_frame->f_globals );
// TODO: Detach is called for module frames, where it is totally not necessary, as
// these cannot be reused. Remove the need for this code.
if ( old_frame->f_globals == old_frame->f_locals )
{
new_frame->f_locals = INCREASE_REFCOUNT( old_frame->f_globals );
}
else
{
new_frame->f_locals = NULL;
}
new_frame->f_builtins = INCREASE_REFCOUNT( old_frame->f_builtins );
new_frame->f_exc_type = INCREASE_REFCOUNT_X( old_frame->f_exc_type );
new_frame->f_exc_value = INCREASE_REFCOUNT_X( old_frame->f_exc_value );
new_frame->f_exc_traceback = INCREASE_REFCOUNT_X( old_frame->f_exc_traceback );
assert( old_frame->f_valuestack == old_frame->f_localsplus );
new_frame->f_valuestack = new_frame->f_localsplus;
assert( old_frame->f_stacktop == old_frame->f_valuestack );
new_frame->f_stacktop = new_frame->f_valuestack;
new_frame->f_tstate = old_frame->f_tstate;
new_frame->f_lasti = -1;
new_frame->f_lineno = old_frame->f_lineno;
assert( old_frame->f_iblock == 0 );
new_frame->f_iblock = 0;
Nuitka_GC_Track( new_frame );
return new_frame;
}
PyFrameObject *detachCurrentFrame()
{
PyFrameObject *old_frame = PyThreadState_GET()->frame;
// Duplicate it.
PyFrameObject *new_frame = duplicateFrame( old_frame );
// The given frame can be put on top now.
PyThreadState_GET()->frame = new_frame;
Py_DECREF( old_frame );
return new_frame;
}
Nuitka-0.5.0.1/nuitka/build/static_src/win32_ucontext_src/ 0000755 0001750 0001750 00000000000 12265271051 023633 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/build/static_src/win32_ucontext_src/fibers_win32.cpp 0000644 0001750 0001750 00000003025 12265264105 026635 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// Implementation of process context switch for Win32
#include "nuitka/prelude.hpp"
#define STACK_SIZE (1024*1024)
void initFiber( Fiber *to )
{
// Need to call this at least once per thread, so we have a main FIBER.
ConvertThreadToFiber( NULL );
assert( to );
to->fiber = NULL;
}
void prepareFiber( Fiber *to, void *code, unsigned long arg )
{
assert( to );
assert( code );
to->fiber = CreateFiber( STACK_SIZE, (LPFIBER_START_ROUTINE)code, (LPVOID)arg );
}
void releaseFiber( Fiber *to )
{
if ( to->fiber )
{
DeleteFiber( to->fiber );
}
to->fiber = NULL;
}
void swapFiber( Fiber *to, Fiber *from )
{
to->fiber = GetCurrentFiber();
assert( from->fiber );
SwitchToFiber( from->fiber );
}
Nuitka-0.5.0.1/nuitka/build/static_src/gen_ucontext_src/ 0000755 0001750 0001750 00000000000 12265271051 023442 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/build/static_src/gen_ucontext_src/fibers_gen.cpp 0000644 0001750 0001750 00000003714 12265264105 026260 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// Implementation of process context switch for generic targets.
#include "nuitka/prelude.hpp"
#define STACK_SIZE (1024*1024)
// Keep one stack around to avoid the overhead of repeated malloc/free in
// case of frequent instantiations in a loop.
static void *last_stack = NULL;
void initFiber( Fiber *to )
{
to->f_context.uc_stack.ss_sp = NULL;
to->f_context.uc_link = NULL;
to->start_stack = NULL;
}
void prepareFiber( Fiber *to, void *code, unsigned long arg )
{
int res = getcontext( &to->f_context );
assert( res == 0 );
to->f_context.uc_stack.ss_size = STACK_SIZE;
to->f_context.uc_stack.ss_sp = last_stack ? (char *)last_stack : (char *)malloc( STACK_SIZE );
to->start_stack = to->f_context.uc_stack.ss_sp;
to->f_context.uc_link = NULL;
last_stack = NULL;
makecontext( &to->f_context, (void (*)())code, 1, (unsigned long)arg );
}
void releaseFiber( Fiber *to )
{
if ( last_stack == NULL )
{
last_stack = to->start_stack;
}
else
{
free( to->start_stack );
}
}
void swapFiber( Fiber *to, Fiber *from )
{
int res =
swapcontext( &to->f_context, &from->f_context );
assert( res == 0 );
}
Nuitka-0.5.0.1/nuitka/build/static_src/arm_ucontext_src/ 0000755 0001750 0001750 00000000000 12265271051 023450 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/build/static_src/arm_ucontext_src/ucontext.cpp 0000644 0001750 0001750 00000004735 12265044767 026052 0 ustar hayen hayen 0000000 0000000 /* Extract of libtask, just the ARM specific portions. */
/*
This software was developed as part of a project at MIT.
Copyright (c) 2005-2007 Russ Cox,
Massachusetts Institute of Technology
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
===
Contains parts of an earlier library that has:
* The authors of this software are Rob Pike, Sape Mullender, and Russ Cox
* Copyright (c) 2003 by Lucent Technologies.
* Permission to use, copy, modify, and distribute this software for any
* purpose without fee is hereby granted, provided that this entire notice
* is included in all copies of any software which is or includes a copy
* or modification of this software and in all copies of the supporting
* documentation for such software.
* THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
* WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
* REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
* OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
*/
#include "ucontext.h"
#include
#include
void makecontext(ucontext_t *uc, void (*fn)(void), int argc, ...)
{
int *sp;
va_list arg;
sp = (int*)uc->uc_stack.ss_sp+uc->uc_stack.ss_size/4;
va_start(arg, argc);
assert( argc == 1 ); // That allows me to hard code r0
uc->uc_mcontext.trap_no = va_arg(arg, unsigned long);
va_end(arg);
uc->uc_mcontext.arm_r10 = (unsigned long)sp;
uc->uc_mcontext.arm_fp = (unsigned long)fn;
}
Nuitka-0.5.0.1/nuitka/build/static_src/arm_ucontext_src/fibers_arm.cpp 0000644 0001750 0001750 00000006663 12265044767 026314 0 ustar hayen hayen 0000000 0000000 /* Extract of libtask, just the ARM specific portions. */
/*
This software was developed as part of a project at MIT.
Copyright (c) 2005-2007 Russ Cox,
Massachusetts Institute of Technology
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
===
Contains parts of an earlier library that has:
* The authors of this software are Rob Pike, Sape Mullender, and Russ Cox
* Copyright (c) 2003 by Lucent Technologies.
* Permission to use, copy, modify, and distribute this software for any
* purpose without fee is hereby granted, provided that this entire notice
* is included in all copies of any software which is or includes a copy
* or modification of this software and in all copies of the supporting
* documentation for such software.
* THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
* WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
* REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
* OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
*/
// Implementation of process context switch for ARM
#include "nuitka/prelude.hpp"
extern "C" int getmcontext( mcontext_t *mcontext );
extern "C" void setmcontext( const mcontext_t *mcontext );
#define setcontext(u) setmcontext( &(u)->uc_mcontext )
#define getcontext(u) getmcontext( &(u)->uc_mcontext )
void makecontext( ucontext_t *uc, void (*fn)(void), int argc, ... );
#define STACK_SIZE (1024*1024)
// Keep one stack around to avoid the overhead of repeated malloc/free in
// case of frequent instantiations in a loop.
static void *last_stack = NULL;
void initFiber( Fiber *to )
{
to->f_context.uc_stack.ss_sp = NULL;
to->f_context.uc_link = NULL;
to->start_stack = NULL;
}
void prepareFiber( Fiber *to, void *code, unsigned long arg )
{
int res = getcontext( &to->f_context );
assert( res == 0 );
to->f_context.uc_stack.ss_size = STACK_SIZE;
to->f_context.uc_stack.ss_sp = last_stack ? last_stack : malloc( STACK_SIZE );
to->start_stack = to->f_context.uc_stack.ss_sp;
to->f_context.uc_link = NULL;
last_stack = NULL;
makecontext( &to->f_context, (void (*)())code, 1, (unsigned long)arg );
}
void releaseFiber( Fiber *to )
{
if ( last_stack == NULL )
{
last_stack = to->start_stack;
}
else
{
free( to->start_stack );
}
}
void swapFiber( Fiber *to, Fiber *from )
{
if ( getcontext( &to->f_context ) == 0 )
{
setcontext( &from->f_context );
}
}
Nuitka-0.5.0.1/nuitka/build/static_src/arm_ucontext_src/getcontext.asm 0000644 0001750 0001750 00000006073 12265044767 026360 0 ustar hayen hayen 0000000 0000000 /* Extract of libtask, just the ARM specific portions. */
/*
This software was developed as part of a project at MIT.
Copyright (c) 2005-2007 Russ Cox,
Massachusetts Institute of Technology
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
===
Contains parts of an earlier library that has:
* The authors of this software are Rob Pike, Sape Mullender, and Russ Cox
* Copyright (c) 2003 by Lucent Technologies.
* Permission to use, copy, modify, and distribute this software for any
* purpose without fee is hereby granted, provided that this entire notice
* is included in all copies of any software which is or includes a copy
* or modification of this software and in all copies of the supporting
* documentation for such software.
* THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
* WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
* REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
* OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
*/
.globl getmcontext
getmcontext:
str r1, [r0,#4]
str r2, [r0,#8]
str r3, [r0,#12]
str r4, [r0,#16]
str r5, [r0,#20]
str r6, [r0,#24]
str r7, [r0,#28]
str r8, [r0,#32]
str r9, [r0,#36]
str r10, [r0,#40]
str r11, [r0,#44]
str r12, [r0,#48]
str r13, [r0,#52]
str r14, [r0,#56]
/* store 1 as r0-to-restore */
mov r1, #1
str r1, [r0]
/* return 0 */
mov r0, #0
mov pc, lr
.globl setmcontext
setmcontext:
ldr r1, [r0,#4]
ldr r2, [r0,#8]
ldr r3, [r0,#12]
ldr r4, [r0,#16]
ldr r5, [r0,#20]
ldr r6, [r0,#24]
ldr r7, [r0,#28]
ldr r8, [r0,#32]
ldr r9, [r0,#36]
ldr r10, [r0,#40]
ldr r11, [r0,#44]
ldr r12, [r0,#48]
ldr r13, [r0,#52]
ldr r14, [r0,#56]
ldr r0, [r0]
mov pc, lr
Nuitka-0.5.0.1/nuitka/build/static_src/arm_ucontext_src/ucontext.h 0000644 0001750 0001750 00000004635 12265044767 025516 0 ustar hayen hayen 0000000 0000000 /* Extract of libtask, just the ARM specific portions. */
/*
This software was developed as part of a project at MIT.
Copyright (c) 2005-2007 Russ Cox,
Massachusetts Institute of Technology
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
===
Contains parts of an earlier library that has:
* The authors of this software are Rob Pike, Sape Mullender, and Russ Cox
* Copyright (c) 2003 by Lucent Technologies.
* Permission to use, copy, modify, and distribute this software for any
* purpose without fee is hereby granted, provided that this entire notice
* is included in all copies of any software which is or includes a copy
* or modification of this software and in all copies of the supporting
* documentation for such software.
* THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
* WARRANTY. IN PARTICULAR, NEITHER THE AUTHORS NOR LUCENT TECHNOLOGIES MAKE ANY
* REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
* OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
*/
#ifndef ARM_UCONTEXT_H
#define ARM_UCONTEXT_H
#include
#include
extern "C" int getmcontext(mcontext_t*);
extern "C" void setmcontext(const mcontext_t*);
int swapcontext(ucontext_t *oucp, const ucontext_t *ucp);
void makecontext(ucontext_t *uc, void (*fn)(void), int argc, ...);
#define setcontext(u) setmcontext(&(u)->uc_mcontext)
#define getcontext(u) getmcontext(&(u)->uc_mcontext)
#endif
Nuitka-0.5.0.1/nuitka/build/static_src/CompiledGeneratorType.cpp 0000644 0001750 0001750 00000047403 12265264105 025054 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#include "nuitka/prelude.hpp"
static PyObject *Nuitka_Generator_tp_repr( Nuitka_GeneratorObject *generator )
{
#if PYTHON_VERSION < 300
return PyString_FromFormat(
#else
return PyUnicode_FromFormat(
#endif
"",
Nuitka_String_AsString( generator->m_name ),
generator
);
}
static long Nuitka_Generator_tp_traverse( PyObject *function, visitproc visit, void *arg )
{
// TODO: Identify the impact of not visiting owned objects and/or if it
// could be NULL instead. The methodobject visits its self and module. I
// understand this is probably so that back references of this function to
// its upper do not make it stay in the memory. A specific test if that
// works might be needed.
return 0;
}
static PyObject *Nuitka_Generator_send( Nuitka_GeneratorObject *generator, PyObject *value )
{
if ( generator->m_status == status_Unused && value != NULL && value != Py_None )
{
PyErr_Format( PyExc_TypeError, "can't send non-None value to a just-started generator" );
return NULL;
}
if ( generator->m_status != status_Finished )
{
PyThreadState *thread_state = PyThreadState_GET();
#if PYTHON_VERSION < 300
PyObject *saved_exception_type = INCREASE_REFCOUNT_X( thread_state->exc_type );
PyObject *saved_exception_value = INCREASE_REFCOUNT_X( thread_state->exc_value );
PyTracebackObject *saved_exception_traceback = INCREASE_REFCOUNT_X( (PyTracebackObject *)thread_state->exc_traceback );
#endif
if ( generator->m_running )
{
PyErr_Format( PyExc_ValueError, "generator already executing" );
return NULL;
}
if ( generator->m_status == status_Unused )
{
generator->m_status = status_Running;
// Prepare the generator context to run. TODO: Make stack size
// rational.
prepareFiber( &generator->m_yielder_context, generator->m_code, (unsigned long)generator );
}
generator->m_yielded = value;
// Put the generator back on the frame stack.
PyFrameObject *return_frame = thread_state->frame;
#ifndef __NUITKA_NO_ASSERT__
if ( return_frame )
{
assertFrameObject( return_frame );
}
#endif
if ( generator->m_frame )
{
// It would be nice if our frame were still alive. Nobody had the
// right to release it.
assertFrameObject( generator->m_frame );
// It's not supposed to be on the top right now.
assert( return_frame != generator->m_frame );
Py_XINCREF( return_frame );
generator->m_frame->f_back = return_frame;
thread_state->frame = generator->m_frame;
}
// Continue the yielder function while preventing recursion.
generator->m_running = true;
swapFiber( &generator->m_caller_context, &generator->m_yielder_context );
generator->m_running = false;
thread_state = PyThreadState_GET();
// Remove the generator from the frame stack.
assert( thread_state->frame == generator->m_frame );
assertFrameObject( generator->m_frame );
thread_state->frame = return_frame;
Py_CLEAR( generator->m_frame->f_back );
if ( generator->m_yielded == NULL )
{
assert( ERROR_OCCURED() );
generator->m_status = status_Finished;
Py_XDECREF( generator->m_frame );
generator->m_frame = NULL;
if ( generator->m_context )
{
// Surpressing exception in cleanup, to restore later before
// return.
PythonException saved_exception;
generator->m_cleanup( generator->m_context );
generator->m_context = NULL;
saved_exception.toPython();
}
assert( ERROR_OCCURED() );
#if PYTHON_VERSION < 300
Py_XDECREF( saved_exception_type );
Py_XDECREF( saved_exception_value );
Py_XDECREF( saved_exception_traceback );
#endif
return NULL;
}
else
{
#if PYTHON_VERSION < 300
_SET_CURRENT_EXCEPTION( saved_exception_type, saved_exception_value, saved_exception_traceback );
Py_XDECREF( saved_exception_type );
Py_XDECREF( saved_exception_value );
Py_XDECREF( saved_exception_traceback );
#endif
return generator->m_yielded;
}
}
else
{
PyErr_SetObject( PyExc_StopIteration, (PyObject *)NULL );
return NULL;
}
}
static PyObject *Nuitka_Generator_tp_iternext( Nuitka_GeneratorObject *generator )
{
return Nuitka_Generator_send( generator, Py_None );
}
static PyObject *Nuitka_Generator_close( Nuitka_GeneratorObject *generator, PyObject *args )
{
if ( generator->m_status == status_Running )
{
generator->m_exception_type = PyExc_GeneratorExit;
generator->m_exception_value = NULL;
generator->m_exception_tb = NULL;
PyObject *result = Nuitka_Generator_send( generator, Py_None );
if (unlikely( result ))
{
Py_DECREF( result );
PyErr_Format( PyExc_RuntimeError, "generator ignored GeneratorExit" );
return NULL;
}
else if ( PyErr_ExceptionMatches( PyExc_StopIteration ) || PyErr_ExceptionMatches( PyExc_GeneratorExit ) )
{
PyErr_Clear();
return INCREASE_REFCOUNT( Py_None );
}
else
{
assert( ERROR_OCCURED() );
return NULL;
}
}
return INCREASE_REFCOUNT( Py_None );
}
static void Nuitka_Generator_tp_dealloc( Nuitka_GeneratorObject *generator )
{
assert( Py_REFCNT( generator ) == 0 );
Py_REFCNT( generator ) = 1;
PyObject *close_result = Nuitka_Generator_close( generator, NULL );
if (unlikely( close_result == NULL ))
{
PyErr_WriteUnraisable( (PyObject *)generator );
}
else
{
Py_DECREF( close_result );
}
assert( Py_REFCNT( generator ) == 1 );
Py_REFCNT( generator ) = 0;
releaseFiber( &generator->m_yielder_context );
// Now it is safe to release references and memory for it.
Nuitka_GC_UnTrack( generator );
if ( generator->m_weakrefs != NULL )
{
PyObject_ClearWeakRefs( (PyObject *)generator );
}
if ( generator->m_context )
{
generator->m_cleanup( generator->m_context );
}
Py_DECREF( generator->m_name );
Py_XDECREF( generator->m_frame );
PyObject_GC_Del( generator );
}
static PyObject *Nuitka_Generator_throw( Nuitka_GeneratorObject *generator, PyObject *args )
{
generator->m_exception_value = NULL;
generator->m_exception_tb = NULL;
int res = PyArg_UnpackTuple( args, "throw", 1, 3, &generator->m_exception_type, &generator->m_exception_value, (PyObject **)&generator->m_exception_tb );
if ( (PyObject *)generator->m_exception_tb == Py_None )
{
generator->m_exception_tb = NULL;
}
else if ( generator->m_exception_tb != NULL && !PyTraceBack_Check( generator->m_exception_tb ) )
{
PyErr_Format( PyExc_TypeError, "throw() third argument must be a traceback object" );
return NULL;
}
if (unlikely( res == 0 ))
{
generator->m_exception_type = NULL;
generator->m_exception_value = NULL;
generator->m_exception_tb = NULL;
return NULL;
}
Py_INCREF( generator->m_exception_type );
Py_XINCREF( generator->m_exception_value );
Py_XINCREF( generator->m_exception_tb );
if ( PyExceptionClass_Check( generator->m_exception_type ))
{
NORMALIZE_EXCEPTION( &generator->m_exception_type, &generator->m_exception_value, &generator->m_exception_tb );
}
else if ( PyExceptionInstance_Check( generator->m_exception_type ) )
{
if ( generator->m_exception_value && generator->m_exception_value != Py_None )
{
PyErr_Format( PyExc_TypeError, "instance exception may not have a separate value" );
return NULL;
}
Py_XDECREF( generator->m_exception_value );
generator->m_exception_value = generator->m_exception_type;
generator->m_exception_type = INCREASE_REFCOUNT( PyExceptionInstance_Class( generator->m_exception_type ) );
}
else
{
PyErr_Format(
PyExc_TypeError,
#if PYTHON_VERSION < 300
"exceptions must be classes, or instances, not %s",
#else
"exceptions must be classes or instances deriving from BaseException, not %s",
#endif
Py_TYPE( generator->m_exception_type )->tp_name
);
return NULL;
}
if ( ( generator->m_exception_tb != NULL ) && ( (PyObject *)generator->m_exception_tb != Py_None ) && ( !PyTraceBack_Check( generator->m_exception_tb ) ) )
{
PyErr_Format( PyExc_TypeError, "throw() third argument must be a traceback object" );
return NULL;
}
PyObject *exception_type = generator->m_exception_type;
PyObject *exception_value = generator->m_exception_value;
PyTracebackObject *exception_tb = generator->m_exception_tb;
if ( generator->m_status != status_Finished )
{
PyObject *result = Nuitka_Generator_send( generator, Py_None );
Py_DECREF( exception_type );
Py_XDECREF( exception_value );
Py_XDECREF( exception_tb );
return result;
}
else
{
PyErr_Restore( generator->m_exception_type, generator->m_exception_value, (PyObject *)generator->m_exception_tb );
return NULL;
}
}
static PyObject *Nuitka_Generator_get_name( Nuitka_GeneratorObject *generator )
{
return INCREASE_REFCOUNT( generator->m_name );
}
static PyObject *Nuitka_Generator_get_code( Nuitka_GeneratorObject *object )
{
return INCREASE_REFCOUNT( (PyObject *)object->m_code_object );
}
static int Nuitka_Generator_set_code( Nuitka_GeneratorObject *object, PyObject *value )
{
PyErr_Format( PyExc_RuntimeError, "gi_code is not writable in Nuitka" );
return -1;
}
static PyObject *Nuitka_Generator_get_frame( Nuitka_GeneratorObject *object )
{
if ( object->m_frame )
{
return INCREASE_REFCOUNT( (PyObject *)object->m_frame );
}
else
{
return INCREASE_REFCOUNT( Py_None );
}
}
static int Nuitka_Generator_set_frame( Nuitka_GeneratorObject *object, PyObject *value )
{
PyErr_Format( PyExc_RuntimeError, "gi_frame is not writable in Nuitka" );
return -1;
}
static PyGetSetDef Nuitka_Generator_getsetlist[] =
{
{ (char *)"__name__", (getter)Nuitka_Generator_get_name, NULL, NULL },
{ (char *)"gi_code", (getter)Nuitka_Generator_get_code, (setter)Nuitka_Generator_set_code, NULL },
{ (char *)"gi_frame", (getter)Nuitka_Generator_get_frame, (setter)Nuitka_Generator_set_frame, NULL },
{ NULL }
};
static PyMethodDef Nuitka_Generator_methods[] =
{
{ "send", (PyCFunction)Nuitka_Generator_send, METH_O, NULL },
{ "throw", (PyCFunction)Nuitka_Generator_throw, METH_VARARGS, NULL },
{ "close", (PyCFunction)Nuitka_Generator_close, METH_NOARGS, NULL },
{ NULL }
};
#include
static PyMemberDef Nuitka_Generator_members[] =
{
{ (char *)"gi_running", T_INT, offsetof( Nuitka_GeneratorObject, m_running ), READONLY },
{ NULL }
};
PyTypeObject Nuitka_Generator_Type =
{
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"compiled_generator", // tp_name
sizeof(Nuitka_GeneratorObject), // tp_basicsize
0, // tp_itemsize
(destructor)Nuitka_Generator_tp_dealloc, // tp_dealloc
0, // tp_print
0, // tp_getattr
0, // tp_setattr
0, // tp_compare
(reprfunc)Nuitka_Generator_tp_repr, // tp_repr
0, // tp_as_number
0, // tp_as_sequence
0, // tp_as_mapping
0, // tp_hash
0, // tp_call
0, // tp_str
PyObject_GenericGetAttr, // tp_getattro
0, // tp_setattro
0, // tp_as_buffer
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,
// tp_flags
0, // tp_doc
(traverseproc)Nuitka_Generator_tp_traverse, // tp_traverse
0, // tp_clear
0, // tp_richcompare
offsetof( Nuitka_GeneratorObject, m_weakrefs ), // tp_weaklistoffset
PyObject_SelfIter, // tp_iter
(iternextfunc)Nuitka_Generator_tp_iternext, // tp_iternext
Nuitka_Generator_methods, // tp_methods
Nuitka_Generator_members, // tp_members
Nuitka_Generator_getsetlist, // tp_getset
0, // tp_base
0, // tp_dict
0, // tp_descr_get
0, // tp_descr_set
0, // tp_dictoffset
0, // tp_init
0, // tp_alloc
0, // tp_new
0, // tp_free
0, // tp_is_gc
0, // tp_bases
0, // tp_mro
0, // tp_cache
0, // tp_subclasses
0, // tp_weaklist
0 // tp_del
};
PyObject *Nuitka_Generator_New( yielder_func code, PyObject *name, PyCodeObject *code_object, void *context, releaser cleanup )
{
Nuitka_GeneratorObject *result = PyObject_GC_New( Nuitka_GeneratorObject, &Nuitka_Generator_Type );
if (unlikely( result == NULL ))
{
PyErr_Format(
PyExc_RuntimeError,
"cannot create genexpr %s",
Nuitka_String_AsString( name )
);
throw PythonException();
}
result->m_code = (void *)code;
result->m_name = INCREASE_REFCOUNT( name );
result->m_context = context;
result->m_cleanup = cleanup;
result->m_weakrefs = NULL;
result->m_status = status_Unused;
result->m_running = false;
initFiber( &result->m_yielder_context );
result->m_exception_type = NULL;
result->m_yielded = NULL;
result->m_frame = NULL;
result->m_code_object = code_object;
Nuitka_GC_Track( result );
return (PyObject *)result;
}
PyObject *Nuitka_Generator_New( yielder_func code, PyObject *name, PyCodeObject *code_object )
{
return Nuitka_Generator_New( code, name, code_object, NULL, NULL );
}
#if PYTHON_VERSION >= 330
// This is for CPython iterator objects, the respective code is not exported as
// API, so we need to redo it. This is an re-implementation that closely follows
// what it does. It's unrelated to compiled generators.
PyObject *PyGen_Send( PyGenObject *generator, PyObject *arg )
{
if (unlikely( generator->gi_running ))
{
PyErr_SetString( PyExc_ValueError, "generator already executing" );
return NULL;
}
PyFrameObject *frame = generator->gi_frame;
if ( frame == NULL || frame->f_stacktop == NULL )
{
// Set exception if called from send()
if ( arg != NULL )
{
PyErr_SetNone( PyExc_StopIteration );
}
return NULL;
}
if ( frame->f_lasti == -1 )
{
if (unlikely( arg && arg != Py_None ))
{
PyErr_SetString(
PyExc_TypeError,
"can't send non-None value to a just-started generator"
);
return NULL;
}
}
else
{
// Put arg on top of the value stack
PyObject *tmp = arg ? arg : Py_None;
*(frame->f_stacktop++) = INCREASE_REFCOUNT( tmp );
}
// Generators always return to their most recent caller, not necessarily
// their creator.
PyThreadState *tstate = PyThreadState_GET();
Py_XINCREF( tstate->frame );
assert( frame->f_back == NULL );
frame->f_back = tstate->frame;
generator->gi_running = 1;
PyObject *result = PyEval_EvalFrameEx( frame, 0 );
generator->gi_running = 0;
// Don't keep the reference to f_back any longer than necessary. It
// may keep a chain of frames alive or it could create a reference
// cycle.
assert( frame->f_back == tstate->frame );
Py_CLEAR( frame->f_back );
// If the generator just returned (as opposed to yielding), signal that the
// generator is exhausted.
if ( result && frame->f_stacktop == NULL )
{
if ( result == Py_None )
{
PyErr_SetNone( PyExc_StopIteration );
}
else {
PyObject *e = PyObject_CallFunctionObjArgs(
PyExc_StopIteration,
result,
NULL
);
if ( e != NULL )
{
PyErr_SetObject( PyExc_StopIteration, e );
Py_DECREF( e );
}
}
Py_CLEAR( result );
}
if ( result == NULL || frame->f_stacktop == NULL )
{
// Generator is finished, remove exception from frame before releasing
// it.
PyObject *type = frame->f_exc_type;
PyObject *value = frame->f_exc_value;
PyObject *traceback = frame->f_exc_traceback;
frame->f_exc_type = NULL;
frame->f_exc_value = NULL;
frame->f_exc_traceback = NULL;
Py_XDECREF( type );
Py_XDECREF( value );
Py_XDECREF( traceback );
// Now release frame.
generator->gi_frame = NULL;
Py_DECREF( frame );
}
return result;
}
PyObject *ERROR_GET_STOP_ITERATION_VALUE()
{
assert ( PyErr_ExceptionMatches( PyExc_StopIteration ));
PyObject *et, *ev, *tb;
PyErr_Fetch( &et, &ev, &tb );
Py_XDECREF(et);
Py_XDECREF(tb);
PyObject *value = NULL;
if ( ev )
{
if ( PyErr_GivenExceptionMatches( ev, PyExc_StopIteration ) )
{
value = ((PyStopIterationObject *)ev)->value;
Py_DECREF( ev );
}
else
{
value = ev;
}
}
if ( value == NULL )
{
value = INCREASE_REFCOUNT( Py_None );
}
return value;
}
#endif
Nuitka-0.5.0.1/nuitka/build/static_src/CompiledCodeHelpers.cpp 0000644 0001750 0001750 00000210140 12265270763 024456 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// Implementations of compiled code helpers.
// The definition of a compiled code helper is that it's being used in
// generated C++ code and provides part of the operations implementation.
// Currently we also have standalone mode related code here, patches to CPython
// runtime that we do, and e.g. the built-in module. TODO: Move these to their
// own files for clarity.
#include "nuitka/prelude.hpp"
extern PyObject *const_str_plain_compile;
extern PyObject *const_str_plain_strip;
extern PyObject *const_str_plain_read;
static PythonBuiltin _python_builtin_compile( &const_str_plain_compile );
PyObject *COMPILE_CODE( PyObject *source_code, PyObject *file_name, PyObject *mode, int flags )
{
// May be a source, but also could already be a compiled object, in which
// case this should just return it.
if ( PyCode_Check( source_code ) )
{
return INCREASE_REFCOUNT( source_code );
}
// Workaround leading whitespace causing a trouble to "compile" builtin, but
// not "eval" builtin
PyObject *source;
if (
(
#if PYTHON_VERSION < 300
PyString_Check( source_code ) ||
#endif
PyUnicode_Check( source_code )
) &&
strcmp( Nuitka_String_AsString( mode ), "exec" ) != 0
)
{
source = PyObject_CallMethodObjArgs( source_code, const_str_plain_strip, NULL );
if (unlikely( source == NULL ))
{
throw PythonException();
}
}
#if PYTHON_VERSION < 300
// Note: Python3 does not support "exec" with file handles.
else if ( PyFile_Check( source_code ) && strcmp( Nuitka_String_AsString( mode ), "exec" ) == 0 )
{
source = PyObject_CallMethodObjArgs( source_code, const_str_plain_read, NULL );
if (unlikely( source == NULL ))
{
throw PythonException();
}
}
#endif
else
{
source = INCREASE_REFCOUNT( source_code );
}
PyObjectTemporary source_temp( source );
PyObjectTemporary future_flags( PyInt_FromLong( flags ) );
return _python_builtin_compile.call_args(
MAKE_TUPLE5(
source,
file_name,
mode,
future_flags.asObject0(), // flags
Py_True // dont_inherit
)
);
}
extern PyObject *const_str_plain_open;
static PythonBuiltin _python_builtin_open( &const_str_plain_open );
PyObject *OPEN_FILE( PyObject *file_name, PyObject *mode, PyObject *buffering )
{
if ( file_name == NULL )
{
return _python_builtin_open.call();
}
else if ( mode == NULL )
{
return _python_builtin_open.call1(
file_name
);
}
else if ( buffering == NULL )
{
return _python_builtin_open.call2(
file_name,
mode
);
}
else
{
return _python_builtin_open.call3(
file_name,
mode,
buffering
);
}
}
PyObject *BUILTIN_CHR( unsigned char c )
{
// TODO: A switch statement might be faster, because no object needs to be
// created at all, this here is how CPython does it.
char s[1];
s[0] = (char)c;
#if PYTHON_VERSION < 300
return PyString_FromStringAndSize( s, 1 );
#else
return PyUnicode_FromStringAndSize( s, 1 );
#endif
}
PyObject *BUILTIN_CHR( PyObject *value )
{
long x = PyInt_AsLong( value );
#if PYTHON_VERSION < 300
if ( x < 0 || x >= 256 )
{
PyErr_Format( PyExc_ValueError, "chr() arg not in range(256)" );
throw PythonException();
}
// TODO: A switch statement might be faster, because no object needs to be
// created at all, this is how CPython does it.
char s[1];
s[0] = (char)x;
return PyString_FromStringAndSize( s, 1 );
#else
PyObject *result = PyUnicode_FromOrdinal( x );
if (unlikely( result == NULL ))
{
throw PythonException();
}
assert( PyUnicode_Check( result ));
return result;
#endif
}
PyObject *BUILTIN_ORD( PyObject *value )
{
long result;
if (likely( PyBytes_Check( value ) ))
{
Py_ssize_t size = PyBytes_GET_SIZE( value );
if (likely( size == 1 ))
{
result = long( ((unsigned char *)PyBytes_AS_STRING( value ))[0] );
}
else
{
PyErr_Format( PyExc_TypeError, "ord() expected a character, but string of length %zd found", size );
throw PythonException();
}
}
else if ( PyByteArray_Check( value ) )
{
Py_ssize_t size = PyByteArray_GET_SIZE( value );
if (likely( size == 1 ))
{
result = long( ((unsigned char *)PyByteArray_AS_STRING( value ))[0] );
}
else
{
PyErr_Format( PyExc_TypeError, "ord() expected a character, but byte array of length %zd found", size );
throw PythonException();
}
}
else if ( PyUnicode_Check( value ) )
{
#if PYTHON_VERSION >= 330
if (unlikely( PyUnicode_READY( value ) == -1 ))
{
throw PythonException();
}
Py_ssize_t size = PyUnicode_GET_LENGTH( value );
#else
Py_ssize_t size = PyUnicode_GET_SIZE( value );
#endif
if (likely( size == 1 ))
{
#if PYTHON_VERSION >= 330
result = long( PyUnicode_READ_CHAR( value, 0 ) );
#else
result = long( *PyUnicode_AS_UNICODE( value ) );
#endif
}
else
{
PyErr_Format( PyExc_TypeError, "ord() expected a character, but unicode string of length %zd found", size );
throw PythonException();
}
}
else
{
PyErr_Format( PyExc_TypeError, "ord() expected string of length 1, but %s found", Py_TYPE( value )->tp_name );
throw PythonException();
}
return PyInt_FromLong( result );
}
PyObject *BUILTIN_BIN( PyObject *value )
{
// Note: I don't really know why ord and hex don't use this as well.
PyObject *result = PyNumber_ToBase( value, 2 );
if ( unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
PyObject *BUILTIN_OCT( PyObject *value )
{
#if PYTHON_VERSION >= 300
PyObject *result = PyNumber_ToBase( value, 8 );
if ( unlikely( result == NULL ))
{
throw PythonException();
}
return result;
#else
if (unlikely( value == NULL ))
{
PyErr_Format( PyExc_TypeError, "oct() argument can't be converted to oct" );
return NULL;
}
PyNumberMethods *nb = Py_TYPE( value )->tp_as_number;
if (unlikely( nb == NULL || nb->nb_oct == NULL ))
{
PyErr_Format( PyExc_TypeError, "oct() argument can't be converted to oct" );
return NULL;
}
PyObject *result = (*nb->nb_oct)( value );
if ( result )
{
if (unlikely( !PyString_Check( result ) ))
{
PyErr_Format( PyExc_TypeError, "__oct__ returned non-string (type %s)", Py_TYPE( result )->tp_name );
Py_DECREF( result );
return NULL;
}
}
return result;
#endif
}
PyObject *BUILTIN_HEX( PyObject *value )
{
#if PYTHON_VERSION >= 300
PyObject *result = PyNumber_ToBase( value, 16 );
if ( unlikely( result == NULL ))
{
throw PythonException();
}
return result;
#else
if (unlikely( value == NULL ))
{
PyErr_Format( PyExc_TypeError, "hex() argument can't be converted to hex" );
return NULL;
}
PyNumberMethods *nb = Py_TYPE( value )->tp_as_number;
if (unlikely( nb == NULL || nb->nb_hex == NULL ))
{
PyErr_Format( PyExc_TypeError, "hex() argument can't be converted to hex" );
return NULL;
}
PyObject *result = (*nb->nb_hex)( value );
if ( result )
{
if (unlikely( !PyString_Check( result ) ))
{
PyErr_Format( PyExc_TypeError, "__hex__ returned non-string (type %s)", Py_TYPE( result )->tp_name );
Py_DECREF( result );
return NULL;
}
}
return result;
#endif
}
// From CPython:
typedef struct {
PyObject_HEAD
PyObject *it_callable;
PyObject *it_sentinel;
} calliterobject;
PyObject *BUILTIN_ITER2( PyObject *callable, PyObject *sentinel )
{
calliterobject *result = PyObject_GC_New( calliterobject, &PyCallIter_Type );
if (unlikely( result == NULL ))
{
throw PythonException();
}
// Note: References were taken at call site already.
result->it_callable = INCREASE_REFCOUNT( callable );
result->it_sentinel = INCREASE_REFCOUNT( sentinel );
Nuitka_GC_Track( result );
return (PyObject *)result;
}
PyObject *BUILTIN_TYPE1( PyObject *arg )
{
return INCREASE_REFCOUNT( (PyObject *)Py_TYPE( arg ) );
}
extern PyObject *const_str_plain___module__;
PyObject *BUILTIN_TYPE3( PyObject *module_name, PyObject *name, PyObject *bases, PyObject *dict )
{
PyObject *result = PyType_Type.tp_new(
&PyType_Type,
PyObjectTemporary( MAKE_TUPLE3( name, bases, dict ) ).asObject0(),
NULL
);
if (unlikely( result == NULL ))
{
throw PythonException();
}
PyTypeObject *type = Py_TYPE( result );
if (likely( PyType_IsSubtype( type, &PyType_Type ) ))
{
if (
#if PYTHON_VERSION < 300
PyType_HasFeature( type, Py_TPFLAGS_HAVE_CLASS ) &&
#endif
type->tp_init != NULL
)
{
int res = type->tp_init( result, MAKE_TUPLE3( name, bases, dict ), NULL );
if (unlikely( res < 0 ))
{
Py_DECREF( result );
throw PythonException();
}
}
}
int res = PyObject_SetAttr( result, const_str_plain___module__, module_name );
if ( res == -1 )
{
throw PythonException();
}
return result;
}
Py_ssize_t ESTIMATE_RANGE( long low, long high, long step )
{
if ( low >= high )
{
return 0;
}
else
{
return ( high - low - 1 ) / step + 1;
}
}
#if PYTHON_VERSION < 300
static PyObject *_BUILTIN_RANGE_INT3( long low, long high, long step )
{
assert( step != 0 );
Py_ssize_t size;
if ( step > 0 )
{
size = ESTIMATE_RANGE( low, high, step );
}
else
{
size = ESTIMATE_RANGE( high, low, -step );
}
PyObject *result = PyList_New( size );
long current = low;
for ( int i = 0; i < size; i++ )
{
PyList_SET_ITEM( result, i, PyInt_FromLong( current ) );
current += step;
}
return result;
}
static PyObject *_BUILTIN_RANGE_INT2( long low, long high )
{
return _BUILTIN_RANGE_INT3( low, high, 1 );
}
static PyObject *_BUILTIN_RANGE_INT( long boundary )
{
return _BUILTIN_RANGE_INT2( 0, boundary );
}
static PyObject *TO_RANGE_ARG( PyObject *value, char const *name )
{
if (likely( PyInt_Check( value ) || PyLong_Check( value ) ))
{
return INCREASE_REFCOUNT( value );
}
PyTypeObject *type = Py_TYPE( value );
PyNumberMethods *tp_as_number = type->tp_as_number;
// Everything that casts to int is allowed.
if (
#if PYTHON_VERSION >= 270
PyFloat_Check( value ) ||
#endif
tp_as_number == NULL ||
tp_as_number->nb_int == NULL
)
{
PyErr_Format( PyExc_TypeError, "range() integer %s argument expected, got %s.", name, type->tp_name );
throw PythonException();
}
PyObject *result = tp_as_number->nb_int( value );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
#endif
extern PyObject *const_str_plain_range;
static PythonBuiltin _python_builtin_range( &const_str_plain_range );
PyObject *BUILTIN_RANGE( PyObject *boundary )
{
#if PYTHON_VERSION < 300
PyObjectTemporary boundary_temp( TO_RANGE_ARG( boundary, "end" ) );
long start = PyInt_AsLong( boundary_temp.asObject0() );
if ( start == -1 && ERROR_OCCURED() )
{
PyErr_Clear();
return _python_builtin_range.call1( boundary_temp.asObject0() );
}
return _BUILTIN_RANGE_INT( start );
#else
return _python_builtin_range.call1( boundary );
#endif
}
PyObject *BUILTIN_RANGE2( PyObject *low, PyObject *high )
{
#if PYTHON_VERSION < 300
PyObjectTemporary low_temp( TO_RANGE_ARG( low, "start" ) );
PyObjectTemporary high_temp( TO_RANGE_ARG( high, "end" ) );
bool fallback = false;
long start = PyInt_AsLong( low_temp.asObject0() );
if (unlikely( start == -1 && ERROR_OCCURED() ))
{
PyErr_Clear();
fallback = true;
}
long end = PyInt_AsLong( high_temp.asObject0() );
if (unlikely( end == -1 && ERROR_OCCURED() ))
{
PyErr_Clear();
fallback = true;
}
if ( fallback )
{
return _python_builtin_range.call_args(
MAKE_TUPLE2(
low_temp.asObject0(),
high_temp.asObject0()
)
);
}
else
{
return _BUILTIN_RANGE_INT2( start, end );
}
#else
return _python_builtin_range.call_args(
MAKE_TUPLE2( low, high )
);
#endif
}
PyObject *BUILTIN_RANGE3( PyObject *low, PyObject *high, PyObject *step )
{
#if PYTHON_VERSION < 300
PyObjectTemporary low_temp( TO_RANGE_ARG( low, "start" ) );
PyObjectTemporary high_temp( TO_RANGE_ARG( high, "end" ) );
PyObjectTemporary step_temp( TO_RANGE_ARG( step, "step" ) );
bool fallback = false;
long start = PyInt_AsLong( low_temp.asObject0() );
if (unlikely( start == -1 && ERROR_OCCURED() ))
{
PyErr_Clear();
fallback = true;
}
long end = PyInt_AsLong( high_temp.asObject0() );
if (unlikely( end == -1 && ERROR_OCCURED() ))
{
PyErr_Clear();
fallback = true;
}
long step_long = PyInt_AsLong( step_temp.asObject0() );
if (unlikely( step_long == -1 && ERROR_OCCURED() ))
{
PyErr_Clear();
fallback = true;
}
if ( fallback )
{
return _python_builtin_range.call_args(
MAKE_TUPLE3(
low_temp.asObject0(),
high_temp.asObject0(),
step_temp.asObject0()
)
);
}
else
{
if (unlikely( step_long == 0 ))
{
PyErr_Format( PyExc_ValueError, "range() step argument must not be zero" );
throw PythonException();
}
return _BUILTIN_RANGE_INT3( start, end, step_long );
}
#else
return _python_builtin_range.call_args(
MAKE_TUPLE3( low, high, step )
);
#endif
}
#if PYTHON_VERSION < 300
extern PyObject *const_str_plain_xrange;
static PythonBuiltin _python_builtin_xrange( &const_str_plain_xrange );
PyObject *BUILTIN_XRANGE( PyObject *low, PyObject *high, PyObject *step )
{
if ( step != NULL )
{
return _python_builtin_xrange.call3( low, high, step );
}
else if ( high != NULL )
{
return _python_builtin_xrange.call2( low, high );
}
else
{
return _python_builtin_xrange.call1( low );
}
}
#endif
PyObject *BUILTIN_LEN( PyObject *value )
{
assertObject( value );
Py_ssize_t res = PyObject_Size( value );
if (unlikely( res < 0 && ERROR_OCCURED() ))
{
throw PythonException();
}
return PyInt_FromSsize_t( res );
}
PyObject *BUILTIN_DIR1( PyObject *arg )
{
assertObject( arg );
PyObject *result = PyObject_Dir( arg );
if (unlikely( result == NULL ))
{
throw PythonException();
}
return result;
}
extern PyObject *const_str_plain___import__;
static PythonBuiltin _python_builtin_import( &const_str_plain___import__ );
PyObject *IMPORT_MODULE( PyObject *module_name, PyObject *globals, PyObject *locals, PyObject *import_items, PyObject *level )
{
assertObject( module_name );
assertObject( globals );
assertObject( locals );
assertObject( import_items );
assertObject( level );
PyObject *import_result;
import_result = _python_builtin_import.call_args(
MAKE_TUPLE5(
module_name,
globals,
locals,
import_items,
level
)
);
return import_result;
}
extern PyObject *const_str_plain___all__;
void IMPORT_MODULE_STAR( PyObject *target, bool is_module, PyObject *module )
{
// Check parameters.
assertObject( module );
assertObject( target );
PyObject *iter;
bool all_case;
if ( PyObject *all = PyObject_GetAttr( module, const_str_plain___all__ ) )
{
iter = MAKE_ITERATOR( all );
all_case = true;
}
else if ( PyErr_ExceptionMatches( PyExc_AttributeError ) )
{
PyErr_Clear();
iter = MAKE_ITERATOR( PyModule_GetDict( module ) );
all_case = false;
}
else
{
throw PythonException();
}
while ( PyObject *item = ITERATOR_NEXT( iter ) )
{
assert( Nuitka_String_Check( item ) );
// TODO: Not yet clear, what happens with __all__ and "_" of its
// contents.
if ( all_case == false )
{
if ( Nuitka_String_AsString_Unchecked( item )[0] == '_' )
{
continue;
}
}
// TODO: Check if the reference is handled correctly
if ( is_module )
{
SET_ATTRIBUTE( LOOKUP_ATTRIBUTE( module, item ), target, item );
}
else
{
SET_SUBSCRIPT( LOOKUP_ATTRIBUTE( module, item ), target, item );
}
Py_DECREF( item );
}
}
// Helper functions for print. Need to play nice with Python softspace
// behaviour.
#if PYTHON_VERSION >= 300
extern PyObject *const_str_plain_print;
extern PyObject *const_str_plain_end;
extern PyObject *const_str_plain_file;
extern PyObject *const_str_empty;
static PythonBuiltin _python_builtin_print( &const_str_plain_print );
#endif
void PRINT_ITEM_TO( PyObject *file, PyObject *object )
{
// The print builtin function cannot replace "softspace" behaviour of CPython
// print statement, so this code is really necessary.
#if PYTHON_VERSION < 300
if ( file == NULL || file == Py_None )
{
file = GET_STDOUT();
}
assertObject( file );
assertObject( object );
// need to hold a reference to the file or else __getattr__ may release "file" in the
// mean time.
Py_INCREF( file );
bool softspace;
if ( PyString_Check( object ) )
{
char *buffer;
Py_ssize_t length;
#ifndef __NUITKA_NO_ASSERT__
int status =
#endif
PyString_AsStringAndSize( object, &buffer, &length );
assert( status != -1 );
softspace = length > 0 && ( buffer[ length - 1 ] == '\t' || buffer[ length - 1 ] == '\n' );
}
else
{
softspace = false;
}
// Check for soft space indicator
if ( PyFile_SoftSpace( file, !softspace ) )
{
if (unlikely( PyFile_WriteString( " ", file ) == -1 ))
{
Py_DECREF( file );
throw PythonException();
}
}
if ( unlikely( PyFile_WriteObject( object, file, Py_PRINT_RAW ) == -1 ))
{
Py_DECREF( file );
throw PythonException();
}
if ( softspace )
{
PyFile_SoftSpace( file, !softspace );
}
assertObject( file );
Py_DECREF( file );
#else
if (likely( file == NULL ))
{
_python_builtin_print.call1(
object
);
}
else
{
PyObjectTemporary print_kw(
MAKE_DICT2(
const_str_empty, const_str_plain_end,
GET_STDOUT(), const_str_plain_file
)
);
PyObjectTemporary print_args(
MAKE_TUPLE1( object )
);
_python_builtin_print.call_args_kw(
print_args.asObject0(),
print_kw.asObject0()
);
}
#endif
}
void PRINT_NEW_LINE_TO( PyObject *file )
{
#if PYTHON_VERSION < 300
if ( file == NULL || file == Py_None )
{
file = GET_STDOUT();
}
// need to hold a reference to the file or else __getattr__ may release "file" in the
// mean time.
Py_INCREF( file );
if (unlikely( PyFile_WriteString( "\n", file ) == -1))
{
Py_DECREF( file );
throw PythonException();
}
PyFile_SoftSpace( file, 0 );
assertObject( file );
Py_DECREF( file );
#else
if (likely( file == NULL ))
{
_python_builtin_print.call();
}
else
{
PyObjectTemporary print_keyargs(
MAKE_DICT1( // Note: Values for first for MAKE_DICT
GET_STDOUT(), const_str_plain_file
)
);
_python_builtin_print.call_kw(
print_keyargs.asObject0()
);
}
#endif
}
void PRINT_REFCOUNT( PyObject *object )
{
#if PYTHON_VERSION < 300
char buffer[ 1024 ];
sprintf( buffer, " refcnt %" PY_FORMAT_SIZE_T "d ", Py_REFCNT( object ) );
if (unlikely( PyFile_WriteString( buffer, GET_STDOUT() ) == -1 ))
{
throw PythonException();
}
#else
assert( false );
#endif
}
PyObject *GET_STDOUT()
{
PyObject *result = PySys_GetObject( (char *)"stdout" );
if (unlikely( result == NULL ))
{
PyErr_Format( PyExc_RuntimeError, "lost sys.stdout" );
throw PythonException();
}
return result;
}
PyObject *GET_STDERR()
{
PyObject *result = PySys_GetObject( (char *)"stderr" );
if (unlikely( result == NULL ))
{
PyErr_Format( PyExc_RuntimeError, "lost sys.stderr" );
throw PythonException();
}
return result;
}
#if PYTHON_VERSION < 300
void PRINT_NEW_LINE( void )
{
PRINT_NEW_LINE_TO( GET_STDOUT() );
}
#endif
PyObject *UNSTREAM_CONSTANT( unsigned char const *buffer, Py_ssize_t size )
{
assert( buffer );
// We unstream difficult constant objects using the "pickle" module, this is
// aimed at being the exception, e.g. unicode that doesn't fit into UTF-8
// will be dealt with like this.
static PyObject *module_pickle = NULL;
if ( module_pickle == NULL )
{
#if PYTHON_VERSION < 300
module_pickle = PyImport_ImportModule( "cPickle" );
#else
module_pickle = PyImport_ImportModule( "pickle" );
#endif
if (unlikely( module_pickle == NULL ))
{
PyErr_Print();
}
assert( module_pickle );
}
static PyObject *function_pickle_loads = NULL;
if ( function_pickle_loads == NULL )
{
function_pickle_loads = PyObject_GetAttrString(
module_pickle,
"loads"
);
if (unlikely( function_pickle_loads == NULL ))
{
PyErr_Print();
}
assert( function_pickle_loads );
}
PyObject *result = PyObject_CallFunction(
function_pickle_loads,
#if PYTHON_VERSION < 300
(char *)"(s#)", // TODO: Why the ()
#else
(char *)"y#",
#endif
buffer,
size
);
if (unlikely( result == NULL ))
{
PyErr_Print();
}
assertObject( result );
return result;
}
#if PYTHON_VERSION < 300
PyObject *UNSTREAM_UNICODE( unsigned char const *buffer, Py_ssize_t size )
{
PyObject *result = PyUnicode_FromStringAndSize( (char const *)buffer, size );
assert( !ERROR_OCCURED() );
assertObject( result );
return result;
}
#endif
PyObject *UNSTREAM_STRING( unsigned char const *buffer, Py_ssize_t size, bool intern )
{
#if PYTHON_VERSION < 300
PyObject *result = PyString_FromStringAndSize( (char const *)buffer, size );
#else
PyObject *result = PyUnicode_FromStringAndSize( (char const *)buffer, size );
#endif
assert( !ERROR_OCCURED() );
assertObject( result );
assert( Nuitka_String_Check( result ) );
#if PYTHON_VERSION < 300
assert( PyString_Size( result ) == size );
#endif
if ( intern )
{
Nuitka_StringIntern( &result );
assertObject( result );
assert( Nuitka_String_Check( result ) );
#if PYTHON_VERSION < 300
assert( PyString_Size( result ) == size );
#else
assert( PyUnicode_GET_SIZE( result ) == size );
#endif
}
return result;
}
PyObject *UNSTREAM_CHAR( unsigned char value, bool intern )
{
#if PYTHON_VERSION < 300
PyObject *result = PyString_FromStringAndSize( (char const *)&value, 1 );
#else
PyObject *result = PyUnicode_FromStringAndSize( (char const *)&value, 1 );
#endif
assert( !ERROR_OCCURED() );
assertObject( result );
assert( Nuitka_String_Check( result ) );
#if PYTHON_VERSION < 300
assert( PyString_Size( result ) == 1 );
#else
assert( PyUnicode_GET_SIZE( result ) == 1 );
#endif
if ( intern )
{
Nuitka_StringIntern( &result );
assertObject( result );
assert( Nuitka_String_Check( result ) );
#if PYTHON_VERSION < 300
assert( PyString_Size( result ) == 1 );
#else
assert( PyUnicode_GET_SIZE( result ) == 1 );
#endif
}
return result;
}
PyObject *UNSTREAM_FLOAT( unsigned char const *buffer )
{
double x = _PyFloat_Unpack8( buffer, 1 );
assert( x != -1.0 || !PyErr_Occurred() );
PyObject *result = PyFloat_FromDouble(x);
assert( result != NULL );
return result;
}
#if PYTHON_VERSION >= 300
PyObject *UNSTREAM_BYTES( unsigned char const *buffer, Py_ssize_t size )
{
PyObject *result = PyBytes_FromStringAndSize( (char const *)buffer, size );
assert( !ERROR_OCCURED() );
assertObject( result );
assert( PyBytes_GET_SIZE( result ) == size );
return result;
}
#endif
#if PYTHON_VERSION < 300
static void set_slot( PyObject **slot, PyObject *value )
{
PyObject *temp = *slot;
Py_XINCREF( value );
*slot = value;
Py_XDECREF( temp );
}
extern PyObject *const_str_plain___getattr__;
extern PyObject *const_str_plain___setattr__;
extern PyObject *const_str_plain___delattr__;
static void set_attr_slots( PyClassObject *klass )
{
set_slot( &klass->cl_getattr, FIND_ATTRIBUTE_IN_CLASS( klass, const_str_plain___getattr__ ) );
set_slot( &klass->cl_setattr, FIND_ATTRIBUTE_IN_CLASS( klass, const_str_plain___setattr__ ) );
set_slot( &klass->cl_delattr, FIND_ATTRIBUTE_IN_CLASS( klass, const_str_plain___delattr__ ) );
}
static bool set_dict( PyClassObject *klass, PyObject *value )
{
if ( value == NULL || !PyDict_Check( value ) )
{
PyErr_SetString( PyExc_TypeError, (char *)"__dict__ must be a dictionary object" );
return false;
}
else
{
set_slot( &klass->cl_dict, value );
set_attr_slots( klass );
return true;
}
}
static bool set_bases( PyClassObject *klass, PyObject *value )
{
if ( value == NULL || !PyTuple_Check( value ) )
{
PyErr_SetString( PyExc_TypeError, (char *)"__bases__ must be a tuple object" );
return false;
}
else
{
Py_ssize_t n = PyTuple_Size( value );
for ( Py_ssize_t i = 0; i < n; i++ )
{
PyObject *base = PyTuple_GET_ITEM( value, i );
if (unlikely( !PyClass_Check( base ) ))
{
PyErr_SetString( PyExc_TypeError, (char *)"__bases__ items must be classes" );
return false;
}
if (unlikely( PyClass_IsSubclass( base, (PyObject *)klass) ))
{
PyErr_SetString( PyExc_TypeError, (char *)"a __bases__ item causes an inheritance cycle" );
return false;
}
}
set_slot( &klass->cl_bases, value );
set_attr_slots( klass );
return true;
}
}
static bool set_name( PyClassObject *klass, PyObject *value )
{
if ( value == NULL || !PyDict_Check( value ) )
{
PyErr_SetString( PyExc_TypeError, (char *)"__name__ must be a string object" );
return false;
}
if ( strlen( PyString_AS_STRING( value )) != (size_t)PyString_GET_SIZE( value ) )
{
PyErr_SetString( PyExc_TypeError, (char *)"__name__ must not contain null bytes" );
return false;
}
set_slot( &klass->cl_name, value );
return true;
}
static int nuitka_class_setattr( PyClassObject *klass, PyObject *attr_name, PyObject *value )
{
char *sattr_name = PyString_AsString( attr_name );
if ( sattr_name[0] == '_' && sattr_name[1] == '_' )
{
Py_ssize_t n = PyString_Size( attr_name );
if ( sattr_name[ n-2 ] == '_' && sattr_name[ n-1 ] == '_' )
{
if ( strcmp( sattr_name, "__dict__" ) == 0 )
{
if ( set_dict( klass, value ) == false )
{
return -1;
}
else
{
return 0;
}
}
else if ( strcmp( sattr_name, "__bases__" ) == 0 )
{
if ( set_bases( klass, value ) == false )
{
return -1;
}
else
{
return 0;
}
}
else if ( strcmp( sattr_name, "__name__" ) == 0 )
{
if ( set_name( klass, value ) == false )
{
return -1;
}
else
{
return 0;
}
}
else if ( strcmp( sattr_name, "__getattr__" ) == 0 )
{
set_slot( &klass->cl_getattr, value );
}
else if ( strcmp(sattr_name, "__setattr__" ) == 0 )
{
set_slot( &klass->cl_setattr, value );
}
else if ( strcmp(sattr_name, "__delattr__" ) == 0 )
{
set_slot( &klass->cl_delattr, value );
}
}
}
if ( value == NULL )
{
int status = PyDict_DelItem( klass->cl_dict, attr_name );
if ( status < 0 )
{
PyErr_Format( PyExc_AttributeError, "class %s has no attribute '%s'", PyString_AS_STRING( klass->cl_name ), sattr_name );
}
return status;
}
else
{
return PyDict_SetItem( klass->cl_dict, attr_name, value );
}
}
static PyObject *nuitka_class_getattr( PyClassObject *klass, PyObject *attr_name )
{
char *sattr_name = PyString_AsString( attr_name );
if ( sattr_name[0] == '_' && sattr_name[1] == '_' )
{
if ( strcmp( sattr_name, "__dict__" ) == 0 )
{
return INCREASE_REFCOUNT( klass->cl_dict );
}
else if ( strcmp(sattr_name, "__bases__" ) == 0 )
{
return INCREASE_REFCOUNT( klass->cl_bases );
}
else if ( strcmp(sattr_name, "__name__" ) == 0 )
{
return klass->cl_name ? INCREASE_REFCOUNT( klass->cl_name ) : INCREASE_REFCOUNT( Py_None );
}
}
PyObject *value = FIND_ATTRIBUTE_IN_CLASS( klass, attr_name );
if (unlikely( value == NULL ))
{
PyErr_Format( PyExc_AttributeError, "class %s has no attribute '%s'", PyString_AS_STRING( klass->cl_name ), sattr_name );
return NULL;
}
PyTypeObject *type = Py_TYPE( value );
descrgetfunc tp_descr_get = PyType_HasFeature( type, Py_TPFLAGS_HAVE_CLASS ) ? type->tp_descr_get : NULL;
if ( tp_descr_get == NULL )
{
return INCREASE_REFCOUNT( value );
}
else
{
return tp_descr_get( value, (PyObject *)NULL, (PyObject *)klass );
}
}
#endif
void enhancePythonTypes( void )
{
#if PYTHON_VERSION < 300
// Our own variant won't call PyEval_GetRestricted, saving quite some cycles
// not doing that.
PyClass_Type.tp_setattro = (setattrofunc)nuitka_class_setattr;
PyClass_Type.tp_getattro = (getattrofunc)nuitka_class_getattr;
#endif
}
#ifdef __APPLE__
extern "C" wchar_t* _Py_DecodeUTF8_surrogateescape(const char *s, Py_ssize_t size);
#endif
#ifdef __FreeBSD__
#include
#endif
#include
#if PYTHON_VERSION >= 300
static wchar_t **argv_copy = NULL;
#endif
void setCommandLineParameters( int argc, char *argv[], bool initial )
{
#if PYTHON_VERSION < 300
if ( initial )
{
Py_SetProgramName( argv[0] );
}
else
{
PySys_SetArgv( argc, argv );
}
#else
if ( initial )
{
// Originally taken from CPython3: There seems to be no sane way to use
argv_copy = (wchar_t **)PyMem_Malloc(sizeof(wchar_t*)*argc);
#ifdef __FreeBSD__
// 754 requires that FP exceptions run in "no stop" mode by default, and
// until C vendors implement C99's ways to control FP exceptions, Python
// requires non-stop mode. Alas, some platforms enable FP exceptions by
// default. Here we disable them.
fp_except_t m;
m = fpgetmask();
fpsetmask( m & ~FP_X_OFL );
#endif
char *oldloc = strdup( setlocale( LC_ALL, NULL ) );
setlocale( LC_ALL, "" );
for ( int i = 0; i < argc; i++ )
{
#ifdef __APPLE__
argv_copy[i] = _Py_DecodeUTF8_surrogateescape( argv[ i ], strlen( argv[ i ] ) );
#else
argv_copy[i] = _Py_char2wchar( argv[ i ], NULL );
#endif
assert ( argv_copy[ i ] );
}
setlocale( LC_ALL, oldloc );
free( oldloc );
}
if ( initial )
{
Py_SetProgramName( argv_copy[0] );
}
else
{
PySys_SetArgv( argc, argv_copy );
}
#endif
}
typedef struct {
PyObject_HEAD
PyTypeObject *type;
PyObject *obj;
PyTypeObject *obj_type;
} superobject;
extern PyObject *const_str_plain___class__;
PyObject *BUILTIN_SUPER( PyObject *type, PyObject *object )
{
assertObject( type );
superobject *result = PyObject_GC_New( superobject, &PySuper_Type );
assert( result );
if ( object == Py_None )
{
object = NULL;
}
if (unlikely( PyType_Check( type ) == false ))
{
PyErr_Format( PyExc_TypeError, "must be type, not %s", Py_TYPE( type )->tp_name );
throw PythonException();
}
result->type = (PyTypeObject *)INCREASE_REFCOUNT( type );
if ( object )
{
result->obj = INCREASE_REFCOUNT( object );
if ( PyType_Check( object ) && PyType_IsSubtype( (PyTypeObject *)object, (PyTypeObject *)type ))
{
result->obj_type = (PyTypeObject *)INCREASE_REFCOUNT( object );
}
else if ( PyType_IsSubtype( Py_TYPE(object ), (PyTypeObject *)type) )
{
result->obj_type = (PyTypeObject *)INCREASE_REFCOUNT( (PyObject *)Py_TYPE( object ) );
}
else
{
PyObject *class_attr = PyObject_GetAttr( object, const_str_plain___class__);
if (likely( class_attr != NULL && PyType_Check( class_attr ) && (PyTypeObject *)class_attr != Py_TYPE( object ) ))
{
result->obj_type = (PyTypeObject *)class_attr;
}
else
{
if ( class_attr == NULL )
{
PyErr_Clear();
}
else
{
Py_DECREF( class_attr );
}
PyErr_Format(
PyExc_TypeError,
"super(type, obj): obj must be an instance or subtype of type"
);
throw PythonException();
}
}
}
else
{
result->obj = NULL;
result->obj_type = NULL;
}
Nuitka_GC_Track( result );
assertObject( (PyObject *)result );
assert( Py_TYPE( result ) == &PySuper_Type );
return (PyObject *)result;
}
PyObject *BUILTIN_CALLABLE( PyObject *value )
{
return PyBool_FromLong( (long)PyCallable_Check( value ) );
}
// Used by InspectPatcher too.
int Nuitka_IsInstance( PyObject *inst, PyObject *cls )
{
assertObject( inst );
assertObject( cls );
// Quick path.
if ( Py_TYPE( inst ) == (PyTypeObject *)cls )
{
return true;
}
if ( cls == (PyObject *)&PyFunction_Type && Nuitka_Function_Check( inst ) )
{
return true;
}
if ( cls == (PyObject *)&PyGen_Type && Nuitka_Generator_Check( inst ) )
{
return true;
}
if ( cls == (PyObject *)&PyMethod_Type && Nuitka_Method_Check( inst ) )
{
return true;
}
if ( cls == (PyObject *)&PyFrame_Type && Nuitka_Frame_Check( inst ) )
{
return true;
}
if ( PyTuple_Check( cls ) )
{
for ( Py_ssize_t i = 0, size = PyTuple_GET_SIZE( cls ); i < size; i++ )
{
PyObject *element = PyTuple_GET_ITEM( cls, i );
if (unlikely( Py_EnterRecursiveCall( (char *)" in __instancecheck__" ) ))
{
return -1;
}
int res = Nuitka_IsInstance( inst, element );
Py_LeaveRecursiveCall();
if ( res != 0 )
{
return res;
}
}
return 0;
}
else
{
return PyObject_IsInstance( inst, cls );
}
}
bool BUILTIN_ISINSTANCE_BOOL( PyObject *inst, PyObject *cls )
{
int res = Nuitka_IsInstance( inst, cls );
if (unlikely( res < 0 ))
{
throw PythonException();
}
return res != 0;
}
PyObject *BUILTIN_GETATTR( PyObject *object, PyObject *attribute, PyObject *default_value )
{
#if PYTHON_VERSION < 300
if ( PyUnicode_Check( attribute ) )
{
attribute = _PyUnicode_AsDefaultEncodedString( attribute, NULL );
if (unlikely( attribute == NULL ))
{
throw PythonException();
}
}
if (unlikely( !PyString_Check( attribute ) ))
{
PyErr_Format( PyExc_TypeError, "getattr(): attribute name must be string" );
throw PythonException();
}
#else
if (!PyUnicode_Check( attribute ))
{
PyErr_Format( PyExc_TypeError, "getattr(): attribute name must be string" );
throw PythonException();
}
#endif
PyObject *result = PyObject_GetAttr( object, attribute );
if ( result == NULL )
{
if ( default_value != NULL && PyErr_ExceptionMatches( PyExc_AttributeError ))
{
PyErr_Clear();
return INCREASE_REFCOUNT( default_value );
}
else
{
throw PythonException();
}
}
else
{
return result;
}
}
void BUILTIN_SETATTR( PyObject *object, PyObject *attribute, PyObject *value )
{
int res = PyObject_SetAttr( object, attribute, value );
if (unlikely( res != 0 ))
{
throw PythonException();
}
}
PyDictObject *dict_builtin = NULL;
PyModuleObject *module_builtin = NULL;
#define ASSIGN_BUILTIN( name ) _python_original_builtin_value_##name = LOOKUP_BUILTIN( const_str_plain_##name );
static PyTypeObject Nuitka_BuiltinModule_Type =
{
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"compiled_module", // tp_name
sizeof(PyModuleObject), // tp_size
};
extern PyObject *const_str_plain_open;
int Nuitka_BuiltinModule_SetAttr( PyModuleObject *module, PyObject *name, PyObject *value )
{
assertObject( (PyObject *)module );
assertObject( name );
// This is used for "del" as well.
assert( value == NULL || Py_REFCNT( value ) > 0 );
// only checks the builtins that we can refresh at this time, if we have
// many value to check maybe need create a dict first.
bool found = false;
int res = PyObject_RichCompareBool( name, const_str_plain_open, Py_EQ );
if (unlikely( res == -1 ))
{
return -1;
}
if ( res == 1 )
{
_python_builtin_open.update( value );
found = true;
}
if ( found == false )
{
res = PyObject_RichCompareBool( name, const_str_plain___import__, Py_EQ );
if (unlikely( res == -1 ))
{
return -1;
}
if ( res == 1 )
{
_python_builtin_import.update( value );
found = true;
}
}
#if PYTHON_VERSION >= 300
if ( found == false )
{
res = PyObject_RichCompareBool( name, const_str_plain_print, Py_EQ );
if (unlikely( res == -1 ))
{
return -1;
}
if ( res == 1 )
{
_python_builtin_print.update( value );
found = true;
}
}
#endif
return PyObject_GenericSetAttr( (PyObject *)module, name, value );
}
void _initBuiltinModule()
{
#if _NUITKA_MODULE
if ( module_builtin ) return;
#else
assert( module_builtin == NULL );
#endif
#if PYTHON_VERSION < 300
module_builtin = (PyModuleObject *)PyImport_ImportModule( "__builtin__" );
#else
module_builtin = (PyModuleObject *)PyImport_ImportModule( "builtins" );
#endif
assert( module_builtin );
dict_builtin = (PyDictObject *)module_builtin->md_dict;
assert( PyDict_Check( dict_builtin ) );
// init Nuitka_BuiltinModule_Type, PyType_Ready wont copy all member from
// base type, so we need copy all members from PyModule_Type manual for
// safety. PyType_Ready will change tp_flags, we need define it again. Set
// tp_setattro to Nuitka_BuiltinModule_SetAttr and we can detect value
// change. Set tp_base to PyModule_Type and PyModule_Check will pass.
Nuitka_BuiltinModule_Type.tp_dealloc = PyModule_Type.tp_dealloc;
Nuitka_BuiltinModule_Type.tp_repr = PyModule_Type.tp_repr;
Nuitka_BuiltinModule_Type.tp_setattro = (setattrofunc)Nuitka_BuiltinModule_SetAttr;
Nuitka_BuiltinModule_Type.tp_getattro = PyModule_Type.tp_getattro;
Nuitka_BuiltinModule_Type.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE;
Nuitka_BuiltinModule_Type.tp_doc = PyModule_Type.tp_doc;
Nuitka_BuiltinModule_Type.tp_traverse = PyModule_Type.tp_traverse;
Nuitka_BuiltinModule_Type.tp_members = PyModule_Type.tp_members;
Nuitka_BuiltinModule_Type.tp_base = &PyModule_Type;
Nuitka_BuiltinModule_Type.tp_dictoffset = PyModule_Type.tp_dictoffset;
Nuitka_BuiltinModule_Type.tp_init = PyModule_Type.tp_init;
Nuitka_BuiltinModule_Type.tp_alloc = PyModule_Type.tp_alloc;
Nuitka_BuiltinModule_Type.tp_new = PyModule_Type.tp_new;
Nuitka_BuiltinModule_Type.tp_free = PyModule_Type.tp_free;
int ret = PyType_Ready( &Nuitka_BuiltinModule_Type );
assert( ret == 0 );
// Replace type of builtin module to take over.
((PyObject *)module_builtin)->ob_type = &Nuitka_BuiltinModule_Type;
assert( PyModule_Check( module_builtin ) == 1 );
}
static PyObject *_fast_function_noargs( PyObject *func )
{
PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE( func );
PyObject *globals = PyFunction_GET_GLOBALS( func );
PyObject *argdefs = PyFunction_GET_DEFAULTS( func );
#if PYTHON_VERSION >= 300
PyObject *kwdefs = PyFunction_GET_KW_DEFAULTS( func );
if ( kwdefs == NULL && argdefs == NULL && co->co_argcount == 0 &&
co->co_flags == ( CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE ))
#else
if ( argdefs == NULL && co->co_argcount == 0 &&
co->co_flags == ( CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE ))
#endif
{
PyThreadState *tstate = PyThreadState_GET();
assertObject( globals );
PyFrameObject *frame = PyFrame_New( tstate, co, globals, NULL );
if (unlikely( frame == NULL ))
{
throw PythonException();
};
PyObject *result = PyEval_EvalFrameEx( frame, 0 );
// Frame release protects against recursion as it may lead to variable
// destruction.
++tstate->recursion_depth;
Py_DECREF( frame );
--tstate->recursion_depth;
if ( result == NULL )
{
throw PythonException();
}
return result;
}
PyObject **defaults = NULL;
int nd = 0;
if ( argdefs != NULL )
{
defaults = &PyTuple_GET_ITEM( argdefs, 0 );
nd = int( Py_SIZE( argdefs ) );
}
PyObject *result = PyEval_EvalCodeEx(
#if PYTHON_VERSION >= 300
(PyObject *)co,
#else
co, // code object
#endif
globals, // globals
NULL, // no locals
NULL, // args
0, // argcount
NULL, // kwds
0, // kwcount
defaults, // defaults
nd, // defcount
#if PYTHON_VERSION >= 300
kwdefs,
#endif
PyFunction_GET_CLOSURE( func )
);
if ( result == 0 )
{
throw PythonException();
}
return result;
}
PyObject *CALL_FUNCTION_NO_ARGS( PyObject *called )
{
assertObject( called );
if ( Nuitka_Function_Check( called ) )
{
if (unlikely( Py_EnterRecursiveCall( (char *)" while calling a Python object" ) ))
{
throw PythonException();
}
Nuitka_FunctionObject *function = (Nuitka_FunctionObject *)called;
PyObject *result;
if ( function->m_direct_arg_parser )
{
result = function->m_direct_arg_parser(
function,
NULL,
0
);
}
else
{
result = function->m_code(
function,
NULL,
0,
NULL
);
}
Py_LeaveRecursiveCall();
if ( result == NULL )
{
throw PythonException();
}
return result;
}
else if ( Nuitka_Method_Check( called ) )
{
Nuitka_MethodObject *method = (Nuitka_MethodObject *)called;
// Unbound method without arguments, let the error path be slow.
if ( method->m_object != NULL )
{
if (unlikely( Py_EnterRecursiveCall( (char *)" while calling a Python object" ) ))
{
throw PythonException();
}
PyObject *args[1] = {
method->m_object
};
PyObject *result;
if ( method->m_function->m_direct_arg_parser )
{
result = method->m_function->m_direct_arg_parser(
method->m_function,
args,
1
);
}
else
{
result = method->m_function->m_code(
method->m_function,
args,
1,
NULL
);
}
Py_LeaveRecursiveCall();
if ( result == NULL )
{
throw PythonException();
}
return result;
}
}
else if ( PyFunction_Check( called ) )
{
return _fast_function_noargs( called );
}
return CALL_FUNCTION(
called,
const_tuple_empty,
NULL
);
}
#if defined(_NUITKA_STANDALONE) || _NUITKA_FROZEN > 0
#include
#if defined(_WIN32)
#include
#elif defined(__APPLE__)
#include
#else
#include
#endif
#if defined(_WIN32) && !defined(PATH_MAX)
#define PATH_MAX MAXPATHLEN
#endif
#if defined( __FreeBSD__ )
#include
#endif
char *getBinaryDirectory()
{
static char binary_directory[ PATH_MAX + 1 ];
static bool init_done = false;
if ( init_done )
{
return binary_directory;
}
#if defined( _WIN32 )
GetModuleFileName( NULL, binary_directory, PATH_MAX + 1 );
PathRemoveFileSpec( binary_directory );
#elif defined(__APPLE__)
uint32_t bufsize = PATH_MAX + 1;
int res =_NSGetExecutablePath( binary_directory, &bufsize );
if (unlikely( res != 0 ))
{
abort();
}
#elif defined( __FreeBSD__ )
int mib[4];
mib[0] = CTL_KERN;
mib[1] = KERN_PROC;
mib[2] = KERN_PROC_PATHNAME;
mib[3] = -1;
size_t cb = sizeof(binary_directory);
sysctl(mib, 4, binary_directory, &cb, NULL, 0);
#else
// Readlink does not terminate result.
memset( binary_directory, 0, PATH_MAX + 1 );
ssize_t res = readlink( "/proc/self/exe", binary_directory, PATH_MAX + 1 );
if (unlikely( res == -1 ))
{
abort();
}
strcpy( binary_directory, dirname( binary_directory ) );
#endif
init_done = true;
return binary_directory;
}
#if _NUITKA_FROZEN > 0
extern struct _frozen Embedded_FrozenModules[];
#endif
#if _NUITKA_STANDALONE
extern PyObject *const_str_plain___file__;
void setEarlyFrozenModulesFileAttribute( void )
{
Py_ssize_t ppos = 0;
PyObject *key, *value;
char buffer[4096];
strcpy(buffer,getBinaryDirectory());
char *w = buffer + strlen(buffer);
*w++ = SEP;
strcpy(w,"not_present.py");
#if PYTHON_VERSION >= 300
PyObject *file_value = PyUnicode_FromString(buffer);
#else
PyObject *file_value = PyString_FromString(buffer);
#endif
while( PyDict_Next( PyImport_GetModuleDict(), &ppos, &key, &value ) )
{
if ( key != NULL && value != NULL && PyModule_Check( value ) )
{
if ( !PyObject_HasAttr( value, const_str_plain___file__ ) )
{
PyObject_SetAttr( value, const_str_plain___file__, file_value );
}
}
}
assert(!ERROR_OCCURED());
}
#endif
void prepareStandaloneEnvironment()
{
// Tell the CPython library to use our precompiled modules as frozen
// modules. This for those modules/packages like "encoding" that will be
// loaded during "Py_Initialize" already, for the others they may be
// compiled.
#if _NUITKA_FROZEN > 0
// The CPython library has some pre-existing frozen modules, we only append
// to that.
_frozen *search = PyImport_FrozenModules;
while( search->name )
{
search++;
}
int pre_existing_count = int( search - PyImport_FrozenModules );
// Allocate new memory and merge the tables.
_frozen *merged = new _frozen[ _NUITKA_FROZEN + pre_existing_count + 1 ];
memcpy(
merged,
PyImport_FrozenModules,
pre_existing_count * sizeof( struct _frozen )
);
memcpy(
merged + pre_existing_count,
Embedded_FrozenModules,
( _NUITKA_FROZEN + 1 ) * sizeof( struct _frozen )
);
PyImport_FrozenModules = merged;
#endif
#ifdef _NUITKA_STANDALONE
// Setup environment variables to tell CPython that we would like it to use
// the provided binary directory as the place to look for DLLs.
char *binary_directory = getBinaryDirectory();
#if defined( _WIN32 ) && defined( _MSC_VER )
SetDllDirectory( getBinaryDirectory() );
#endif
// get orignal value
char *orignal_home = getenv( "PYTHONHOME" );
char *orignal_path = getenv( "PYTHONPATH" );
size_t orignal_home_size = ( orignal_home ) ? strlen( orignal_home ) : 0;
size_t orignal_path_size = ( orignal_path ) ? strlen( orignal_path ) : 0;
// get insert value
size_t insert_size = strlen( binary_directory ) * 2 + 50;
char *insert_path = (char *) malloc( insert_size );
#if defined( _WIN32 )
char const env_string[] = "%s;";
#else
char const env_string[] = "%s:";
#endif
memset( insert_path, 0, insert_size );
snprintf( insert_path, insert_size, env_string, binary_directory );
// set environment
size_t python_home_size = orignal_home_size + insert_size;
size_t python_path_size = orignal_path_size + insert_size;
char *python_home = (char *) malloc( python_home_size );
char *python_path = (char *) malloc( python_path_size );
memset( python_home, 0, python_home_size );
memset( python_path, 0, python_path_size );
snprintf( python_home, python_home_size, "%s%s",
insert_path, orignal_home ? orignal_home : "" );
snprintf( python_path, python_path_size, "%s%s",
insert_path, orignal_path ? orignal_path : "" );
if ( !( orignal_home && strstr( orignal_home, insert_path ) ) )
{
#if defined( _WIN32 )
SetEnvironmentVariable( "PYTHONHOME", python_home );
#else
setenv( "PYTHONHOME", python_home, 1 );
#endif
}
if ( !( orignal_path && strstr( orignal_path, insert_path ) ) )
{
#if defined( _WIN32 )
SetEnvironmentVariable( "PYTHONPATH", python_path );
#else
setenv( "PYTHONPATH", python_path, 1 );
#endif
}
// clean up
free( insert_path );
#endif
}
#endif
#ifdef _NUITKA_EXE
#define DEFINE_BUILTIN( name ) extern PyObject *const_str_plain_##name; PyObject *_python_original_builtin_value_##name = NULL;
DEFINE_BUILTIN( type )
DEFINE_BUILTIN( len )
DEFINE_BUILTIN( range )
DEFINE_BUILTIN( repr )
DEFINE_BUILTIN( int )
DEFINE_BUILTIN( iter )
#if PYTHON_VERSION < 300
DEFINE_BUILTIN( long )
#endif
void _initBuiltinOriginalValues()
{
ASSIGN_BUILTIN( type );
ASSIGN_BUILTIN( len );
ASSIGN_BUILTIN( range );
ASSIGN_BUILTIN( repr );
ASSIGN_BUILTIN( int );
ASSIGN_BUILTIN( iter );
#if PYTHON_VERSION < 300
ASSIGN_BUILTIN( long );
#endif
assertObject( _python_original_builtin_value_range );
}
#endif
// Used for threading.
#if PYTHON_VERSION >= 300
volatile int _Py_Ticker = _Py_CheckInterval;
#endif
// Reverse operation mapping.
static int const swapped_op[] =
{
Py_GT, Py_GE, Py_EQ, Py_NE, Py_LT, Py_LE
};
#if PYTHON_VERSION < 300
extern PyObject *const_str_plain___cmp__;
cmpfunc default_tp_compare;
void initSlotCompare()
{
// Create a class with "__cmp__" attribute, to get a hand at the default
// implementation of tp_compare. It's not part of the API and with shared
// libraries it's not accessible. The name does not matter, nor does the
// actual value used for "__cmp__".
PyObject *c = PyObject_CallFunctionObjArgs(
(PyObject *)&PyType_Type,
const_str_plain___cmp__,
PyObjectTemporary( MAKE_TUPLE1( (PyObject *)&PyInt_Type ) ).asObject0(),
PyObjectTemporary( MAKE_DICT1( Py_True, const_str_plain___cmp__ ) ).asObject0(),
NULL
);
PyObject *r = PyObject_CallFunctionObjArgs(
c,
NULL
);
Py_DECREF( c );
assertObject( r );
assert( Py_TYPE( r )->tp_compare );
default_tp_compare = Py_TYPE( r )->tp_compare;
Py_DECREF( r );
}
#define RICHCOMPARE(t) (PyType_HasFeature((t), Py_TPFLAGS_HAVE_RICHCOMPARE) ? (t)->tp_richcompare : NULL)
static inline int adjust_tp_compare( int c )
{
if ( PyErr_Occurred() )
{
return -2;
}
else if (c < -1 || c > 1)
{
return c < -1 ? -1 : 1;
}
else
{
return c;
}
}
static inline int coerce_objects( PyObject **pa, PyObject **pb )
{
PyObject *a = *pa;
PyObject *b = *pb;
// Shortcut only for old-style types
if ( a->ob_type == b->ob_type && !PyType_HasFeature( a->ob_type, Py_TPFLAGS_CHECKTYPES ))
{
Py_INCREF( a );
Py_INCREF( b );
return 0;
}
if ( a->ob_type->tp_as_number && a->ob_type->tp_as_number->nb_coerce )
{
int res = (*a->ob_type->tp_as_number->nb_coerce)( pa, pb );
if ( res <= 0 )
{
return res;
}
}
if ( b->ob_type->tp_as_number && b->ob_type->tp_as_number->nb_coerce )
{
int res = (*b->ob_type->tp_as_number->nb_coerce)( pb, pa );
if ( res <= 0 )
{
return res;
}
}
return 1;
}
static int try_3way_compare( PyObject *a, PyObject *b )
{
cmpfunc f1 = a->ob_type->tp_compare;
cmpfunc f2 = b->ob_type->tp_compare;
int c;
// Same compares, just use it.
if ( f1 != NULL && f1 == f2 )
{
c = (*f1)( a, b );
return adjust_tp_compare( c );
}
// If one slot is _PyObject_SlotCompare (which we got our hands on under a
// different name in case it's a shared library), prefer it.
if ( f1 == default_tp_compare || f2 == default_tp_compare )
{
return default_tp_compare( a, b );
}
// Try coercion.
c = coerce_objects( &a, &b );
if (c < 0)
{
return -2;
}
if (c > 0)
{
return 2;
}
f1 = a->ob_type->tp_compare;
if ( f1 != NULL && f1 == b->ob_type->tp_compare )
{
c = (*f1)( a, b );
Py_DECREF( a );
Py_DECREF( b );
return adjust_tp_compare(c);
}
// No comparison defined.
Py_DECREF( a );
Py_DECREF( b );
return 2;
}
PyObject *MY_RICHCOMPARE( PyObject *a, PyObject *b, int op )
{
// TODO: Type a-ware rich comparison would be really nice, but this is what
// CPython does, and should be even in "richcomparisons.hpp" as the first
// thing, so it's even cheaper.
if ( PyInt_CheckExact( a ) && PyInt_CheckExact( b ))
{
long aa, bb;
#ifdef __NUITKA_NO_ASSERT__
bool res;
#else
bool res = false;
#endif
aa = PyInt_AS_LONG( a );
bb = PyInt_AS_LONG( b );
switch( op )
{
case Py_LT: res = aa < bb; break;
case Py_LE: res = aa <= bb; break;
case Py_EQ: res = aa == bb; break;
case Py_NE: res = aa != bb; break;
case Py_GT: res = aa > bb; break;
case Py_GE: res = aa >= bb; break;
default: assert( false );
}
return INCREASE_REFCOUNT( BOOL_FROM( res ) );
}
// TODO: Get hint from recursion control if that's needed.
if (unlikely( Py_EnterRecursiveCall((char *)" in cmp") ))
{
return NULL;
}
PyObject *result;
// If the types are equal, we may get away immediately.
if ( a->ob_type == b->ob_type && !PyInstance_Check( a ) )
{
richcmpfunc frich = RICHCOMPARE( a->ob_type );
if ( frich != NULL )
{
result = (*frich)( a, b, op );
if (result != Py_NotImplemented)
{
Py_LeaveRecursiveCall();
return result;
}
Py_DECREF( result );
}
// No rich comparison, but maybe compare works.
cmpfunc fcmp = a->ob_type->tp_compare;
if ( fcmp != NULL )
{
int c = (*fcmp)( a, b );
c = adjust_tp_compare( c );
Py_LeaveRecursiveCall();
if ( c == -2 )
{
return NULL;
}
switch( op )
{
case Py_LT: c = c < 0; break;
case Py_LE: c = c <= 0; break;
case Py_EQ: c = c == 0; break;
case Py_NE: c = c != 0; break;
case Py_GT: c = c > 0; break;
case Py_GE: c = c >= 0; break;
}
return INCREASE_REFCOUNT( BOOL_FROM( c != 0 ) );
}
}
// Fast path was not successful or not taken
richcmpfunc f;
if ( a->ob_type != b->ob_type && PyType_IsSubtype( b->ob_type, a->ob_type ) )
{
f = RICHCOMPARE( b->ob_type );
if ( f != NULL)
{
result = (*f)( b, a, swapped_op[ op ] );
if ( result != Py_NotImplemented )
{
Py_LeaveRecursiveCall();
return result;
}
Py_DECREF( result );
}
}
f = RICHCOMPARE( a->ob_type );
if ( f != NULL )
{
result = (*f)( a, b, op );
if ( result != Py_NotImplemented )
{
Py_LeaveRecursiveCall();
return result;
}
Py_DECREF( result );
}
f = RICHCOMPARE( b->ob_type );
if ( f != NULL )
{
result = (*f)( b, a, swapped_op[ op ] );
if ( result != Py_NotImplemented )
{
Py_LeaveRecursiveCall();
return result;
}
Py_DECREF( result );
}
int c;
if ( PyInstance_Check( a ) )
{
c = (*a->ob_type->tp_compare)( a, b );
}
else if ( PyInstance_Check( b ) )
{
c = (*b->ob_type->tp_compare)( a, b );
}
else
{
c = try_3way_compare( a, b );
}
if ( c >= 2 )
{
if ( a->ob_type == b->ob_type )
{
Py_uintptr_t aa = (Py_uintptr_t)a;
Py_uintptr_t bb = (Py_uintptr_t)b;
c = ( aa < bb ) ? -1 : ( aa > bb ) ? 1 : 0;
}
else if ( a == Py_None )
{
// None is smaller than everything else
c = -1;
}
else if ( b == Py_None )
{
// None is smaller than everything else
c = 1;
}
else if ( PyNumber_Check( a ) )
{
// different type: compare type names but numbers are smaller than
// others.
if ( PyNumber_Check( b ) )
{
// Both numbers, need to make a decision based on types.
Py_uintptr_t aa = (Py_uintptr_t)Py_TYPE( a );
Py_uintptr_t bb = (Py_uintptr_t)Py_TYPE( b );
c = ( aa < bb ) ? -1 : ( aa > bb ) ? 1 : 0;
}
else
{
c = -1;
}
}
else if ( PyNumber_Check( b ) )
{
c = 1;
}
else
{
int s = strcmp( a->ob_type->tp_name, b->ob_type->tp_name );
if ( s < 0 )
{
c = -1;
}
else if ( s > 0 )
{
c = 1;
}
else
{
// Same type name need to make a decision based on types.
Py_uintptr_t aa = (Py_uintptr_t)Py_TYPE( a );
Py_uintptr_t bb = (Py_uintptr_t)Py_TYPE( b );
c = ( aa < bb ) ? -1 : ( aa > bb ) ? 1 : 0;
}
}
}
Py_LeaveRecursiveCall();
if (unlikely( c <= -2 ))
{
return NULL;
}
switch( op )
{
case Py_LT: c = c < 0; break;
case Py_LE: c = c <= 0; break;
case Py_EQ: c = c == 0; break;
case Py_NE: c = c != 0; break;
case Py_GT: c = c > 0; break;
case Py_GE: c = c >= 0; break;
}
return INCREASE_REFCOUNT( BOOL_FROM( c != 0 ) );
}
#else
// Table for operation names as strings.
static char const *op_strings[] =
{
"<", "<=", "==", "!=", ">", ">="
};
PyObject *MY_RICHCOMPARE( PyObject *a, PyObject *b, int op )
{
if (unlikely( Py_EnterRecursiveCall( (char *)" in comparison" ) ))
{
return NULL;
}
bool checked_reverse_op = false;
PyObject *result = NULL;
richcmpfunc f;
if ( a->ob_type != b->ob_type && PyType_IsSubtype( b->ob_type, a->ob_type ) )
{
f = b->ob_type->tp_richcompare;
if ( f != NULL )
{
checked_reverse_op = true;
result = (*f)( b, a, swapped_op[ op ] );
if (unlikely( result == NULL ))
{
Py_LeaveRecursiveCall();
return NULL;
}
if ( result == Py_NotImplemented )
{
Py_DECREF( result );
result = NULL;
}
}
}
if ( result == NULL )
{
f = a->ob_type->tp_richcompare;
if ( f != NULL )
{
result = (*f)( a, b, op );
if (unlikely( result == NULL ))
{
Py_LeaveRecursiveCall();
return NULL;
}
if ( result == Py_NotImplemented )
{
Py_DECREF( result );
result = NULL;
}
}
}
if ( result == NULL && checked_reverse_op == false )
{
f = b->ob_type->tp_richcompare;
if ( f != NULL )
{
result = (*f)( b, a, swapped_op[ op ] );
if (unlikely( result == NULL ))
{
Py_LeaveRecursiveCall();
return NULL;
}
if ( result == Py_NotImplemented )
{
Py_DECREF( result );
result = NULL;
}
}
}
Py_LeaveRecursiveCall();
if ( result != NULL )
{
return result;
}
// If it is not implemented, do identify checks as "==" and "!=" and
// otherwise give an error
if ( op == Py_EQ )
{
return INCREASE_REFCOUNT( BOOL_FROM( a == b ) );
}
else if ( op == Py_NE )
{
return INCREASE_REFCOUNT( BOOL_FROM( a != b ) );
}
else
{
PyErr_Format(
PyExc_TypeError,
"unorderable types: %s() %s %s()",
a->ob_type->tp_name,
op_strings[ op ],
b->ob_type->tp_name
);
return NULL;
}
}
#endif
PyObject *DEEP_COPY( PyObject *value )
{
if ( PyDict_Check( value ) )
{
// For Python3.3, this can be done much faster in the same way as it is
// done in parameter parsing.
#if PYTHON_VERSION < 330
PyObject *result = _PyDict_NewPresized( ((PyDictObject *)value)->ma_used );
for ( Py_ssize_t i = 0; i <= ((PyDictObject *)value)->ma_mask; i++ )
{
PyDictEntry *entry = &((PyDictObject *)value)->ma_table[ i ];
if ( entry->me_value != NULL )
{
int res = PyDict_SetItem(
result,
entry->me_key,
PyObjectTemporary( DEEP_COPY( entry->me_value ) ).asObject0()
);
if (unlikely( res == -1 ))
{
throw PythonException();
}
}
}
return result;
#else
if ( _PyDict_HasSplitTable( (PyDictObject *)value) )
{
PyDictObject *mp = (PyDictObject *)value;
PyObject **newvalues = PyMem_NEW( PyObject *, mp->ma_keys->dk_size );
assert (newvalues != NULL);
PyDictObject *result = PyObject_GC_New( PyDictObject, &PyDict_Type );
assert( result != NULL );
result->ma_values = newvalues;
result->ma_keys = mp->ma_keys;
result->ma_used = mp->ma_used;
mp->ma_keys->dk_refcnt += 1;
Nuitka_GC_Track( result );
Py_ssize_t size = mp->ma_keys->dk_size;
for ( Py_ssize_t i = 0; i < size; i++ )
{
if ( mp->ma_values[ i ] )
{
result->ma_values[ i ] = DEEP_COPY( mp->ma_values[ i ] );
}
else
{
result->ma_values[ i ] = NULL;
}
}
return (PyObject *)result;
}
else
{
PyObject *result = _PyDict_NewPresized( ((PyDictObject *)value)->ma_used );
PyDictObject *mp = (PyDictObject *)value;
Py_ssize_t size = mp->ma_keys->dk_size;
for ( Py_ssize_t i = 0; i < size; i++ )
{
PyDictKeyEntry *entry = &mp->ma_keys->dk_entries[i];
PyObject *value;
if ( mp->ma_values )
{
value = mp->ma_values[ i ];
}
else
{
value = entry->me_value;
}
if ( value != NULL )
{
PyDict_SetItem(
result,
entry->me_key,
PyObjectTemporary( DEEP_COPY( value ) ).asObject0()
);
}
}
return result;
}
#endif
}
else if ( PyTuple_Check( value ) )
{
Py_ssize_t n = PyTuple_Size( value );
PyObject *result = PyTuple_New( n );
for( Py_ssize_t i = 0; i < n; i++ )
{
PyTuple_SET_ITEM( result, i, DEEP_COPY( PyTuple_GET_ITEM( value, i ) ) );
}
return result;
}
else if ( PyList_Check( value ) )
{
Py_ssize_t n = PyList_GET_SIZE( value );
PyObject *result = PyList_New( n );
for( Py_ssize_t i = 0; i < n; i++ )
{
PyList_SET_ITEM( result, i, DEEP_COPY( PyList_GET_ITEM( value, i ) ) );
}
return result;
}
else if ( PySet_Check( value ) )
{
// Sets cannot contain unhashable types, so they must be immutable.
return PySet_New( value );
}
else if (
#if PYTHON_VERSION < 300
PyString_Check( value ) ||
#endif
PyUnicode_Check( value ) ||
#if PYTHON_VERSION < 300
PyInt_Check( value ) ||
#endif
PyLong_Check( value ) ||
value == Py_None ||
PyBool_Check( value ) ||
PyFloat_Check( value ) ||
PyBytes_Check( value ) ||
#if PYTHON_VERSION >= 300
PyRange_Check( value ) ||
#endif
PyType_Check( value ) ||
PyComplex_Check( value )
)
{
return INCREASE_REFCOUNT( value );
}
else
{
PyErr_Format(
PyExc_TypeError,
"DEEP_COPY does not implement: %s",
value->ob_type->tp_name
);
throw PythonException();
}
}
Nuitka-0.5.0.1/nuitka/build/static_src/CompiledMethodType.cpp 0000644 0001750 0001750 00000043465 12265264105 024352 0 ustar hayen hayen 0000000 0000000 // Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
//
// Part of "Nuitka", an optimizing Python compiler that is compatible and
// integrates with CPython, but also works on its own.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
#include "nuitka/prelude.hpp"
#include "nuitka/compiled_method.hpp"
#include "structmember.h"
static PyObject *Nuitka_Method_get__doc__( Nuitka_MethodObject *method, void *closure )
{
return INCREASE_REFCOUNT( method->m_function->m_doc );
}
static PyGetSetDef Nuitka_Method_getsets[] =
{
{ (char *)"__doc__", (getter)Nuitka_Method_get__doc__, NULL, NULL },
{ NULL }
};
#define OFF( x ) offsetof( Nuitka_MethodObject, x )
static PyMemberDef Nuitka_Method_members[] =
{
{ (char *)"im_class", T_OBJECT, OFF( m_class ), READONLY | RESTRICTED, (char *)"the class associated with a method"},
{ (char *)"im_func", T_OBJECT, OFF( m_function ), READONLY | RESTRICTED, (char *)"the function (or other callable) implementing a method" },
{ (char *)"__func__", T_OBJECT, OFF( m_function ), READONLY | RESTRICTED, (char *)"the function (or other callable) implementing a method" },
{ (char *)"im_self", T_OBJECT, OFF( m_object ), READONLY | RESTRICTED, (char *)"the instance to which a method is bound; None for unbound method" },
{ (char *)"__self__", T_OBJECT, OFF( m_object ), READONLY | RESTRICTED, (char *)"the instance to which a method is bound; None for unbound method" },
{ NULL }
};
static PyObject *Nuitka_Method_reduce( Nuitka_MethodObject *method )
{
return MAKE_TUPLE2(
(PyObject *)Py_TYPE( method ),
PyObjectTemporary(
MAKE_TUPLE2(
(PyObject *)method->m_function,
method->m_object
)
).asObject0()
);
}
static PyMethodDef Nuitka_Method_methods[] =
{
{ "__reduce__", (PyCFunction)Nuitka_Method_reduce, METH_NOARGS, NULL },
{ NULL }
};
extern PyObject *const_str_plain___name__;
static char const *GET_CLASS_NAME( PyObject *klass )
{
if ( klass == NULL )
{
return "?";
}
else
{
PyObject *name = PyObject_GetAttr( klass, const_str_plain___name__ );
if (unlikely( name == NULL ))
{
PyErr_Clear();
return "?";
}
else
{
if ( !Nuitka_String_Check( name ) )
{
Py_DECREF( name );
return "?";
}
char *const result = Nuitka_String_AsString_Unchecked( name );
Py_DECREF( name );
return result;
}
}
}
static char const *GET_INSTANCE_CLASS_NAME( PyObject *instance )
{
PyObject *klass = PyObject_GetAttrString( instance, "__class__" );
// Fallback to type as this cannot fail.
if ( klass == NULL )
{
PyErr_Clear();
klass = INCREASE_REFCOUNT( (PyObject *)Py_TYPE( instance ) );
}
char const *result = GET_CLASS_NAME( klass );
Py_DECREF( klass );
return result;
}
static char const *GET_CALLABLE_DESC( PyObject *object )
{
if ( Nuitka_Function_Check( object ) || Nuitka_Generator_Check( object ) || PyMethod_Check( object ) || PyFunction_Check( object ) || PyCFunction_Check( object ) )
{
return "()";
}
#if PYTHON_VERSION < 300
else if ( PyClass_Check( object ) )
{
return " constructor";
}
else if ( PyInstance_Check( object ))
{
return " instance";
}
#endif
else
{
return " object";
}
}
static char const *GET_CALLABLE_NAME( PyObject *object )
{
if ( Nuitka_Function_Check( object ) )
{
return Nuitka_String_AsString( Nuitka_Function_GetName( object ) );
}
else if ( Nuitka_Generator_Check( object ) )
{
return Nuitka_String_AsString( Nuitka_Generator_GetName( object ) );
}
else if ( PyMethod_Check( object ) )
{
return PyEval_GetFuncName( PyMethod_GET_FUNCTION( object ) );
}
else if ( PyFunction_Check( object ) )
{
return Nuitka_String_AsString( ((PyFunctionObject*)object)->func_name );
}
#if PYTHON_VERSION < 300
else if ( PyInstance_Check( object ) )
{
return Nuitka_String_AsString( ((PyInstanceObject*)object)->in_class->cl_name );
}
else if ( PyClass_Check( object ) )
{
return Nuitka_String_AsString( ((PyClassObject*)object)->cl_name );
}
#endif
else if ( PyCFunction_Check( object ) )
{
return ((PyCFunctionObject*)object)->m_ml->ml_name;
}
else
{
return Py_TYPE( object )->tp_name;
}
}
#ifdef _MSC_VER
// Using _alloca below.
#include
#endif
static PyObject *Nuitka_Method_tp_call( Nuitka_MethodObject *method, PyObject *args, PyObject *kw )
{
int arg_count = int( PyTuple_Size( args ) );
if ( method->m_object == NULL )
{
if (unlikely( arg_count < 1 ))
{
PyErr_Format(
PyExc_TypeError,
"unbound compiled_method %s%s must be called with %s instance as first argument (got nothing instead)",
GET_CALLABLE_NAME( (PyObject *)method->m_function ),
GET_CALLABLE_DESC( (PyObject *)method->m_function ),
GET_CLASS_NAME( method->m_class )
);
return NULL;
}
else
{
PyObject *self = PyTuple_GET_ITEM( args, 0 );
assertObject( self );
int result = PyObject_IsInstance( self, method->m_class );
if (unlikely( result < 0 ))
{
return NULL;
}
else if (unlikely( result == 0 ))
{
PyErr_Format(
PyExc_TypeError,
"unbound compiled_method %s%s must be called with %s instance as first argument (got %s instance instead)",
GET_CALLABLE_NAME( (PyObject *)method->m_function ),
GET_CALLABLE_DESC( (PyObject *)method->m_function ),
GET_CLASS_NAME( method->m_class ),
GET_INSTANCE_CLASS_NAME( (PyObject *)self )
);
return NULL;
}
}
return Py_TYPE( method->m_function )->tp_call(
(PyObject *)method->m_function, args, kw
);
}
else
{
#ifdef _MSC_VER
PyObject **new_args = (PyObject **)_alloca( sizeof( PyObject * ) *( arg_count + 1 ) );
#else
PyObject *new_args[ arg_count + 1 ];
#endif
new_args[ 0 ] = method->m_object;
for ( int i = 0; i < arg_count; i++ )
{
new_args[ i + 1 ] = PyTuple_GET_ITEM( args, i );
}
if ( kw || method->m_function->m_direct_arg_parser == NULL )
{
return method->m_function->m_code(
method->m_function,
new_args,
arg_count + 1,
kw
);
}
else
{
return method->m_function->m_direct_arg_parser(
method->m_function,
new_args,
arg_count + 1
);
}
}
}
static PyObject *Nuitka_Method_tp_descr_get( Nuitka_MethodObject *method, PyObject *object, PyObject *klass )
{
// Don't rebind already bound methods.
if ( method->m_object != NULL )
{
return INCREASE_REFCOUNT( (PyObject *)method );
}
if ( method->m_class != NULL && klass != NULL )
{
// Quick subclass test, bound methods remain the same if the class is a sub class
int result = PyObject_IsSubclass( klass, method->m_class );
if (unlikely( result < 0 ))
{
return NULL;
}
else if ( result == 0 )
{
return INCREASE_REFCOUNT( (PyObject *)method );
}
}
return Nuitka_Method_New( method->m_function, object, klass );
}
static PyObject *Nuitka_Method_tp_getattro( Nuitka_MethodObject *method, PyObject *name )
{
PyObject *descr = _PyType_Lookup( &Nuitka_Method_Type, name );
if ( descr != NULL )
{
if (
#if PYTHON_VERSION < 300
PyType_HasFeature( Py_TYPE( descr ), Py_TPFLAGS_HAVE_CLASS ) &&
#endif
( Py_TYPE( descr )->tp_descr_get != NULL )
)
{
return Py_TYPE( descr )->tp_descr_get(
descr,
(PyObject *)method,
(PyObject *)Py_TYPE( method )
);
}
else
{
return INCREASE_REFCOUNT( descr );
}
}
return PyObject_GetAttr( (PyObject *)method->m_function, name );
}
static long Nuitka_Method_tp_traverse( Nuitka_MethodObject *method, visitproc visit, void *arg )
{
Py_VISIT( method->m_function );
Py_VISIT( method->m_object );
Py_VISIT( method->m_class );
return 0;
}
// tp_repr slot, decide how a function shall be output
static PyObject *Nuitka_Method_tp_repr( Nuitka_MethodObject *method )
{
if ( method->m_object == NULL )
{
#if PYTHON_VERSION < 300
return PyString_FromFormat(
"",
GET_CLASS_NAME( method->m_class ),
Nuitka_String_AsString( method->m_function->m_name )
);
#else
return PyUnicode_FromFormat(
"",
Nuitka_String_AsString( method->m_function->m_name ),
method->m_function
);
#endif
}
else
{
// Note: CPython uses repr ob the object, although a comment despises it, we
// do it for compatibility.
PyObject *object_repr = PyObject_Repr( method->m_object );
if ( object_repr == NULL )
{
return NULL;
}
#if PYTHON_VERSION < 300
else if ( !PyString_Check( object_repr ) )
{
Py_DECREF( object_repr );
return NULL;
}
#else
else if ( !PyUnicode_Check( object_repr ) )
{
Py_DECREF( object_repr );
return NULL;
}
#endif
#if PYTHON_VERSION < 300
PyObject *result = PyString_FromFormat(
#else
PyObject *result = PyUnicode_FromFormat(
#endif
"",
GET_CLASS_NAME( method->m_class ),
Nuitka_String_AsString( method->m_function->m_name ),
Nuitka_String_AsString_Unchecked( object_repr )
);
Py_DECREF( object_repr );
return result;
}
}
#if PYTHON_VERSION < 300
static int Nuitka_Method_tp_compare( Nuitka_MethodObject *a, Nuitka_MethodObject *b )
{
if ( a->m_function->m_counter < b->m_function->m_counter )
{
return -1;
}
else if ( a->m_function->m_counter > b->m_function->m_counter )
{
return 1;
}
else if ( a->m_object == b->m_object )
{
return 0;
}
else if ( a->m_object == NULL )
{
return -1;
}
else if ( b->m_object == NULL )
{
return 1;
}
else
{
return PyObject_Compare( a->m_object, b->m_object );
}
}
#endif
static PyObject *Nuitka_Method_tp_richcompare( Nuitka_MethodObject *a, Nuitka_MethodObject *b, int op )
{
if ( op != Py_EQ && op != Py_NE )
{
return INCREASE_REFCOUNT( Py_NotImplemented );
}
if ( Nuitka_Method_Check( (PyObject *)a ) == false || Nuitka_Method_Check( (PyObject *)b ) == false )
{
return INCREASE_REFCOUNT( Py_NotImplemented );
}
bool result = a->m_function->m_counter == b->m_function->m_counter;
// If the underlying function objects are the same, check the objects, which
// may be NULL in case of unbound methods, which would be the same again.
if ( result )
{
if ( a->m_object == NULL )
{
result = b->m_object == NULL;
}
else if ( b->m_object == NULL )
{
result = 0;
}
else
{
int res = PyObject_RichCompareBool(
a->m_object,
b->m_object,
Py_EQ
);
result = res != 0;
}
}
if ( op == Py_EQ )
{
return INCREASE_REFCOUNT( BOOL_FROM( result ) );
}
else
{
return INCREASE_REFCOUNT( BOOL_FROM( !result ) );
}
}
static long Nuitka_Method_tp_hash( Nuitka_MethodObject *method )
{
// Just give the hash of the method function, that ought to be good enough.
return method->m_function->m_counter;
}
// Cache for method object, try to avoid malloc overhead.
static Nuitka_MethodObject *method_cache_head = NULL;
static int method_cache_size = 0;
static const int max_method_cache_size = 4096;
static void Nuitka_Method_tp_dealloc( Nuitka_MethodObject *method )
{
Nuitka_GC_UnTrack( method );
if ( method->m_weakrefs != NULL )
{
PyObject_ClearWeakRefs( (PyObject *)method );
}
Py_XDECREF( method->m_object );
Py_XDECREF( method->m_class );
Py_DECREF( (PyObject *)method->m_function );
if (likely( method_cache_size < max_method_cache_size ))
{
method->m_object = (PyObject *)method_cache_head;
method_cache_head = method;
method_cache_size += 1;
}
else {
PyObject_GC_Del( method );
}
}
static PyObject *Nuitka_Method_tp_new( PyTypeObject* type, PyObject* args, PyObject *kw )
{
PyObject *func;
PyObject *self;
PyObject *klass = NULL;
if ( !_PyArg_NoKeywords( "instancemethod", kw ) )
{
return NULL;
}
else if ( !PyArg_UnpackTuple( args, "compiled_method", 2, 3, &func, &self, &klass ) )
{
return NULL;
}
else if ( !PyCallable_Check( func ) )
{
PyErr_Format( PyExc_TypeError, "first argument must be callable" );
return NULL;
}
else
{
if ( self == Py_None )
{
self = NULL;
}
if ( self == NULL && klass == NULL )
{
PyErr_Format( PyExc_TypeError, "unbound methods must have non-NULL im_class" );
return NULL;
}
}
assert( Nuitka_Function_Check( func ) );
return Nuitka_Method_New( (Nuitka_FunctionObject *)func, self, klass );
}
static const long tp_flags =
Py_TPFLAGS_DEFAULT |
#if PYTHON_VERSION < 300
Py_TPFLAGS_HAVE_WEAKREFS |
#endif
Py_TPFLAGS_HAVE_GC;
PyTypeObject Nuitka_Method_Type =
{
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"compiled_method",
sizeof(Nuitka_MethodObject),
0,
(destructor)Nuitka_Method_tp_dealloc, // tp_dealloc
0, // tp_print
0, // tp_getattr
0, // tp_setattr
#if PYTHON_VERSION < 300
(cmpfunc)Nuitka_Method_tp_compare, // tp_compare
#else
0,
#endif
(reprfunc)Nuitka_Method_tp_repr, // tp_repr
0, // tp_as_number
0, // tp_as_sequence
0, // tp_as_mapping
(hashfunc)Nuitka_Method_tp_hash, // tp_hash
(ternaryfunc)Nuitka_Method_tp_call, // tp_call
0, // tp_str
(getattrofunc)Nuitka_Method_tp_getattro, // tp_getattro
PyObject_GenericSetAttr, // tp_setattro
0, // tp_as_buffer
tp_flags, // tp_flags
0, // tp_doc
(traverseproc)Nuitka_Method_tp_traverse, // tp_traverse
0, // tp_clear
(richcmpfunc)Nuitka_Method_tp_richcompare, // tp_richcompare
offsetof( Nuitka_MethodObject, m_weakrefs ), // tp_weaklistoffset
0, // tp_iter
0, // tp_iternext
Nuitka_Method_methods, // tp_methods
Nuitka_Method_members, // tp_members
Nuitka_Method_getsets, // tp_getset
0, // tp_base
0, // tp_dict
(descrgetfunc)Nuitka_Method_tp_descr_get, // tp_descr_get
0, // tp_descr_set
0, // tp_dictoffset
0, // tp_init
0, // tp_alloc
Nuitka_Method_tp_new // tp_new
};
PyObject *Nuitka_Method_New( Nuitka_FunctionObject *function, PyObject *object, PyObject *klass )
{
Nuitka_MethodObject *result = method_cache_head;
if ( result != NULL )
{
method_cache_head = (Nuitka_MethodObject *)method_cache_head->m_object;
method_cache_size -= 1;
PyObject_INIT( result, &Nuitka_Method_Type );
}
else
{
result = PyObject_GC_New( Nuitka_MethodObject, &Nuitka_Method_Type );
}
if (unlikely( result == NULL ))
{
PyErr_Format(
PyExc_RuntimeError,
"cannot create method %s",
Nuitka_String_AsString( function->m_name )
);
throw PythonException();
}
result->m_function = (Nuitka_FunctionObject * )INCREASE_REFCOUNT( (PyObject *)function );
result->m_object = INCREASE_REFCOUNT_X( object );
result->m_class = INCREASE_REFCOUNT_X( klass );
result->m_weakrefs = NULL;
Nuitka_GC_Track( result );
return (PyObject *)result;
}
Nuitka-0.5.0.1/nuitka/build/__init__.py 0000644 0001750 0001750 00000001501 12265264105 020043 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Dummy file to make this directory a package. """
Nuitka-0.5.0.1/nuitka/finalizations/ 0000755 0001750 0001750 00000000000 12265271051 017506 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/finalizations/Finalization.py 0000644 0001750 0001750 00000003470 12265264105 022515 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Finalizations. Last steps directly before code creation is called.
Here the final tasks are executed. Things normally volatile during optimization
can be computed here, so the code generation can be quick and doesn't have to
check it many times.
"""
from .FinalizeMarkups import FinalizeMarkups
from .FinalizeClosureTaking import FinalizeClosureTaking
from .FinalizeVariableVisibility import FinalizeVariableVisibility
# Bug of pylint, it's there but it reports it wrongly, pylint: disable=E0611
from nuitka.tree import Operations
def prepareCodeGeneration(tree):
visitor = FinalizeMarkups()
Operations.visitTree( tree, visitor )
for function in tree.getUsedFunctions():
Operations.visitTree( function, visitor )
visitor = FinalizeClosureTaking()
for function in tree.getUsedFunctions():
Operations.visitFunction( function, visitor )
visitor = FinalizeVariableVisibility()
Operations.visitFunction( tree, visitor )
for function in tree.getUsedFunctions():
Operations.visitFunction( function, visitor )
Nuitka-0.5.0.1/nuitka/finalizations/FinalizeClosureTaking.py 0000644 0001750 0001750 00000004313 12265264105 024317 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Finalize the closure.
If a taker wants a variable, make sure that the closure taker in between all do
forward it for this use or else it will not be available. We do this late so it
is easier to remove closure variables and keep track of references, by not
having it spoiled with these transitive only references.
"""
from .FinalizeBase import FinalizationVisitorBase
class FinalizeClosureTaking(FinalizationVisitorBase):
def onEnterNode(self, node):
assert node.isExpressionFunctionBody(), node
# print node, node.provider
for variable in node.getClosureVariables():
referenced = variable.getReferenced()
referenced_owner = referenced.getOwner()
assert not referenced.isModuleVariable()
current = node
while current is not referenced_owner:
if current.isExpressionFunctionBody():
for current_variable in current.getClosureVariables():
if current_variable.getReferenced() is referenced:
break
else:
current.addClosureVariable( referenced )
# Detect loops in the provider relationship
assert current.getParentVariableProvider() is not current
current = current.getParentVariableProvider()
# Not found?!
assert current is not None, ( variable, referenced )
Nuitka-0.5.0.1/nuitka/finalizations/FinalizeMarkups.py 0000644 0001750 0001750 00000016036 12265264105 023174 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Finalize the markups
Set flags on functions and classes to indicate if a locals dict is really
needed.
Set a flag on loops if they really need to catch Continue and Break exceptions
or if it can be more simple code.
Set a flag on return statements and functions that require the use of
"ReturnValue" exceptions, or if it can be more simple code.
Set a flag on re-raises of exceptions if they can be simple throws or if they
are in another context.
"""
from nuitka import Options, Utils
from .FinalizeBase import FinalizationVisitorBase
from logging import warning
class FinalizeMarkups(FinalizationVisitorBase):
def onEnterNode(self, node):
# This has many different things it deals with, so there need to be a
# lot of branches and statements, pylint: disable=R0912,R0915
if node.isExpressionFunctionBody():
if node.isUnoptimized():
node.markAsLocalsDict()
if node.needsLocalsDict():
provider = node.getParentVariableProvider()
if provider.isExpressionFunctionBody():
provider.markAsLocalsDict()
if node.isStatementBreakLoop():
search = node.getParent()
# Search up to the containing loop.
while not search.isStatementLoop():
last_search = search
search = search.getParent()
if search.isStatementTryFinally() and last_search == search.getBlockTry():
search.markAsExceptionBreak()
node.markAsExceptionDriven()
if node.isExceptionDriven():
search.markAsExceptionBreak()
if node.isStatementContinueLoop():
search = node.getParent()
# Search up to the containing loop.
while not search.isStatementLoop():
last_search = search
search = search.getParent()
if search.isStatementTryFinally() and \
last_search == search.getBlockTry():
search.markAsExceptionContinue()
node.markAsExceptionDriven()
if node.isExceptionDriven():
search.markAsExceptionContinue()
if node.isExpressionYield() or node.isExpressionYieldFrom():
search = node.getParent()
while not search.isExpressionFunctionBody():
last_search = search
search = search.getParent()
if Utils.python_version >= 300 and \
search.isStatementTryFinally() and \
last_search == search.getBlockTry():
node.markAsExceptionPreserving()
break
if search.isStatementExceptHandler():
node.markAsExceptionPreserving()
break
if node.isStatementReturn() or node.isStatementGeneratorReturn():
search = node.getParent()
exception_driven = False
last_found = None
# Search up to the containing function, and check for a try/finally
# containing the "return" statement.
while not search.isExpressionFunctionBody():
last_search = search
search = search.getParent()
if search.isStatementTryFinally() and \
last_search == search.getBlockTry():
search.markAsExceptionReturnValueCatch()
exception_driven = True
if last_found is not None:
last_found.markAsExceptionReturnValueReraise()
last_found = search
if exception_driven:
search.markAsExceptionReturnValue()
node.setExceptionDriven( exception_driven )
if node.isStatementRaiseException() and node.isReraiseException():
search = node.getParent()
# Check if it's in a try/except block.
while not search.isParentVariableProvider():
if search.isStatementsSequence():
if search.getParent().isStatementExceptHandler():
node.markAsReraiseLocal()
break
if search.getParent().isStatementTryFinally() and \
Utils.python_version >= 300:
node.markAsReraiseFinally()
search = search.getParent()
search = node.getParent()
if node.isStatementDelVariable():
variable = node.getTargetVariableRef().getVariable()
while variable.isReference():
variable = variable.getReferenced()
variable.setHasDelIndicator()
if node.isStatementTryExcept():
provider = node.getParentVariableProvider()
provider.markAsTryExceptContaining()
if not node.isStatementTryExceptOptimized():
parent_frame = node.getParentStatementsFrame()
parent_frame.markAsFrameExceptionPreserving()
if node.isStatementTryFinally():
provider = node.getParentVariableProvider()
provider.markAsTryFinallyContaining()
if Utils.python_version >= 300:
parent_frame = node.getParentStatementsFrame()
parent_frame.markAsFrameExceptionPreserving()
if node.isStatementRaiseException():
provider = node.getParentVariableProvider()
provider.markAsRaiseContaining()
if node.isExpressionBuiltinImport() and \
not Options.getShallFollowExtra():
warning( """Unresolved '__import__' call at '%s' may require use \
of '--recurse-directory'.""" % (
node.getSourceReference().getAsString()
)
)
if node.isExpressionFunctionCreation():
if not node.getParent().isExpressionFunctionCall():
node.getFunctionRef().getFunctionBody().markAsNeedsCreation()
if node.isExpressionFunctionCall():
node.getFunction().getFunctionRef().getFunctionBody().\
markAsDirectlyCalled()
if node.isExpressionFunctionRef():
parent_module = node.getFunctionBody().getParentModule()
if node.getParentModule() is not parent_module:
node.getFunctionBody().markAsCrossModuleUsed()
Nuitka-0.5.0.1/nuitka/finalizations/__init__.py 0000644 0001750 0001750 00000001501 12265264105 021616 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Dummy file to make this directory a package. """
Nuitka-0.5.0.1/nuitka/finalizations/FinalizeBase.py 0000644 0001750 0001750 00000001746 12265264105 022426 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Base for all finalization modules
Provides a class that all finalization visitors should inherit from.
"""
from nuitka.tree import Operations
class FinalizationVisitorBase(Operations.VisitorNoopMixin):
pass
Nuitka-0.5.0.1/nuitka/finalizations/FinalizeVariableVisibility.py 0000644 0001750 0001750 00000012247 12265264105 025347 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Finalize the variable visibility.
In order to declare variables with minimum scope in the generated code, we need
to find the common ancestor statement and attach it to the variable for
declaration.
"""
from .FinalizeBase import FinalizationVisitorBase
def selectStatement(one, two, mode):
merge = []
for c1, c2 in zip( one, two ):
if c1 is not c2:
break
merge.append( c1 )
# print( "M", merge )
# If it's both in an expression, go up until it's a statement.
if merge[-1].isExpression():
while merge[-1].isExpression():
del merge[-1]
elif merge[-1].isStatementsSequence():
if len( one ) > len( merge ):
c1 = one[ len( merge ) ]
c2 = two[ len( merge ) ]
statements = merge[-1].getStatements()
if statements.index( c1 ) < statements.index( c2 ):
merge.append( c1 if mode else c2 )
else:
merge.append( c2 if mode else c1 )
assert None not in merge
return merge
class FinalizeVariableVisibility(FinalizationVisitorBase):
def onEnterNode(self, node):
collection = node.collection
assert collection is not None, node
# TODO: This could easily be expanded to all variables.
for variable in node.getTempVariables():
variable_traces = collection.getVariableTraces( variable )
uses = set()
# Will remain "None" only if no assignments are done, which is
# probably an error for at least temporary variables.
needs_free = None
for variable_trace in variable_traces:
# Safe to ignore this likely, although the merge point may be
# important, but it should come up as potential use.
if variable_trace.isMergeTrace():
continue
for use in variable_trace.getPotentialUsages():
if use.__class__.__name__.startswith("VariableMerge"):
continue
uses.add( use )
if variable_trace.isAssignTrace():
assign_node = variable_trace.getAssignNode()
uses.add( assign_node.getTargetVariableRef() )
# In preparation for code generation, we are here checking
# if it's required to hold a reference in this variable,
# because if it is not, we can create faster code.
if assign_node.getAssignSource().mayProvideReference():
needs_free = True
elif needs_free is None:
needs_free = False
for use in variable_trace.getReleases():
uses.add( use )
# For temporary variables, we can expect to know that as it should
# have been removed, if it's without assignments or if it has
# references left-over, these would be bogus and should be converted
# to some form of raise or assertion previously.
assert needs_free is not None or not uses, variable
assert variable.getNeedsFree() is None
variable.setNeedsFree( needs_free )
# Determine the first and last useing statements for a variable,
# which means for scopes that late initialisation might be used, or
# RAII style release.
first = None
last = None
for use in uses:
other = use.getParents()
if first is None:
first = other
else:
first = selectStatement( first, other, True )
if last is None:
last = other
else:
last = selectStatement( last, other, False )
if first is not None and first[-1].isStatementAssignmentVariable():
assigned_to = first[-1].getTargetVariableRef().getVariable()
if assigned_to.isReference():
assigned_to = assigned_to.getReferenced()
assert assigned_to is variable, ( variable, assigned_to )
if not variable.isShared():
variable.markAsNeedsLateDeclaration()
assert last is not None
if last[-1].isStatementDelVariable():
variable.markAsDeleteScope( last[-1] )
Nuitka-0.5.0.1/nuitka/Variables.py 0000644 0001750 0001750 00000044512 12265270513 017126 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Variables link the storage and use of a Python variable together.
Different kinds of variables represent different scopes and owners types,
and their links between each other, i.e. references as in closure or
module variable references.
"""
from nuitka import Utils
class Variable:
def __init__(self, owner, variable_name):
assert type( variable_name ) is str, variable_name
assert type( owner ) not in ( tuple, list ), owner
self.variable_name = variable_name
self.owner = owner
self.references = []
self.read_only_indicator = None
self.has_del = False
self.version_number = 0
def getName(self):
return self.variable_name
def getOwner(self):
return self.owner
# TODO: Seems obsolete now
def getRealOwner(self):
return self.owner
def addReference(self, reference):
self.references.append( reference )
def getReferences(self):
return self.references
def getReferenced(self):
# Abstract method, pylint: disable=R0201,W0613
return None
def getReadOnlyIndicator(self):
return self.read_only_indicator
def setReadOnlyIndicator(self, value):
assert value in ( True, False )
self.read_only_indicator = value
def getHasDelIndicator(self):
return self.has_del
def setHasDelIndicator(self):
self.has_del = True
def allocateTargetNumber(self):
self.version_number += 1
return self.version_number
# pylint: disable=R0201
def isLocalVariable(self):
return False
def isClassVariable(self):
return False
def isMaybeLocalVariable(self):
return False
def isParameterVariable(self):
return False
def isNestedParameterVariable(self):
return False
def isVariableReference(self):
return False
def isClosureReference(self):
return False
def isModuleVariableReference(self):
return False
def isReference(self):
return False
def isModuleVariable(self):
return False
def isTempVariableReference(self):
return False
def isTempVariable(self):
return False
def isTempKeeperVariable(self):
return False
# pylint: enable=R0201
def _checkShared(self, variable, technical):
for reference in variable.references:
# print( "Checking", reference, "of", variable )
if self._checkShared( reference, technical ):
return True
top_owner = reference.getReferenced().getOwner()
owner = reference.getOwner()
# The generators and functions that are not created, get things
# passed, and do not need the variable to share.
while technical and \
owner != top_owner and \
owner.isExpressionFunctionBody() and \
not owner.isGenerator() and not owner.needsCreation():
owner = owner.getParentVariableProvider()
# List contractions in Python2 do not really own their variables.
# TODO: They ought to not be variable providers/takers at all.
# TODO: This code seems unnecessary now due to "needsCreation" not
# being true.
if Utils.python_version < 300:
while owner != top_owner and owner.code_prefix == "listcontr":
owner = owner.getParentVariableProvider()
# This defines being shared. Owned by one, and references that are
# owned by another node.
if owner != top_owner:
return True
else:
return False
def isShared(self, technical = False):
variable = self
while variable.isClosureReference():
variable = variable.getReferenced()
return self._checkShared( variable, technical )
reference_class = None
def makeReference(self, owner):
# Need to provider a reference class, or else making references cannot
# work.
assert self.reference_class, self
# Search for existing references to be re-used before making a new one.
for reference in self.references:
if reference.getOwner() is owner:
return reference
else:
# The reference_class will be overloaded with something callable,
# pylint: disable=E1102
return self.reference_class(
owner = owner,
variable = self
)
def getDeclarationCode(self):
return self.getDeclarationTypeCode( in_context = False ) + \
" &" + self.getCodeName()
def getMangledName(self):
""" Get the mangled name of the variable.
By default no mangling is applied.
"""
return self.getName()
def getDeclarationTypeCode(self, in_context):
# Abstract method, pylint: disable=R0201,W0613
assert False
def getCodeName(self):
# Abstract method, pylint: disable=R0201
assert False, self
class VariableReferenceBase(Variable):
def __init__(self, owner, variable):
Variable.__init__(
self,
owner = owner,
variable_name = variable.getName()
)
if self.reference_class is None:
self.reference_class = variable.reference_class
variable.addReference( self )
self.variable = variable
del self.read_only_indicator
def getReadOnlyIndicator(self):
return self.getReferenced().read_only_indicator
def __repr__(self):
return "<%s to %s>" % (
self.__class__.__name__,
str( self.variable )[1:-1]
)
def isVariableReference(self):
return True
def isReference(self):
return True
def getReferenced(self):
return self.variable
def __cmp__(self, other):
# Compare the referenced variable, so de-reference until it's no more
# possible.
while other.getReferenced() is not None:
other = other.getReferenced()
this = self
while this.getReferenced() is not None:
this = this.getReferenced()
return cmp( this, other )
def __hash__(self):
return hash( self.getReferenced() )
class ClosureVariableReference(VariableReferenceBase):
def __init__(self, owner, variable):
assert not variable.isModuleVariable()
VariableReferenceBase.__init__(
self,
owner = owner,
variable = variable
)
def isClosureReference(self):
return True
def getProviderVariable(self):
current = self.getOwner().getParentVariableProvider()
if current is self.getReferenced().getOwner():
return self.getReferenced()
else:
for variable in current.getClosureVariables():
if variable.getName() == self.getName():
return variable
else:
assert False, self
def getDeclarationTypeCode(self, in_context):
if self.getReferenced().isShared( True ):
if in_context:
return "PyObjectClosureVariable"
else:
return "PyObjectSharedLocalVariable"
else:
return self.getReferenced().getDeclarationTypeCode(
in_context = in_context
)
def getCodeName(self):
return "closure_%s" % Utils.encodeNonAscii( self.getName() )
class ModuleVariableReference(VariableReferenceBase):
def __init__(self, owner, variable):
# Module variable access are direct pass-through, so de-reference them
# if possible.
while variable.isModuleVariableReference():
variable = variable.getReferenced()
assert variable.isModuleVariable()
VariableReferenceBase.__init__(
self,
owner = owner,
variable = variable
)
self.global_statement = False
self.exec_statement = False
def __repr__(self):
return "" % (
self.variable_name,
self.getReferenced().getModuleName(),
" from global statement" if self.global_statement else "",
" from exec statement" if self.exec_statement else "",
)
def markFromGlobalStatement(self):
self.global_statement = True
def isFromGlobalStatement(self):
return self.global_statement
def markFromExecStatement(self):
self.exec_statement = True
def isFromExecStatement(self):
return self.exec_statement
def isModuleVariableReference(self):
return True
def isModuleVariable(self):
return True
class LocalVariable(Variable):
reference_class = ClosureVariableReference
def __init__(self, owner, variable_name):
Variable.__init__(
self,
owner = owner,
variable_name = variable_name
)
assert not owner.isExpressionFunctionBody() or \
owner.local_locals or \
self.__class__ is not LocalVariable
def __repr__(self):
return "<%s '%s' of '%s'>" % (
self.__class__.__name__,
self.variable_name,
self.owner.getName()
)
def isLocalVariable(self):
return True
def getCodeName(self):
return "var_" + Utils.encodeNonAscii( self.getName() )
def getDeclarationTypeCode(self, in_context):
if self.isShared( True ):
return "PyObjectSharedLocalVariable"
else:
return "PyObjectLocalVariable"
def getMangledName(self):
if not self.variable_name.startswith( "__" ) or \
self.variable_name.endswith( "__" ):
return self.variable_name
else:
# The mangling of function variable names depends on being inside a
# class. TODO: ClassVariable seems unnecessary now.
class_container = self.owner.getContainingClassDictCreation()
if class_container is None:
return self.variable_name
else:
return "_%s%s" % (
class_container.getName().lstrip("_"),
self.variable_name
)
class ClassVariable(LocalVariable):
def getMangledName(self):
""" Get the mangled name of the variable.
In classes, names like "__name__" are not mangled, only "__name"
would be.
"""
if not self.variable_name.startswith( "__" ) or \
self.variable_name.endswith( "__" ):
return self.variable_name
else:
return "_%s%s" % (
self.getOwner().getName().lstrip("_"),
self.variable_name
)
class MaybeLocalVariable(Variable):
reference_class = ClosureVariableReference
def __init__(self, owner, variable_name):
Variable.__init__(
self,
owner = owner,
variable_name = variable_name
)
def __repr__(self):
return "<%s '%s' of '%s' maybe a global reference>" % (
self.__class__.__name__,
self.variable_name,
self.owner.getName()
)
def isMaybeLocalVariable(self):
return True
class ParameterVariable(LocalVariable):
def __init__(self, owner, parameter_name, kw_only):
LocalVariable.__init__(
self,
owner = owner,
variable_name = parameter_name
)
self.kw_only = kw_only
def isParameterVariable(self):
return True
def isParameterVariableKwOnly(self):
return self.kw_only
def getCodeName(self):
return "par_" + Utils.encodeNonAscii( self.getName() )
def getDeclarationTypeCode(self, in_context):
if self.isShared( True ):
return "PyObjectSharedLocalVariable"
elif self.getHasDelIndicator():
return "PyObjectLocalParameterVariableWithDel"
else:
return "PyObjectLocalParameterVariableNoDel"
class NestedParameterVariable(ParameterVariable):
def __init__(self, owner, parameter_name, parameter_spec):
ParameterVariable.__init__(
self,
owner = owner,
parameter_name = parameter_name,
kw_only = False
)
self.parameter_spec = parameter_spec
def isNestedParameterVariable(self):
return True
def getVariables(self):
return self.parameter_spec.getVariables()
def getAllVariables(self):
return self.parameter_spec.getAllVariables()
def getTopLevelVariables(self):
return self.parameter_spec.getTopLevelVariables()
def getParameterNames(self):
return self.parameter_spec.getParameterNames()
class ModuleVariable(Variable):
reference_class = ModuleVariableReference
def __init__(self, module, variable_name):
assert type( variable_name ) is str, repr( variable_name )
Variable.__init__(
self,
owner = module,
variable_name = variable_name
)
self.module = module
def __repr__(self):
return "" % (
self.variable_name,
self.getModuleName()
)
def isModuleVariable(self):
return True
def getModule(self):
return self.module
def getModuleName(self):
return self.module.getFullName()
def _checkShared(self, variable, technical):
assert False, variable
class TempVariableClosureReference(VariableReferenceBase):
reference_class = None
def isClosureReference(self):
# Virtual method, pylint: disable=R0201
return True
def isTempVariableReference(self):
# Virtual method, pylint: disable=R0201
return True
def getDeclarationTypeCode(self, in_context):
return self.getReferenced().getReferenced().getDeclarationTypeCode(
in_context = in_context
)
def getCodeName(self):
# Abstract method, pylint: disable=R0201
return self.getReferenced().getReferenced().getCodeName()
def getProviderVariable(self):
return self.getReferenced()
class TempVariableReference(VariableReferenceBase):
reference_class = TempVariableClosureReference
def isTempVariableReference(self):
# Virtual method, pylint: disable=R0201
return True
def makeReference(self, owner):
# Search for existing references to be re-used before making a new one.
for reference in self.references:
if reference.getOwner() is owner:
return reference
else:
if owner is self.owner:
return TempVariableReference(
owner = owner,
variable = self
)
else:
return TempVariableClosureReference(
owner = owner,
variable = self
)
class TempVariable(Variable):
reference_class = TempVariableReference
def __init__(self, owner, variable_name):
Variable.__init__(
self,
owner = owner,
variable_name = variable_name
)
# For code generation.
self.late_declaration = False
self.late_declared = False
self.needs_free = None
self.delete_statement = None
def __repr__(self):
return "" % (
self.getName(),
self.getOwner()
)
def isTempVariable(self):
# Virtual method, pylint: disable=R0201
return True
def getNeedsFree(self):
return self.needs_free
def setNeedsFree(self, needs_free):
self.needs_free = needs_free
def getDeclarationTypeCode(self, in_context):
assert self.needs_free is not None, self
if self.isShared( True ):
return "PyObjectSharedTempVariable"
elif self.needs_free:
if self.late_declaration:
if self.getHasDelIndicator():
return "PyObjectTemporaryWithDel"
else:
return "PyObjectTemporary"
else:
return "PyObjectTempVariable"
else:
return "PyObject *"
def getCodeName(self):
return "tmp_%s" % self.getName()
def getDeclarationInitValueCode(self):
# Virtual method, pylint: disable=R0201
return "NULL"
def markAsNeedsLateDeclaration(self):
self.late_declaration = True
def needsLateDeclaration(self):
return self.late_declaration
def markAsDeclared(self):
self.late_declared = True
def markAsDeleteScope(self, delete_statement):
self.delete_statement = delete_statement
def getDeleteScope(self):
return self.delete_statement
def isDeclared(self):
return self.late_declared
class TempKeeperVariable(TempVariable):
def __init__(self, owner, variable_name):
TempVariable.__init__(
self,
owner = owner,
variable_name = variable_name
)
self.write_only = False
def __repr__(self):
return "" % (
self.getName(),
self.getOwner()
)
def isTempKeeperVariable(self):
return True
def isWriteOnly(self):
return self.write_only
def setWriteOnly(self):
self.write_only = True
def getNames(variables):
return [ variable.getName() for variable in variables ]
Nuitka-0.5.0.1/nuitka/Importing.py 0000644 0001750 0001750 00000036467 12265270763 017207 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" The virtue of importing modules and packages.
The actual import of a module may already execute code that changes things.
Imagine a module that does "os.system()", it will be done. People often connect
to databases, and these kind of things, at import time. Not a good style, but
it's being done.
Therefore CPython exhibits the interfaces in an "imp" module in standard
library, which one can use those to know ahead of time, what file import would
load. For us unfortunately there is nothing in CPython that is easily accessible
and gives us this functionality for packages and search paths exactly like
CPython does, so we implement here a multi step search process that is
compatible.
This approach is much safer of course and there is no loss. To determine if it's
from the standard library, one can abuse the attribute "__file__" of the "os"
module like it's done in "isStandardLibraryPath" of this module.
"""
from . import Options, Utils
import sys, os, imp
from logging import warning
_debug_module_finding = False
warned_about = set()
# Directory where the main script lives. Should attempt to import from there.
main_path = None
def setMainScriptDirectory(main_dir):
# We need to set this from the outside, pylint: disable=W0603
global main_path
main_path = main_dir
def isPackageDir(dirname):
return Utils.isDir( dirname ) and \
(
Utils.python_version >= 330 or
Utils.isFile( Utils.joinpath( dirname, "__init__.py" ) )
)
def findModule(source_ref, module_name, parent_package, level, warn):
# We have many branches here, because there are a lot of cases to try.
# pylint: disable=R0912
if level > 1 and parent_package is not None:
parent_package = ".".join( parent_package.split(".")[ : -level+1 ] )
if parent_package == "":
parent_package = None
if module_name != "" or parent_package is not None:
try:
module_filename, module_package_name = _findModule(
module_name = module_name,
parent_package = parent_package
)
except ImportError:
if warn and not _isWhiteListedNotExistingModule( module_name ):
key = module_name, parent_package, level
if key not in warned_about:
warned_about.add( key )
if level == 0:
level_desc = "as absolute import"
elif level == -1:
level_desc = "as relative or absolute import"
elif level == 1:
level_desc = "%d package level up" % level
else:
level_desc = "%d package levels up" % level
if parent_package is not None:
warning(
"%s: Cannot find '%s' in package '%s' %s.",
source_ref.getAsString(),
module_name,
parent_package,
level_desc
)
else:
warning(
"%s: Cannot find '%s' %s.",
source_ref.getAsString(),
module_name,
level_desc
)
if "." in module_name:
module_package_name = module_name[ : module_name.rfind( "." ) ]
else:
module_package_name = None
module_filename = None
else:
if "." in module_name:
module_package_name = module_name[ : module_name.rfind( "." ) ]
else:
module_package_name = None
module_filename = None
if _debug_module_finding:
print(
"findModule: Result",
module_package_name,
module_name,
module_filename
)
return module_package_name, module_name, module_filename
def _impFindModuleWrapper(module_name, search_path):
""" This wraps imp.find_module because Python3.3 bugs.
Python3.3 accepts imports on directory names in PYTHONPATH, but does not
return them from imp.find_module, which it also deprecated, but would be
asking us to use the many variants in importlib manually. So this only
fixes up the one issue it has, that it won't accept these namespace dirs.
TODO: That probably is not sufficient to cover actual namespace packages,
where multiple such directories are to be logically joined.
"""
try:
# Does not accept keyword arguments, another thing this wrapper gives
# us then.
module_fh, module_filename, _module_desc = imp.find_module(
module_name,
search_path
)
except ImportError:
if Utils.python_version >= 330:
for path_element in search_path:
candidate = Utils.joinpath( path_element, module_name )
if Utils.isDir( candidate ):
module_filename = candidate
module_fh = None
break
else:
raise
else:
raise
# Close the file handle, we won't use it.
if module_fh is not None:
module_fh.close()
return module_filename
def _findModuleInPath(module_name, package_name):
# We have many branches here, because there are a lot of cases to try.
# pylint: disable=R0912
if _debug_module_finding:
print( "_findModuleInPath: Enter", module_name, "in", package_name )
assert main_path is not None
extra_paths = [ os.getcwd(), main_path ]
if package_name is not None:
# Work around imp.find_module bug on at least Windows. Won't handle
# module name empty in find_module. And thinking of it, how could it
# anyway.
if module_name == "":
module_name = package_name.split( "." )[ -1 ]
package_name = ".".join( package_name.split( "." )[:-1] )
def getPackageDirname(element):
return Utils.joinpath( element, *package_name.split( "." ) )
ext_path = [
getPackageDirname( element )
for element in
extra_paths + sys.path
if isPackageDir( getPackageDirname( element ) )
]
if _debug_module_finding:
print( "_findModuleInPath: Package, using extended path", ext_path )
try:
module_filename = _impFindModuleWrapper(
module_name = module_name,
search_path = ext_path
)
if _debug_module_finding:
print( "_findModuleInPath: imp.find_module worked",
module_filename, package_name )
return module_filename, package_name
except ImportError:
if _debug_module_finding:
print( "_findModuleInPath: imp.find_module failed to locate" )
except SyntaxError:
# Warn user, as this is kind of unusual.
warning(
"%s: Module cannot be imported due to syntax errors.",
module_name,
)
return None, None
ext_path = extra_paths + sys.path
if _debug_module_finding:
print( "_findModuleInPath: Non-package, using extended path", ext_path )
try:
module_filename = _impFindModuleWrapper(
module_name = module_name,
search_path = ext_path
)
except SyntaxError:
# Warn user, as this is kind of unusual.
warning(
"%s: Module cannot be imported due to syntax errors.",
module_name,
)
return None, None
if _debug_module_finding:
print( "_findModuleInPath: imp.find_module gave", module_filename )
return module_filename, None
def _findModule(module_name, parent_package):
if _debug_module_finding:
print( "_findModule: Enter", module_name, "in", parent_package )
# The os.path is strangely hacked into the os module, dispatching per
# platform, we either cannot look into it, or we require that we resolve it
# here correctly.
if module_name == "os.path" and parent_package is None:
parent_package = "os"
module_name = Utils.basename( os.path.__file__ )
if module_name.endswith( ".pyc" ):
module_name = module_name[ : -4 ]
assert module_name != "" or parent_package is not None
# Built-in module names must not be searched any further.
if module_name in sys.builtin_module_names:
return None, None
if "." in module_name:
package_part = module_name[ : module_name.rfind( "." ) ]
module_name = module_name[ module_name.rfind( "." ) + 1 : ]
# Relative import
if parent_package is not None:
try:
return _findModule(
module_name = module_name,
parent_package = parent_package + "." + package_part
)
except ImportError:
pass
# Absolute import
return _findModule(
module_name = module_name,
parent_package = package_part
)
else:
module_filename, package = _findModuleInPath(
module_name = module_name,
package_name = parent_package
)
if package == "":
package = None
return module_filename, package
def _isWhiteListedNotExistingModule(module_name):
white_list = (
"mac", "nt", "os2", "posix", "_emx_link", "riscos", "ce", "riscospath",
"riscosenviron", "Carbon.File", "org.python.core", "_sha", "_sha256",
"array", "_sha512", "_md5", "_subprocess", "msvcrt", "cPickle",
"marshal", "imp", "sys", "itertools", "cStringIO", "time", "zlib",
"thread", "math", "errno", "operator", "signal", "gc", "exceptions",
"win32process", "unicodedata", "__builtin__", "fcntl", "_socket",
"_ssl", "pwd", "spwd", "_random", "grp", "_io", "_string", "select",
"__main__", "_winreg", "_warnings", "_sre", "_functools", "_hashlib",
"_collections", "_locale", "_codecs", "_weakref", "_struct",
"_dummy_threading", "binascii", "datetime", "_ast", "xxsubtype",
"_bytesio", "cmath", "_fileio", "aetypes", "aepack", "MacOS", "cd",
"cl", "gdbm", "gl", "GL", "aetools", "_bisect", "_heapq", "_symtable",
"syslog", "_datetime", "_elementtree", "_pickle", "_posixsubprocess",
"_thread", "atexit", "pyexpat", "_imp", "_sha1", "faulthandler",
# Python-Qt4 does these if missing python3 parts:
"PyQt4.uic.port_v3.string_io", "PyQt4.uic.port_v3.load_plugin",
"PyQt4.uic.port_v3.ascii_upper", "PyQt4.uic.port_v3.proxy_base",
"PyQt4.uic.port_v3.as_string",
# CPython3 does these:
"builtins", "UserDict", "os.path", "StringIO",
# test_frozen.py
"__hello__", "__phello__", "__phello__.spam", "__phello__.foo",
# test_import.py
"RAnDoM", "infinite_reload", "test_trailing_slash",
# test_importhooks.py
"hooktestmodule", "hooktestpackage", "hooktestpackage.sub",
"reloadmodule", "hooktestpackage.sub.subber", "hooktestpackage.oldabs",
"hooktestpackage.newrel", "hooktestpackage.sub.subber.subest",
"hooktestpackage.futrel", "sub", "hooktestpackage.newabs",
# test_new.py
"Spam",
# test_pkg.py
"t1", "t2", "t2.sub", "t2.sub.subsub", "t3.sub.subsub", "t5", "t6",
"t7", "t7.sub", "t7.sub.subsub",
# test_pkgutil.py
"foo", "zipimport",
# test_platform.py
"gestalt",
# test_repr.py
"""areallylongpackageandmodulenametotestreprtruncation.\
areallylongpackageandmodulenametotestreprtruncation""",
# test_runpy.py
"test.script_helper",
# test_strftime.py
"java",
# test_strop.py
"strop",
# test_applesingle.py
"applesingle",
# test_compile.py
"__package__.module", "__mangled_mod",
# test_distutils.py
"distutils.tests", "distutils.mwerkscompiler",
# test_emails.py
"email.test.test_email", "email.test.test_email_renamed",
# test_imageop.py
"imgfile",
# test_json.py
"json.tests",
# test_lib2to3.py
"lib2to3.tests",
# test_macostools.py
"macostools",
# test_pkg.py
"t8",
# test_tk.py
"runtktests",
# test_traceback.py
"test_bug737473",
# test_zipimport_support.py
"test_zipped_doctest", "zip_pkg",
# test/test_zipimport_support.py
"test.test_cmd_line_script",
# Python3: modules that no longer exist
"commands", "dummy_thread", "_dummy_thread", "httplib", "Queue", "sets",
# Python2: modules that don't yet exit
"http.client", "queue", "winreg",
# Very old modules with older names
"simplejson", "sets",
# Standalone mode "site" import flexibilities
"sitecustomize", "usercustomize", "apport_python_hook",
"_frozen_importlib",
# Standard library stuff that is optional
"comtypes.server.inprocserver", "_tkinter", "_scproxy", "EasyDialogs",
"SOCKS", "rourl2path", "_winapi", "win32api", "win32con", "_gestalt",
"java.lang", "vms_lib", "ic", "readline", "termios", "_sysconfigdata",
)
# TODO: Turn this into a warning that encourages reporting.
if False and Options.isDebug():
for module_name in sys.builtin_module_names:
assert module_name in white_list, module_name
return module_name in white_list
def isStandardLibraryPath(path):
path = Utils.normcase( path )
# In virtual-env, the "site.py" lives in a place that suggests it is not in
# standard library, although it is.
if os.path.basename( path ) == "site.py":
return True
# These never are in standard library paths.
if "dist-packages" in path or "site-packages" in path:
return False
os_filename = os.__file__
if os_filename.endswith( ".pyc" ):
os_filename = os_filename[:-1]
os_path = Utils.normcase( Utils.dirname( os_filename ) )
candidates = [ os_path ]
# Happens for virtual-env situation, some modules will come from the link
# this points to.
if os.path.islink( os_filename ):
os_filename = os.readlink( os_filename )
candidates.append( Utils.normcase( Utils.dirname( os_filename ) ) )
for candidate in candidates:
if path.startswith( candidate ):
return True
else:
return False
Nuitka-0.5.0.1/nuitka/gui/ 0000755 0001750 0001750 00000000000 12265271051 015420 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/gui/TreeDisplay.py 0000644 0001750 0001750 00000020015 12265264105 020217 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Module with functions to display a node tree.
Useful to getting an idea of what the internal representation of Nuitka is about a source
code.
"""
from nuitka import SourceCodeReferences, Utils
from PyQt4 import QtCore, QtGui, uic
import sys
# The API requires a signature, sometimes we don't use it, pylint: disable=R0201
# Also using private stuff from classes, probably ok, pylint: disable=W0212
class NodeTreeModelItem:
def __init__(self, node, parent = None):
self.parent_treeitem = parent
self.node = node
self.children = None
def appendChild(self, _item):
assert False
def _children(self):
if self.children is None:
self.children = [
NodeTreeModelItem( child, self )
for child in
self.node.getVisitableNodes()
]
return self.children
def child(self, row):
return self._children()[ row ]
def childCount(self):
return len( self._children() )
def columnCount(self):
return 2
def data(self, column):
if column == 0:
result = self.node.getDescription()
elif column == 1:
result = self.node.getDetail()
else:
assert False
return QtCore.QVariant( result )
def parent(self):
return self.parent_treeitem
def row(self):
return self.parent_treeitem._children().index( self ) if self.parent else 0
class NodeTreeModel(QtCore.QAbstractItemModel):
def __init__(self, root, parent = None):
QtCore.QAbstractItemModel.__init__( self, parent )
self.root_node = root
self.root_item = NodeTreeModelItem( root )
def columnCount(self, _parent):
return self.root_item.columnCount()
def data(self, index, role):
if not index.isValid():
return QtCore.QVariant()
if role != QtCore.Qt.DisplayRole:
return QtCore.QVariant()
item = index.internalPointer()
return QtCore.QVariant( item.data( index.column() ) )
def flags(self, index):
if not index.isValid():
return QtCore.Qt.ItemIsEnabled
return QtCore.Qt.ItemIsEnabled | QtCore.Qt.ItemIsSelectable
def headerData(self, section, orientation, role):
if orientation == QtCore.Qt.Horizontal and role == QtCore.Qt.DisplayRole:
if section == 0:
return QtCore.QVariant( "Node Type" )
elif section == 1:
return QtCore.QVariant( "Node Detail" )
return self.root_item.data( section )
return QtCore.QVariant()
def index(self, row, column, parent):
if row < 0 or column < 0 or row >= self.rowCount( parent ) or column >= self.columnCount( parent ):
return QtCore.QModelIndex()
if not parent.isValid():
parent = self.root_item
else:
parent = parent.internalPointer()
child = parent.child( row )
if child:
return self.createIndex( row, column, child )
else:
return QtCore.QModelIndex()
def parent(self, index):
if not index.isValid():
return QtCore.QModelIndex()
child = index.internalPointer()
parent = child.parent()
if parent == self.root_item:
return QtCore.QModelIndex()
return self.createIndex( parent.row(), 0, parent )
def rowCount(self, parent):
if parent.column() > 0:
return 0
if not parent.isValid():
parent = self.root_item
else:
parent = parent.internalPointer()
return parent.childCount()
def getNodeFromPath(self, tree_path):
tree_path = list( tree_path )
current = self.root_node
while tree_path:
current = current.getVisitableNodes()[ tree_path[0] ]
del tree_path[0]
return current
def getItemFromSourceRef(self, source_ref):
def check(item):
if item.node.getSourceReference() == source_ref:
return item
for child in item._children():
result = check( child )
if result is not None:
return result
return check( self.root_item )
class InspectNodeTreeDialog(QtGui.QDialog):
def __init__(self, *args):
QtGui.QDialog.__init__( self, *args )
ui_dir = Utils.dirname( __file__ )
ui_filename = Utils.joinpath( ui_dir, "dialogs", "InspectPythonTree.ui" )
uic.loadUi( ui_filename, self )
self.treeview_nodes.setSelectionMode( self.treeview_nodes.SingleSelection )
self.displayed = None
self.source_code = None
self.model = None
self.moving = None
def setModel(self, model):
self.treeview_nodes.setModel( model )
self.treeview_nodes.expandAll()
@QtCore.pyqtSignature("on_treeview_nodes_clicked(QModelIndex)")
def onTreeviewNodesClicked(self, item):
tree_path = []
while item.isValid():
tree_path.insert( 0, item.row() )
item = item.parent()
clicked_node = self.model.getNodeFromPath( tree_path )
source_ref = clicked_node.getSourceReference()
self.moving = True
self.textedit_source.moveCursor( 1, 0 )
for _i in range( 1, source_ref.getLineNumber() ):
self.textedit_source.moveCursor( 12, 0 )
self.textedit_source.setFocus()
self.moving = False
@QtCore.pyqtSignature( "on_textedit_source_cursorPositionChanged()")
def onTexteditSourceCursorMoved(self):
if self.moving:
return
pos = self.textedit_source.textCursor().position()
code = self.source_code[:pos]
line = 1
for char in code:
if char == "\n":
line += 1
# print "Line", line
item = self.model.getItemFromSourceRef(
self.displayed.atLineNumber(
line = line
)
)
if item is not None:
item_path = []
while item:
item_path.insert( 0, item )
item = item.parent()
index = QtCore.QModelIndex()
parent = self.model.root_item
for item in item_path[1:]:
index = index.child( parent._children().index( item )+1, 1 )
parent = item
# print self.treeview_nodes.visualRect( index )
def loadSource(self, filename):
self.moving = True
self.source_code = open( filename ).read()
self.textedit_source.setPlainText( self.source_code )
self.moving = False
self.displayed = SourceCodeReferences.fromFilename(
filename = filename,
future_spec = None
)
def displayTreeInspector(tree):
app = QtGui.QApplication( sys.argv )
model = NodeTreeModel( tree )
dialog = InspectNodeTreeDialog()
dialog.setModel( model )
dialog.model = model
from . import SyntaxHighlighting
SyntaxHighlighting.addPythonHighlighter(
document = dialog.textedit_source.document()
)
dialog.loadSource( tree.getFilename() )
dialog.setWindowFlags( QtCore.Qt.Window )
dialog.show()
import signal
signal.signal(signal.SIGINT, signal.SIG_DFL)
app.exec_()
Nuitka-0.5.0.1/nuitka/gui/dialogs/ 0000755 0001750 0001750 00000000000 12265271051 017042 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/gui/dialogs/InspectPythonTree.ui 0000644 0001750 0001750 00000002120 12265044767 023037 0 ustar hayen hayen 0000000 0000000
Dialog
0
0
1030
728
Nuitka Node Tree Inspector
-
-
Node Tree
-
-
-
Source Code
Nuitka-0.5.0.1/nuitka/gui/SyntaxHighlighting.py 0000644 0001750 0001750 00000016052 12265264105 021614 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Syntax highlighting for Python.
Inspired/copied from by http://diotavelli.net/PyQtWiki/Python%20syntax%20highlighting
"""
from PyQt4.QtCore import QRegExp
from PyQt4.QtGui import QColor, QTextCharFormat, QFont, QSyntaxHighlighter
def createTextFormat(color, style=''):
"""Return a QTextCharFormat with the given attributes.
"""
_color = QColor()
_color.setNamedColor(color)
_format = QTextCharFormat()
_format.setForeground(_color)
if 'bold' in style:
_format.setFontWeight(QFont.Bold)
if 'italic' in style:
_format.setFontItalic(True)
return _format
# Syntax styles that can be shared by all languages
STYLES = {
'keyword' : createTextFormat( 'blue'),
'operator' : createTextFormat( 'red'),
'brace' : createTextFormat( 'darkGray'),
'defclass' : createTextFormat( 'black', 'bold'),
'string' : createTextFormat( 'magenta'),
'string2' : createTextFormat( 'darkMagenta'),
'comment' : createTextFormat( 'darkGreen', 'italic'),
'self' : createTextFormat( 'black', 'italic'),
'numbers' : createTextFormat( 'brown'),
}
class PythonHighlighter(QSyntaxHighlighter):
""" Syntax highlighter for the Python language.
"""
# Python keywords
keywords = [
'and', 'assert', 'break', 'class', 'continue', 'def',
'del', 'elif', 'else', 'except', 'exec', 'finally',
'for', 'from', 'global', 'if', 'import', 'in',
'is', 'lambda', 'not', 'or', 'pass', 'print',
'raise', 'return', 'try', 'while', 'with', 'yield',
'None', 'True', 'False',
]
# Python operators
operators = [
'=',
# Comparison
'==', '!=', '<', '<=', '>', '>=',
# Arithmetic
'\+', '-', '\*', '/', '//', '\%', '\*\*',
# In-place
'\+=', '-=', '\*=', '/=', '\%=',
# Bitwise
'\^', '\|', '\&', '\~', '>>', '<<',
]
# Python braces
braces = [
'\{', '\}', '\(', '\)', '\[', '\]',
]
def __init__(self, document):
QSyntaxHighlighter.__init__( self, document )
# Multi-line strings (expression, flag, style)
# The triple-quotes in these two lines will mess up the
# syntax highlighting from this point onward
self.tri_single = (QRegExp("'''"), 1, STYLES['string2'])
self.tri_double = (QRegExp('"""'), 2, STYLES['string2'])
rules = []
# Keyword, operator, and brace rules
rules += [(r'\b%s\b' % w, 0, STYLES['keyword'])
for w in PythonHighlighter.keywords]
rules += [(r'%s' % o, 0, STYLES['operator'])
for o in PythonHighlighter.operators]
rules += [(r'%s' % b, 0, STYLES['brace'])
for b in PythonHighlighter.braces]
# All other rules
rules += [
# 'self'
(r'\bself\b', 0, STYLES['self']),
# Double-quoted string, possibly containing escape sequences
(r'"[^"\\]*(\\.[^"\\]*)*"', 0, STYLES['string']),
# Single-quoted string, possibly containing escape sequences
(r"'[^'\\]*(\\.[^'\\]*)*'", 0, STYLES['string']),
# 'def' followed by an identifier
(r'\bdef\b\s*(\w+)', 1, STYLES['defclass']),
# 'class' followed by an identifier
(r'\bclass\b\s*(\w+)', 1, STYLES['defclass']),
# From '#' until a newline
(r'#[^\n]*', 0, STYLES['comment']),
# Numeric literals
(r'\b[+-]?[0-9]+[lL]?\b', 0, STYLES['numbers']),
(r'\b[+-]?0[xX][0-9A-Fa-f]+[lL]?\b', 0, STYLES['numbers']),
(r'\b[+-]?[0-9]+(?:\.[0-9]+)?(?:[eE][+-]?[0-9]+)?\b', 0, STYLES['numbers']),
]
# Build a QRegExp for each pattern
self.rules = [(QRegExp(pat), index, fmt)
for (pat, index, fmt) in rules]
def highlightBlock(self, text):
"""Apply syntax highlighting to the given block of text.
"""
# Do other syntax formatting
for expression, nth, display_format in self.rules:
index = expression.indexIn(text, 0)
while index >= 0:
# We actually want the index of the nth match
index = expression.pos(nth)
length = expression.cap(nth).length()
self.setFormat(index, length, display_format)
index = expression.indexIn(text, index + length)
self.setCurrentBlockState(0)
# Do multi-line strings
in_multiline = self.match_multiline(text, *self.tri_single)
if not in_multiline:
in_multiline = self.match_multiline(text, *self.tri_double)
def match_multiline(self, text, delimiter, in_state, style):
"""Do highlighting of multi-line strings. ``delimiter`` should be a
``QRegExp`` for triple-single-quotes or triple-double-quotes, and
``in_state`` should be a unique integer to represent the corresponding
state changes when inside those strings. Returns True if we're still
inside a multi-line string when this function is finished.
"""
# If inside triple-single quotes, start at 0
if self.previousBlockState() == in_state:
start = 0
add = 0
# Otherwise, look for the delimiter on this line
else:
start = delimiter.indexIn(text)
# Move past this match
add = delimiter.matchedLength()
# As long as there's a delimiter match on this line...
while start >= 0:
# Look for the ending delimiter
end = delimiter.indexIn(text, start + add)
# Ending delimiter on this line?
if end >= add:
length = end - start + add + delimiter.matchedLength()
self.setCurrentBlockState(0)
# No; multi-line string
else:
self.setCurrentBlockState(in_state)
length = text.length() - start + add
# Apply formatting
self.setFormat(start, length, style)
# Look for the next match
start = delimiter.indexIn(text, start + length)
# Return True if still inside a multi-line string, False otherwise
if self.currentBlockState() == in_state:
return True
else:
return False
def addPythonHighlighter(document):
PythonHighlighter( document )
Nuitka-0.5.0.1/nuitka/gui/__init__.py 0000644 0001750 0001750 00000001501 12265264105 017530 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Dummy file to make this directory a package. """
Nuitka-0.5.0.1/nuitka/tree/ 0000755 0001750 0001750 00000000000 12265271051 015573 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/tree/VariableClosure.py 0000644 0001750 0001750 00000033464 12265270513 021243 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Variable closure taking.
This is the completion of variable object completion. The variables were not
immediately resolved to be bound to actual scopes, but are only now.
Only after this is executed, variable reference nodes can be considered
complete.
"""
from nuitka import SyntaxErrors
from nuitka.Utils import python_version
from nuitka.Options import isFullCompat
from .Operations import VisitorNoopMixin, visitTree
from nuitka.nodes.ReturnNodes import StatementGeneratorReturn
# Note: We do the variable scope assignment, as an extra step from tree
# building, because tree building creates the tree without any consideration of
# evaluation order. And the ordered way these visitors are entered, will ensure
# this order.
# The main complexity is that there are two ways of visiting. One where variable
# lookups are to be done immediately, and one where it is delayed. This is
# basically class vs. function scope handling.
class VariableClosureLookupVisitorPhase1(VisitorNoopMixin):
""" Variable closure phase 1: Find assignments and early closure references.
In class context, a reference to a variable must be obeyed immediately,
so that "variable = variable" takes first "variable" as a closure and
then adds a new local "variable" to override it from there on. For the
not early closure case of a function, this will not be done and only
assigments shall add local variables, and references will be ignored
until phase 2.
"""
def onEnterNode(self, node):
if node.isExpressionTargetVariableRef():
if node.getVariable() is None:
variable_name = node.getVariableName()
provider = node.getParentVariableProvider()
variable = provider.getVariableForAssignment(
variable_name = variable_name
)
# Inside an exec, we need to ignore global declarations that are
# not ours, so we replace it with ours, unless it came from an
# 'global' declaration inside the exec
if node.source_ref.isExecReference() and not provider.isPythonModule():
if variable.isModuleVariableReference() and not variable.isFromExecStatement():
variable = provider.providing[ variable_name ] = provider.createProvidedVariable(
variable_name = variable_name
)
node.setVariable( variable )
elif node.isExpressionVariableRef():
if node.getVariable() is None:
provider = node.getParentVariableProvider()
if provider.isEarlyClosure():
node.setVariable(
provider.getVariableForReference(
variable_name = node.getVariableName()
)
)
elif node.isExpressionTempVariableRef():
if node.getVariable().getOwner() != node.getParentVariableProvider():
node.setVariable(
node.getParentVariableProvider().addClosureVariable(
node.getVariable()
)
)
assert node.getVariable().isClosureReference(), node.getVariable()
elif python_version >= 300 and node.isExpressionFunctionBody():
# Take closure variables for non-local declarations.
for non_local_names, source_ref in node.getNonlocalDeclarations():
for non_local_name in non_local_names:
# print( "nonlocal reference from", node, "to name", non_local_name )
variable = node.getClosureVariable(
variable_name = non_local_name
)
node.registerProvidedVariable( variable )
if variable.isModuleVariableReference():
SyntaxErrors.raiseSyntaxError(
"no binding for nonlocal '%s' found" % (
non_local_name
),
source_ref = None if isFullCompat() else source_ref,
display_file = not isFullCompat(),
display_line = not isFullCompat()
)
# Attribute access of names of class functions should be mangled, if
# they start with "__", but do not end in "__" as well.
elif node.isExpressionAttributeLookup() or node.isStatementAssignmentAttribute() or \
node.isStatementDelAttribute():
attribute_name = node.getAttributeName()
if attribute_name.startswith( "__" ) and not attribute_name.endswith( "__" ):
seen_function = False
current = node
while True:
current = current.getParentVariableProvider()
if current.isPythonModule():
break
assert current.isExpressionFunctionBody()
if current.isClassDictCreation():
if seen_function:
node.setAttributeName(
"_%s%s" % (
current.getName().lstrip("_"),
attribute_name
)
)
break
else:
seen_function = True
# Check if continue and break are properly in loops. If not, raise a
# syntax error.
elif node.isStatementBreakLoop() or node.isStatementContinueLoop():
current = node
while True:
if current.isPythonModule() or current.isExpressionFunctionBody():
if node.isStatementContinueLoop():
message = "'continue' not properly in loop"
col_offset = 16 if python_version >= 300 else None
display_line = True
source_line = None
else:
message = "'break' outside loop"
if isFullCompat():
col_offset = 2 if python_version >= 300 else None
display_line = True
source_line = "" if python_version >= 300 else None
else:
col_offset = 13
display_line = True
source_line = None
source_ref = node.getSourceReference()
# source_ref.line += 1
SyntaxErrors.raiseSyntaxError(
message,
source_ref = node.getSourceReference(),
col_offset = col_offset,
display_line = display_line,
source_line = source_line
)
current = current.getParent()
if current.isStatementLoop():
break
def onLeaveNode(self, node):
# Return statements in generators are not really that, instead they are
# exception raises, fix that up now. Doing it right from the onset,
# would be a bit more difficult, as the knowledge that something is a
# generator, requires a second pass.
if node.isStatementReturn() and \
node.getParentVariableProvider().isGenerator():
return_value = node.getExpression()
if python_version < 330:
if not return_value.isExpressionConstantRef() or \
return_value.getConstant() is not None:
SyntaxErrors.raiseSyntaxError(
"'return' with argument inside generator",
source_ref = node.getSourceReference(),
)
node.replaceWith(
StatementGeneratorReturn(
expression = return_value,
source_ref = node.getSourceReference()
)
)
class VariableClosureLookupVisitorPhase2(VisitorNoopMixin):
""" Variable closure phase 2: Find assignments and references.
In class context, a reference to a variable must be obeyed immediately,
so that "variable = variable" takes first "variable" as a closure and
then adds a new local "variable" to override it from there on.
So, assignments for early closure, accesses will already have a
variable set now, the others, only in this phase.
"""
def onEnterNode(self, node):
if node.isExpressionVariableRef() and node.getVariable() is None:
provider = node.getParentVariableProvider()
# print "Late reference", node.getVariableName(), "for", provider, "caused at", node, "of", node.getParent()
variable = provider.getVariableForReference(
variable_name = node.getVariableName()
)
node.setVariable(
variable
)
assert not (node.getParent().isStatementDelVariable())
# Need to catch functions with "exec" not allowed.
if python_version < 300 and \
provider.isExpressionFunctionBody() and \
variable.isReference() and \
(not variable.isModuleVariableReference() or \
not variable.isFromGlobalStatement() ):
parent_provider = provider.getParentVariableProvider()
while parent_provider.isExpressionFunctionBody() and \
parent_provider.isClassDictCreation():
parent_provider = parent_provider.getParentVariableProvider()
if parent_provider.isExpressionFunctionBody() and \
parent_provider.isUnqualifiedExec():
lines = open(
node.source_ref.getFilename(),
"rU"
).readlines()
exec_line_number = parent_provider.getExecSourceRef().getLineNumber()
raise SyntaxError(
"""\
unqualified exec is not allowed in function '%s' it \
contains a nested function with free variables""" % parent_provider.getName(),
(
node.source_ref.getFilename(),
exec_line_number,
None,
lines[ exec_line_number - 1 ]
)
)
# For Python3, every function in a class is supposed to take "__class__" as
# a reference, so make sure that happens.
if python_version >= 300:
def onLeaveNode(self, node):
if node.isExpressionFunctionBody() and node.isClassClosureTaker():
if python_version < 340:
node.getVariableForReference(
variable_name = "__class__"
)
else:
parent_provider = node.getParentVariableProvider()
variable = parent_provider.getTempVariable(
temp_scope = None,
name = "__class__"
)
variable = variable.makeReference( parent_provider )
node.addClosureVariable( variable )
class VariableClosureLookupVisitorPhase3(VisitorNoopMixin):
""" Variable closure phase 3: Find errors.
In this phase, the only task remaining is to find errors. We might e.g.
detect that a "del" was executed on a shared variable, which is not
allowed for Python 2.x, so it must be caught. The parsing wouldn't do
that. Currently this phase is Python2 only, but that may change.
"""
def onEnterNode(self, node):
assert python_version < 300
if node.isStatementDelVariable():
variable = node.getTargetVariableRef().getVariable()
if variable.isShared():
SyntaxErrors.raiseSyntaxError(
reason = """\
can not delete variable '%s' referenced in nested scope""" % (
variable.getName()
),
source_ref = (
None if isFullCompat() else node.getSourceReference()
),
display_file = not isFullCompat(),
display_line = not isFullCompat()
)
def completeVariableClosures(tree):
if python_version < 300:
visitors = (
VariableClosureLookupVisitorPhase1(),
VariableClosureLookupVisitorPhase2(),
VariableClosureLookupVisitorPhase3()
)
else:
visitors = (
VariableClosureLookupVisitorPhase1(),
VariableClosureLookupVisitorPhase2(),
)
for visitor in visitors:
visitTree( tree, visitor )
if tree.isPythonModule():
for function in tree.getFunctions():
visitTree( function, visitor )
Nuitka-0.5.0.1/nuitka/tree/ReformulationAssertStatements.py 0000644 0001750 0001750 00000006563 12265264105 024241 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Reformulation of assert statements.
Consult the developmer manual for information. TODO: Add ability to sync
source code comments with developer manual sections.
"""
from nuitka import Utils
from nuitka.nodes.BuiltinRefNodes import ExpressionBuiltinExceptionRef
from nuitka.nodes.ExceptionNodes import (
ExpressionBuiltinMakeException,
StatementRaiseException
)
from nuitka.nodes.StatementNodes import StatementsSequence
from nuitka.nodes.OperatorNodes import ExpressionOperationNOT
from nuitka.nodes.ConditionalNodes import StatementConditional
from .Helpers import buildNode
def buildAssertNode(provider, node, source_ref):
# Build assert statements. These are re-formulated as described in the
# developer manual too. They end up as conditional statement with raises of
# AssertionError exceptions.
# Underlying assumption:
#
# Assert x, y is the same as:
# if not x:
# raise AssertionError, y
# Therefore assert statements are really just conditional statements with a
# static raise contained.
#
# Starting with CPython2.7, it is, which means the creation of the exception
# object is no more delayed:
# if not x:
# raise AssertionError( y )
if Utils.python_version < 270 or node.msg is None:
raise_statement = StatementRaiseException(
exception_type = ExpressionBuiltinExceptionRef(
exception_name = "AssertionError",
source_ref = source_ref
),
exception_value = buildNode( provider, node.msg, source_ref, True ),
exception_trace = None,
exception_cause = None,
source_ref = source_ref
)
else:
raise_statement = StatementRaiseException(
exception_type = ExpressionBuiltinMakeException(
exception_name = "AssertionError",
args = (
buildNode( provider, node.msg, source_ref, True ),
),
source_ref = source_ref
),
exception_value = None,
exception_trace = None,
exception_cause = None,
source_ref = source_ref
)
return StatementConditional(
condition = ExpressionOperationNOT(
operand = buildNode( provider, node.test, source_ref ),
source_ref = source_ref
),
yes_branch = StatementsSequence(
statements = (
raise_statement,
),
source_ref = source_ref
),
no_branch = None,
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/tree/ReformulationComparisonExpressions.py 0000644 0001750 0001750 00000006417 12265264105 025303 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka.nodes.KeeperNodes import (
ExpressionAssignmentTempKeeper,
ExpressionTempKeeperRef
)
from .ReformulationBooleanExpressions import buildAndNode
from .Helpers import (
buildNode,
getKind
)
def buildComparisonNode(provider, node, source_ref):
from nuitka.nodes.NodeMakingHelpers import makeComparisonNode
assert len( node.comparators ) == len( node.ops )
# Comparisons are re-formulated as described in the developer manual. When
# having multiple compators, things require assignment expressions and
# references of them to work properly. Then they can become normal "and"
# code.
# The operands are split out
left = buildNode( provider, node.left, source_ref )
rights = [
buildNode( provider, comparator, source_ref )
for comparator in
node.comparators
]
# Only the first comparison has as left operands as the real thing, the
# others must reference the previous comparison right one temp variable ref.
result = []
# For PyLint to like it, this will hold the previous one, normally.
keeper_variable = None
for comparator, right in zip( node.ops, rights ):
if result:
# Now we know it's not the only one, so we change the "left" to be a
# reference to the previously saved right side.
left = ExpressionTempKeeperRef(
variable = keeper_variable.makeReference( provider ),
source_ref = source_ref
)
keeper_variable = None
if right is not rights[-1]:
# Now we know it's not the last one, so we ought to preseve the
# "right" so it can be referenced by the next part that will
# come. We do it by assining it to a temp variable to be shared with
# the next part.
keeper_variable = provider.allocateTempKeeperVariable()
right = ExpressionAssignmentTempKeeper(
variable = keeper_variable.makeReference( provider ),
source = right,
source_ref = source_ref
)
comparator = getKind( comparator )
result.append(
makeComparisonNode(
left = left,
right = right,
comparator = comparator,
source_ref = source_ref
)
)
assert keeper_variable is None
return buildAndNode(
provider = provider,
values = result,
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/tree/ReformulationClasses.py 0000644 0001750 0001750 00000077426 12265264105 022333 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Reformulation of classes
Consult the developmer manual for information. TODO: Add ability to sync
source code comments with developer manual sections.
"""
from nuitka.nodes.VariableRefNodes import (
ExpressionTargetTempVariableRef,
ExpressionTargetVariableRef,
ExpressionTempVariableRef,
ExpressionVariableRef
)
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.BuiltinRefNodes import ExpressionBuiltinRef
from nuitka.nodes.ComparisonNodes import ExpressionComparison
from nuitka.nodes.CallNodes import (
ExpressionCallNoKeywords,
ExpressionCall
)
from nuitka.nodes.TypeNodes import ExpressionBuiltinType1
from nuitka.nodes.AttributeNodes import (
ExpressionAttributeLookup,
ExpressionBuiltinHasattr
)
from nuitka.nodes.SubscriptNodes import ExpressionSubscriptLookup
from nuitka.nodes.FunctionNodes import (
ExpressionFunctionCreation,
ExpressionFunctionBody,
ExpressionFunctionCall,
ExpressionFunctionRef
)
from nuitka.nodes.ClassNodes import ExpressionSelectMetaclass
from nuitka.nodes.ContainerMakingNodes import ExpressionMakeTuple
from nuitka.nodes.ContainerOperationNodes import (
StatementDictOperationRemove,
ExpressionDictOperationGet
)
from nuitka.nodes.StatementNodes import StatementsSequence
from nuitka.nodes.ConditionalNodes import (
ExpressionConditional,
StatementConditional
)
from nuitka.nodes.ReturnNodes import StatementReturn
from nuitka.nodes.AssignNodes import (
StatementAssignmentVariable,
StatementDelVariable
)
from nuitka.nodes.GlobalsLocalsNodes import (
ExpressionBuiltinLocals,
StatementSetLocals
)
from nuitka.nodes.ParameterSpecs import ParameterSpec
from .Helpers import (
makeStatementsSequenceFromStatement,
makeSequenceCreationOrConstant,
makeDictCreationOrConstant,
makeStatementsSequence,
buildStatementsNode,
extractDocFromBody,
buildNodeList,
buildNode,
getKind
)
from nuitka import Utils
# TODO: Once we start to modify these, we should make sure, the copy is not
# shared.
make_class_parameters = ParameterSpec(
name = "class",
normal_args = (),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
)
def _buildClassNode3(provider, node, source_ref):
# Many variables, due to the huge re-formulation that is going on here,
# which just has the complexity, pylint: disable=R0914
# This function is the Python3 special case with special re-formulation as
# according to developer manual.
class_statements, class_doc = extractDocFromBody(node)
# We need a scope for the temporary variables, and they might be closured.
temp_scope = provider.allocateTempScope(
name = "class_creation",
allow_closure = True
)
tmp_bases = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "bases"
)
tmp_class_decl_dict = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "class_decl_dict"
)
tmp_metaclass = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "metaclass"
)
tmp_prepared = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "prepared"
)
class_creation_function = ExpressionFunctionBody(
provider = provider,
is_class = True,
parameters = make_class_parameters,
name = node.name,
doc = class_doc,
source_ref = source_ref
)
# Hack: This allows some APIs to work although this is not yet officially a
# child yet.
class_creation_function.parent = provider
body = buildStatementsNode(
provider = class_creation_function,
nodes = class_statements,
frame = True,
source_ref = source_ref
)
source_ref_orig = source_ref
source_ref = source_ref.atInternal()
if body is not None:
# The frame guard has nothing to tell its line number to.
body.source_ref = source_ref
statements = [
StatementSetLocals(
new_locals = ExpressionTempVariableRef(
variable = tmp_prepared.makeReference( provider ),
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__module__",
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = provider.getParentModule().getFullName(),
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref.atInternal()
)
]
if class_doc is not None:
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__doc__",
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = class_doc,
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref.atInternal()
)
)
# The "__qualname__" attribute is new in Python 3.3.
if Utils.python_version >= 330:
qualname = class_creation_function.getFunctionQualname()
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__qualname__",
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = qualname,
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
)
)
if Utils.python_version >= 340:
tmp_class = class_creation_function.allocateTempVariable(
temp_scope = None,
name = "__class__"
)
class_target_variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_class.makeReference(class_creation_function),
source_ref = source_ref
)
class_variable_ref = ExpressionTempVariableRef(
variable = tmp_class.makeReference(class_creation_function),
source_ref = source_ref
)
else:
class_target_variable_ref = ExpressionTargetVariableRef(
variable_name = "__class__",
source_ref = source_ref
)
class_variable_ref = ExpressionVariableRef(
variable_name = "__class__",
source_ref = source_ref
)
statements += [
body,
StatementAssignmentVariable(
variable_ref = class_target_variable_ref,
source = ExpressionCall(
called = ExpressionTempVariableRef(
variable = tmp_metaclass.makeReference(provider),
source_ref = source_ref
),
args = makeSequenceCreationOrConstant(
sequence_kind = "tuple",
elements = (
ExpressionConstantRef(
constant = node.name,
source_ref = source_ref,
user_provided = True
),
ExpressionTempVariableRef(
variable = tmp_bases.makeReference(provider),
source_ref = source_ref
),
ExpressionBuiltinLocals(
source_ref = source_ref
)
),
source_ref = source_ref
),
kw = ExpressionTempVariableRef(
variable = tmp_class_decl_dict.makeReference(provider),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
StatementReturn(
expression = class_variable_ref,
source_ref = source_ref
)
]
body = makeStatementsSequence(
statements = statements,
allow_none = True,
source_ref = source_ref
)
# The class body is basically a function that implicitely, at the end
# returns its locals and cannot have other return statements contained.
class_creation_function.setBody(body)
# The class body is basically a function that implicitely, at the end
# returns its created class and cannot have other return statements
# contained.
decorated_body = ExpressionFunctionCall(
function = ExpressionFunctionCreation(
function_ref = ExpressionFunctionRef(
function_body = class_creation_function,
source_ref = source_ref
),
defaults = (),
kw_defaults = None,
annotations = None,
source_ref = source_ref
),
values = (),
source_ref = source_ref
)
for decorator in buildNodeList(
provider,
reversed( node.decorator_list ),
source_ref
):
decorated_body = ExpressionCallNoKeywords(
called = decorator,
args = ExpressionMakeTuple(
elements = ( decorated_body, ),
source_ref = source_ref
),
source_ref = decorator.getSourceReference()
)
statements = (
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_bases.makeReference( provider ),
source_ref = source_ref
),
source = makeSequenceCreationOrConstant(
sequence_kind = "tuple",
elements = buildNodeList(
provider, node.bases, source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_class_decl_dict.makeReference(provider),
source_ref = source_ref
),
source = makeDictCreationOrConstant(
keys = [
ExpressionConstantRef(
constant = keyword.arg,
source_ref = source_ref,
user_provided = True
)
for keyword in
node.keywords
],
values = [
buildNode( provider, keyword.value, source_ref )
for keyword in
node.keywords
],
lazy_order = False,
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_metaclass.makeReference(provider),
source_ref = source_ref
),
source = ExpressionSelectMetaclass(
metaclass = ExpressionConditional(
condition = ExpressionComparison(
comparator = "In",
left = ExpressionConstantRef(
constant = "metaclass",
source_ref = source_ref,
user_provided = True
),
right = ExpressionTempVariableRef(
variable = tmp_class_decl_dict.makeReference(
provider
),
source_ref = source_ref
),
source_ref = source_ref
),
yes_expression = ExpressionDictOperationGet(
dicte = ExpressionTempVariableRef(
variable = tmp_class_decl_dict.makeReference(
provider
),
source_ref = source_ref
),
key = ExpressionConstantRef(
constant = "metaclass",
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
),
no_expression = ExpressionConditional(
condition = ExpressionTempVariableRef(
variable = tmp_bases.makeReference( provider ),
source_ref = source_ref
),
no_expression = ExpressionBuiltinRef(
builtin_name = "type",
source_ref = source_ref
),
yes_expression = ExpressionBuiltinType1(
value = ExpressionSubscriptLookup(
expression = ExpressionTempVariableRef(
variable = tmp_bases.makeReference(
provider
),
source_ref = source_ref
),
subscript = ExpressionConstantRef(
constant = 0,
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
bases = ExpressionTempVariableRef(
variable = tmp_bases.makeReference( provider ),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref_orig
),
StatementConditional(
condition = ExpressionComparison(
comparator = "In",
left = ExpressionConstantRef(
constant = "metaclass",
source_ref = source_ref,
user_provided = True
),
right = ExpressionTempVariableRef(
variable = tmp_class_decl_dict.makeReference(
provider
),
source_ref = source_ref
),
source_ref = source_ref
),
no_branch = None,
yes_branch = makeStatementsSequenceFromStatement(
statement = StatementDictOperationRemove(
dicte = ExpressionTempVariableRef(
variable = tmp_class_decl_dict.makeReference(
provider
),
source_ref = source_ref
),
key = ExpressionConstantRef(
constant = "metaclass",
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
)
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_prepared.makeReference(
provider
),
source_ref = source_ref
),
source = ExpressionConditional(
condition = ExpressionBuiltinHasattr(
object = ExpressionTempVariableRef(
variable = tmp_metaclass.makeReference(
provider
),
source_ref = source_ref
),
name = ExpressionConstantRef(
constant = "__prepare__",
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
),
no_expression = ExpressionConstantRef(
constant = {},
source_ref = source_ref,
user_provided = True
),
yes_expression = ExpressionCall(
called = ExpressionAttributeLookup(
expression = ExpressionTempVariableRef(
variable = tmp_metaclass.makeReference(
provider
),
source_ref = source_ref
),
attribute_name = "__prepare__",
source_ref = source_ref
),
args = ExpressionMakeTuple(
elements = (
ExpressionConstantRef(
constant = node.name,
source_ref = source_ref,
user_provided = True
),
ExpressionTempVariableRef(
variable = tmp_bases.makeReference(
provider
),
source_ref = source_ref
)
),
source_ref = source_ref
),
kw = ExpressionTempVariableRef(
variable = tmp_class_decl_dict.makeReference(
provider
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = node.name,
source_ref = source_ref
),
source = decorated_body,
source_ref = source_ref
),
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_bases.makeReference( provider ),
source_ref = source_ref
),
tolerant = False,
source_ref = source_ref
),
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_class_decl_dict.makeReference(provider),
source_ref = source_ref
),
tolerant = False,
source_ref = source_ref
),
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_metaclass.makeReference(provider),
source_ref = source_ref
),
tolerant = False,
source_ref = source_ref
),
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_prepared.makeReference( provider ),
source_ref = source_ref
),
tolerant = False,
source_ref = source_ref
)
)
return StatementsSequence(
statements = statements,
source_ref = source_ref
)
def _buildClassNode2(provider, node, source_ref):
class_statements, class_doc = extractDocFromBody(node)
# This function is the Python2 special case with special re-formulation as
# according to developer manual.
function_body = ExpressionFunctionBody(
provider = provider,
is_class = True,
parameters = make_class_parameters,
name = node.name,
doc = class_doc,
source_ref = source_ref
)
body = buildStatementsNode(
provider = function_body,
nodes = class_statements,
frame = True,
source_ref = source_ref
)
if body is not None:
# The frame guard has nothing to tell its line number to.
body.source_ref = source_ref.atInternal()
# The class body is basically a function that implicitely, at the end
# returns its locals and cannot have other return statements contained, and
# starts out with a variables "__module__" and potentially "__doc__" set.
statements = [
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__module__",
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = provider.getParentModule().getFullName(),
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref.atInternal()
)
]
if class_doc is not None:
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__doc__",
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = class_doc,
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref.atInternal()
)
)
statements += [
body,
StatementReturn(
expression = ExpressionBuiltinLocals(
source_ref = source_ref
),
source_ref = source_ref.atInternal()
)
]
body = makeStatementsSequence(
statements = statements,
allow_none = True,
source_ref = source_ref
)
# The class body is basically a function that implicitely, at the end
# returns its locals and cannot have other return statements contained.
function_body.setBody( body )
temp_scope = provider.allocateTempScope( "class_creation" )
tmp_bases = provider.allocateTempVariable( temp_scope, "bases" )
tmp_class_dict = provider.allocateTempVariable(temp_scope, "class_dict")
tmp_metaclass = provider.allocateTempVariable(temp_scope, "metaclass")
tmp_class = provider.allocateTempVariable(temp_scope, "class")
statements = [
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_bases.makeReference( provider ),
source_ref = source_ref
),
source = makeSequenceCreationOrConstant(
sequence_kind = "tuple",
elements = buildNodeList(
provider, node.bases, source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_class_dict.makeReference(provider),
source_ref = source_ref
),
source = ExpressionFunctionCall(
function = ExpressionFunctionCreation(
function_ref = ExpressionFunctionRef(
function_body = function_body,
source_ref = source_ref
),
defaults = (),
kw_defaults = None,
annotations = None,
source_ref = source_ref
),
values = (),
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_metaclass.makeReference(provider),
source_ref = source_ref
),
source = ExpressionConditional(
condition = ExpressionComparison(
comparator = "In",
left = ExpressionConstantRef(
constant = "__metaclass__",
source_ref = source_ref,
user_provided = True
),
right = ExpressionTempVariableRef(
variable = tmp_class_dict.makeReference(provider),
source_ref = source_ref
),
source_ref = source_ref
),
yes_expression = ExpressionDictOperationGet(
dicte = ExpressionTempVariableRef(
variable = tmp_class_dict.makeReference(provider),
source_ref = source_ref
),
key = ExpressionConstantRef(
constant = "__metaclass__",
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
),
no_expression = ExpressionSelectMetaclass(
metaclass = None,
bases = ExpressionTempVariableRef(
variable = tmp_bases.makeReference( provider ),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_class.makeReference(provider),
source_ref = source_ref
),
source = ExpressionCallNoKeywords(
called = ExpressionTempVariableRef(
variable = tmp_metaclass.makeReference(provider),
source_ref = source_ref
),
args = ExpressionMakeTuple(
elements = (
ExpressionConstantRef(
constant = node.name,
source_ref = source_ref,
user_provided = True
),
ExpressionTempVariableRef(
variable = tmp_bases.makeReference( provider ),
source_ref = source_ref
),
ExpressionTempVariableRef(
variable = tmp_class_dict.makeReference(
provider
),
source_ref = source_ref
)
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_bases.makeReference( provider ),
source_ref = source_ref
),
tolerant = False,
source_ref = source_ref
),
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_class_dict.makeReference(provider),
source_ref = source_ref
),
tolerant = False,
source_ref = source_ref
),
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_metaclass.makeReference(provider),
source_ref = source_ref
),
tolerant = False,
source_ref = source_ref
)
]
for decorator in buildNodeList(
provider,
reversed( node.decorator_list ),
source_ref
):
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_class.makeReference(provider),
source_ref = source_ref
),
source = ExpressionCallNoKeywords(
called = decorator,
args = ExpressionMakeTuple(
elements = (
ExpressionTempVariableRef(
variable = tmp_class.makeReference(
provider
),
source_ref = source_ref
),
),
source_ref = source_ref
),
source_ref = decorator.getSourceReference()
),
source_ref = decorator.getSourceReference()
)
)
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = node.name,
source_ref = source_ref
),
source = ExpressionTempVariableRef(
variable = tmp_class.makeReference(provider),
source_ref = source_ref
),
source_ref = source_ref
)
)
statements.append(
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_class.makeReference(provider),
source_ref = source_ref
),
tolerant = False,
source_ref = source_ref
)
)
return StatementsSequence(
statements = statements,
source_ref = source_ref
)
def buildClassNode(provider, node, source_ref):
assert getKind( node ) == "ClassDef"
# Python2 and Python3 are similar, but fundamentally different, so handle
# them in dedicated code.
if Utils.python_version >= 300:
return _buildClassNode3( provider, node, source_ref )
else:
return _buildClassNode2( provider, node, source_ref )
Nuitka-0.5.0.1/nuitka/tree/ReformulationFunctionStatements.py 0000644 0001750 0001750 00000023673 12265264105 024566 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka import Utils, SyntaxErrors
from nuitka.nodes.ParameterSpecs import ParameterSpec
from nuitka.nodes.VariableRefNodes import ExpressionTargetVariableRef
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.BuiltinRefNodes import ExpressionBuiltinRef
from nuitka.nodes.CallNodes import ExpressionCallNoKeywords
from nuitka.nodes.FunctionNodes import (
ExpressionFunctionCreation,
ExpressionFunctionBody,
ExpressionFunctionRef
)
from nuitka.nodes.ContainerMakingNodes import ExpressionMakeTuple
from nuitka.nodes.ReturnNodes import StatementReturn
from nuitka.nodes.AssignNodes import StatementAssignmentVariable
from .Helpers import (
makeStatementsSequenceFromStatement,
makeDictCreationOrConstant,
buildStatementsNode,
extractDocFromBody,
buildNodeList,
buildNode,
getKind
)
def buildFunctionNode(provider, node, source_ref):
assert getKind( node ) == "FunctionDef"
# Remove "exec" flag if any.
source_ref = source_ref.getExecReference( False )
function_statements, function_doc = extractDocFromBody( node )
function_body = ExpressionFunctionBody(
provider = provider,
name = node.name,
doc = function_doc,
parameters = buildParameterSpec( node.name, node, source_ref ),
source_ref = source_ref
)
# Hack:
function_body.parent = provider
decorators = buildNodeList(
provider = provider,
nodes = reversed( node.decorator_list ),
source_ref = source_ref
)
defaults = buildNodeList(
provider = provider,
nodes = node.args.defaults,
source_ref = source_ref
)
kw_defaults = buildParameterKwDefaults(
provider, node, function_body, source_ref
)
function_statements_body = buildStatementsNode(
provider = function_body,
nodes = function_statements,
frame = True,
source_ref = source_ref
)
if function_body.isExpressionFunctionBody() and function_body.isGenerator():
# TODO: raise generator exit?
pass
elif function_statements_body is None:
function_statements_body = makeStatementsSequenceFromStatement(
statement = StatementReturn(
expression = ExpressionConstantRef(
constant = None,
source_ref = source_ref.atInternal()
),
source_ref = source_ref.atInternal()
)
)
elif not function_statements_body.isStatementAborting():
function_statements_body.setStatements(
function_statements_body.getStatements() +
(
StatementReturn(
expression = ExpressionConstantRef(
constant = None,
source_ref = source_ref
),
source_ref = source_ref.atInternal()
),
)
)
function_body.setBody( function_statements_body )
annotations = buildParameterAnnotations( provider, node, source_ref )
decorated_body = ExpressionFunctionCreation(
function_ref = ExpressionFunctionRef(
function_body,
source_ref = source_ref
),
defaults = defaults,
kw_defaults = kw_defaults,
annotations = annotations,
source_ref = source_ref
)
# Add the staticmethod decorator to __new__ methods if not provided.
# CPython made these optional, but applies them to every class __new__. We
# add them early, so our optimization will see it.
if node.name == "__new__" and not decorators and \
provider.isExpressionFunctionBody() and provider.isClassDictCreation():
decorators = (
ExpressionBuiltinRef(
builtin_name = "staticmethod",
source_ref = source_ref
),
)
for decorator in decorators:
decorated_body = ExpressionCallNoKeywords(
called = decorator,
args = ExpressionMakeTuple(
elements = ( decorated_body, ),
source_ref = source_ref
),
source_ref = decorator.getSourceReference()
)
return StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = node.name,
source_ref = source_ref
),
source = decorated_body,
source_ref = source_ref
)
def buildParameterKwDefaults(provider, node, function_body, source_ref):
# Build keyword only arguments default values. We are hiding here, that it
# is a Python3 only feature.
if Utils.python_version >= 300:
kw_only_names = function_body.getParameters().getKwOnlyParameterNames()
if kw_only_names:
keys = []
values = []
for kw_only_name, kw_default in \
zip( kw_only_names, node.args.kw_defaults ):
if kw_default is not None:
keys.append(
ExpressionConstantRef(
constant = kw_only_name,
source_ref = source_ref
)
)
values.append(
buildNode( provider, kw_default, source_ref )
)
kw_defaults = makeDictCreationOrConstant(
keys = keys,
values = values,
lazy_order = False,
source_ref = source_ref
)
else:
kw_defaults = None
else:
kw_defaults = None
return kw_defaults
def buildParameterAnnotations(provider, node, source_ref):
# Too many branches, because there is too many cases, pylint: disable=R0912
# Build annotations. We are hiding here, that it is a Python3 only feature.
if Utils.python_version < 300:
return None
keys = []
values = []
def addAnnotation(key, value):
keys.append(
ExpressionConstantRef(
constant = key,
source_ref = source_ref,
user_provided = True
)
)
values.append( value )
def extractArg(arg):
if getKind( arg ) == "Name":
assert arg.annotation is None
elif getKind( arg ) == "arg":
if arg.annotation is not None:
addAnnotation(
key = arg.arg,
value = buildNode( provider, arg.annotation, source_ref )
)
elif getKind( arg ) == "Tuple":
for arg in arg.elts:
extractArg( arg )
else:
assert False, getKind( arg )
for arg in node.args.args:
extractArg( arg )
for arg in node.args.kwonlyargs:
extractArg( arg )
if Utils.python_version < 340:
if node.args.varargannotation is not None:
addAnnotation(
key = node.args.vararg,
value = buildNode(
provider, node.args.varargannotation, source_ref
)
)
if node.args.kwargannotation is not None:
addAnnotation(
key = node.args.kwarg,
value = buildNode(
provider, node.args.kwargannotation, source_ref
)
)
else:
if node.args.vararg is not None:
extractArg( node.args.vararg )
if node.args.kwarg is not None:
extractArg( node.args.kwarg )
# Return value annotation (not there for lambdas)
if hasattr( node, "returns" ) and node.returns is not None:
addAnnotation(
key = "return",
value = buildNode( provider, node.returns, source_ref )
)
if keys:
return makeDictCreationOrConstant(
keys = keys,
values = values,
lazy_order = False,
source_ref = source_ref
)
else:
return None
def buildParameterSpec(name, node, source_ref):
kind = getKind( node )
assert kind in ( "FunctionDef", "Lambda" ), "unsupported for kind " + kind
def extractArg(arg):
if type( arg ) is str or arg is None:
return arg
elif getKind( arg ) == "Name":
return arg.id
elif getKind( arg ) == "arg":
return arg.arg
elif getKind( arg ) == "Tuple":
return tuple( extractArg( arg ) for arg in arg.elts )
else:
assert False, getKind( arg )
result = ParameterSpec(
name = name,
normal_args = [ extractArg( arg ) for arg in node.args.args ],
kw_only_args = [ extractArg( arg ) for arg in node.args.kwonlyargs ]
if Utils.python_version >= 300 else
[],
list_star_arg = extractArg( node.args.vararg ),
dict_star_arg = extractArg( node.args.kwarg ),
default_count = len(node.args.defaults)
)
message = result.checkValid()
if message is not None:
SyntaxErrors.raiseSyntaxError(
message,
source_ref
)
return result
Nuitka-0.5.0.1/nuitka/tree/ReformulationNamespacePackages.py 0000644 0001750 0001750 00000006556 12265264105 024265 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Namespace packages of Python3.3
"""
from nuitka.nodes.ModuleNodes import PythonPackage
from nuitka.SourceCodeReferences import SourceCodeReference
from nuitka.nodes.FutureSpecs import FutureSpec
from nuitka.tree.Helpers import makeStatementsSequenceFromStatement
from nuitka.nodes.AssignNodes import StatementAssignmentVariable
from nuitka.nodes.VariableRefNodes import ExpressionTargetVariableRef
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.tree.VariableClosure import completeVariableClosures
from nuitka.nodes.CallNodes import ExpressionCallNoKeywords
from nuitka.nodes.ImportNodes import (
ExpressionImportName,
ExpressionImportModule
)
def createNamespacePackage(package_name, module_relpath):
parts = package_name.split(".")
source_ref = SourceCodeReference.fromFilenameAndLine(
module_relpath,
1,
FutureSpec(),
False
)
source_ref = source_ref.atInternal()
package_package_name = ".".join(parts[:-1]) or None
package = PythonPackage(
name = parts[-1],
package_name = package_package_name,
source_ref = source_ref,
)
package.setBody(
makeStatementsSequenceFromStatement(
statement = (
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__path__",
source_ref = source_ref
),
source = ExpressionCallNoKeywords(
called = ExpressionImportName(
module = ExpressionImportModule(
module_name = "_frozen_importlib",
import_list = (),
level = 0,
source_ref = source_ref
),
import_name = "_NamespacePath",
source_ref = source_ref
),
args = ExpressionConstantRef(
constant = (
package_name,
[ module_relpath ],
None
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
)
)
)
completeVariableClosures( package )
return source_ref, package
Nuitka-0.5.0.1/nuitka/tree/Operations.py 0000644 0001750 0001750 00000003163 12265264105 020275 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Operations on the tree.
This is mostly for the different kinds of visits that the node tree can have.
You can visit a scope, a tree (module), or every scope of a tree (module).
"""
def visitTree(tree, visitor):
visitor.onEnterNode( tree )
for visitable in tree.getVisitableNodes():
if visitable is None:
raise AssertionError( "'None' child encountered", tree, tree.source_ref )
visitTree( visitable, visitor )
visitor.onLeaveNode( tree )
def visitFunction(function, visitor):
visitor.onEnterNode( function )
visitor.onLeaveNode( function )
class VisitorNoopMixin:
def onEnterNode(self, node):
""" Overloaded for operation before the node children were done. """
pass
def onLeaveNode(self, node):
""" Overloaded for operation after the node children were done. """
pass
Nuitka-0.5.0.1/nuitka/tree/SourceReading.py 0000644 0001750 0001750 00000011677 12265264105 020715 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Read source code from files.
This is tremendously more complex than one might think, due to encoding issues
and version differences of Python.
"""
from nuitka import Utils, SyntaxErrors, SourceCodeReferences
import re
def _readSourceCodeFromFilename3(source_filename):
with open(source_filename, "rb") as source_file:
source_code = source_file.read()
if source_code.startswith( b'\xef\xbb\xbf' ):
source_code = source_code[3:]
new_line = source_code.find(b"\n")
if new_line is not -1:
line = source_code[ : new_line ]
line_match = re.search( b"coding[:=]\\s*([-\\w.]+)", line )
if line_match:
encoding = line_match.group(1).decode("ascii")
# Detect encoding problem, as decode won't raise the compatible
# thing.
try:
import codecs
codecs.lookup(encoding)
except LookupError:
SyntaxErrors.raiseSyntaxError(
reason = "unknown encoding: %s" % encoding,
source_ref = SourceCodeReferences.fromFilename(
source_filename,
None
),
display_line = False
)
return source_code[ new_line : ].decode(encoding)
new_line = source_code.find(b"\n", new_line+1)
if new_line is not -1:
line = source_code[ : new_line ]
line_match = re.search(b"coding[:=]\\s*([-\\w.]+)", line)
if line_match:
encoding = line_match.group(1).decode("ascii")
return "\n" + source_code[ new_line : ].decode(encoding)
return source_code.decode("utf-8")
def _detectEncoding2(source_filename):
# Detect the encoding.
encoding = "ascii"
with open( source_filename, "rb" ) as source_file:
line1 = source_file.readline()
if line1.startswith(b'\xef\xbb\xbf'):
encoding = "utf-8"
else:
line1_match = re.search(b"coding[:=]\\s*([-\\w.]+)", line1)
if line1_match:
encoding = line1_match.group(1)
else:
line2 = source_file.readline()
line2_match = re.search(b"coding[:=]\\s*([-\\w.]+)", line2)
if line2_match:
encoding = line2_match.group(1)
return encoding
def _readSourceCodeFromFilename2(source_filename):
# Detect the encoding.
encoding = _detectEncoding2(source_filename)
with open(source_filename, "rU") as source_file:
source_code = source_file.read()
# Try and detect SyntaxError from missing or wrong encodings.
if type(source_code) is not unicode and encoding == "ascii":
try:
source_code = source_code.decode(encoding)
except UnicodeDecodeError as e:
lines = source_code.split("\n")
so_far = 0
for count, line in enumerate(lines):
so_far += len(line) + 1
if so_far > e.args[2]:
break
else:
# Cannot happen, decode error implies non-empty.
count = -1
wrong_byte = re.search(
"byte 0x([a-f0-9]{2}) in position",
str( e )
).group( 1 )
SyntaxErrors.raiseSyntaxError(
reason = """\
Non-ASCII character '\\x%s' in file %s on line %d, but no encoding declared; \
see http://www.python.org/peps/pep-0263.html for details""" % (
wrong_byte,
source_filename,
count+1,
),
source_ref = SourceCodeReferences.fromFilename(
source_filename,
None
).atLineNumber(count+1),
display_line = False
)
return source_code
def readSourceCodeFromFilename(source_filename):
if Utils.python_version < 300:
return _readSourceCodeFromFilename2(source_filename)
else:
return _readSourceCodeFromFilename3(source_filename)
Nuitka-0.5.0.1/nuitka/tree/Recursion.py 0000644 0001750 0001750 00000024357 12265264105 020133 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Recursion into other modules.
"""
from nuitka import Options, Utils, Importing, ModuleRegistry
from nuitka.freezer.BytecodeModuleFreezer import isFrozenModule
from . import ImportCache, Building
from logging import debug, warning
def recurseTo(module_package, module_filename, module_relpath, module_kind,
reason ):
if not ImportCache.isImportedModuleByPath( module_relpath ):
module, source_ref, source_filename = Building.decideModuleTree(
filename = module_filename,
package = module_package,
is_top = False,
is_main = False,
is_shlib = module_kind == "shlib"
)
# Check if the module name is known. In order to avoid duplicates,
# learn the new filename, and continue build if its not.
if not ImportCache.isImportedModuleByName( module.getFullName() ):
debug(
"Recurse to import '%s' from %s. (%s)",
module.getFullName(),
module_relpath,
reason
)
if module_kind == "py" and source_filename is not None:
try:
Building.createModuleTree(
module = module,
source_ref = source_ref,
source_filename = source_filename,
is_main = False
)
except ( SyntaxError, IndentationError ) as e:
if module_filename not in Importing.warned_about:
Importing.warned_about.add( module_filename )
warning(
"""\
Cannot recurse to import module '%s' (%s) because of '%s'""",
module_relpath,
module_filename,
e.__class__.__name__
)
return None, False
ImportCache.addImportedModule(
module_relpath,
module
)
is_added = True
else:
ImportCache.addImportedModule(
module_relpath,
ImportCache.getImportedModuleByName( module.getFullName() )
)
module = ImportCache.getImportedModuleByName(
module.getFullName()
)
is_added = False
assert not module_relpath.endswith( "/__init__.py" ), module
return module, is_added
else:
return ImportCache.getImportedModuleByPath( module_relpath ), False
def decideRecursion(module_filename, module_name, module_package,
module_kind ):
# Many branches, which make decisions immediately, pylint: disable=R0911
if module_kind == "shlib":
if Options.isStandaloneMode():
return True, "Shared library for inclusion."
else:
return False, "Shared library cannot be inspected."
if module_package is None:
full_name = module_name
else:
full_name = module_package + "." + module_name
if isFrozenModule(full_name):
return False, "Module is frozen."
no_case_modules = Options.getShallFollowInNoCase()
for no_case_module in no_case_modules:
if full_name == no_case_module:
return (
False,
"Module listed explicitely to not recurse to."
)
if full_name.startswith( no_case_module + "." ):
return (
False,
"Module in package listed explicitely to not recurse to."
)
any_case_modules = Options.getShallFollowModules()
for any_case_module in any_case_modules:
if full_name == any_case_module:
return (
True,
"Module listed explicitely to recurse to."
)
if full_name.startswith( any_case_module + "." ):
return (
True,
"Module in package listed explicitely to recurse to."
)
if Options.shallFollowNoImports():
return (
False,
"Requested to not recurse at all."
)
if Importing.isStandardLibraryPath( module_filename ):
return (
Options.shallFollowStandardLibrary(),
"Requested to %srecurse to standard library." % (
"" if Options.shallFollowStandardLibrary() else "not "
)
)
if Options.shallFollowAllImports():
return (
True,
"Requested to recurse to all non-standard library modules."
)
# Means, we were not given instructions how to handle things.
return (
None,
"Default behaviour, not recursing without request."
)
def considerFilename(module_filename, module_package):
assert module_package is None or \
( type( module_package ) is str and module_package != "" )
module_filename = Utils.normpath( module_filename )
if Utils.isDir( module_filename ):
module_filename = Utils.abspath( module_filename )
module_name = Utils.basename( module_filename )
module_relpath = Utils.relpath( module_filename )
return module_filename, module_relpath, module_name
elif module_filename.endswith( ".py" ):
module_name = Utils.basename( module_filename )[:-3]
module_relpath = Utils.relpath( module_filename )
return module_filename, module_relpath, module_name
else:
return None
def isSameModulePath(path1, path2):
if Utils.basename(path1) == "__init__.py":
path1 = Utils.dirname(path1)
if Utils.basename(path2) == "__init__.py":
path2 = Utils.dirname(path2)
return Utils.abspath(path1) == Utils.abspath(path2)
def _checkPluginPath(plugin_filename, module_package):
debug(
"Checking detail plugin path %s %s",
plugin_filename,
module_package
)
plugin_info = considerFilename(
module_package = module_package,
module_filename = plugin_filename
)
if plugin_info is not None:
module, is_added = recurseTo(
module_filename = plugin_info[0],
module_relpath = plugin_info[1],
module_package = module_package,
module_kind = "py",
reason = "Lives in plugin directory."
)
if module:
if not is_added:
warning(
"Recursed to %s '%s' at '%s' twice.",
"package" if module.isPythonPackage() else "module",
module.getName(),
plugin_info[0]
)
if not isSameModulePath(module.getFilename(), plugin_info[0]):
warning(
"Duplicate ignored '%s'.",
plugin_info[1]
)
return
debug(
"Recursed to %s %s %s",
module.getName(),
module.getPackage(),
module
)
if module.isPythonPackage():
package_filename = module.getFilename()
if Utils.isDir(package_filename):
# Must be a namespace package.
assert Utils.python_version >= 330
package_dir = package_filename
# Only include it, if it contains actual modules, which will
# recurse to this one and find it again.
useful = False
else:
package_dir = Utils.dirname(package_filename)
# Real packages will always be included.
useful = True
debug(
"Package directory %s",
package_dir
)
for sub_path, sub_filename in Utils.listDir(package_dir):
if sub_filename in ("__init__.py", "__pycache__"):
continue
assert sub_path != plugin_filename
if Importing.isPackageDir(sub_path) or \
sub_path.endswith(".py"):
_checkPluginPath(sub_path, module.getFullName())
else:
# Modules should always be included.
useful = True
if useful:
ModuleRegistry.addRootModule(module)
else:
warning( "Failed to include module from '%s'.", plugin_info[0] )
def checkPluginPath(plugin_filename, module_package):
debug(
"Checking top level plugin path %s %s",
plugin_filename,
module_package
)
plugin_info = considerFilename(
module_package = module_package,
module_filename = plugin_filename
)
if plugin_info is not None:
# File or package makes a difference, handle that
if Utils.isFile(plugin_info[0]) or \
Importing.isPackageDir(plugin_info[0]):
_checkPluginPath(plugin_filename, module_package)
elif Utils.isDir(plugin_info[0]):
for sub_path, sub_filename in Utils.listDir(plugin_info[0]):
assert sub_filename != "__init__.py"
if Importing.isPackageDir(sub_path) or \
sub_path.endswith(".py"):
_checkPluginPath(sub_path, None)
else:
warning("Failed to include module from '%s'.", plugin_info[0])
else:
warning("Failed to recurse to directory '%s'.", plugin_filename)
Nuitka-0.5.0.1/nuitka/tree/ReformulationContractionExpressions.py 0000644 0001750 0001750 00000035334 12265264105 025454 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka import Utils
from nuitka.nodes.ParameterSpecs import ParameterSpec
from nuitka.nodes.VariableRefNodes import (
ExpressionTargetTempVariableRef,
ExpressionTempVariableRef,
ExpressionVariableRef
)
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.AssignNodes import (
StatementAssignmentVariable,
StatementDelVariable
)
from nuitka.nodes.StatementNodes import (
StatementExpressionOnly,
StatementsSequence,
StatementsFrame
)
from nuitka.nodes.FunctionNodes import (
ExpressionFunctionCreation,
ExpressionFunctionBody,
ExpressionFunctionCall,
ExpressionFunctionRef
)
from nuitka.nodes.LoopNodes import (
StatementBreakLoop,
StatementLoop
)
from nuitka.nodes.ConditionalNodes import StatementConditional
from nuitka.nodes.BuiltinIteratorNodes import (
ExpressionBuiltinNext1,
ExpressionBuiltinIter1
)
from nuitka.nodes.ContainerOperationNodes import (
ExpressionListOperationAppend,
ExpressionDictOperationSet,
ExpressionSetOperationAdd
)
from nuitka.nodes.ReturnNodes import StatementReturn
from nuitka.nodes.YieldNodes import ExpressionYield
make_contraction_parameters = ParameterSpec(
name = "contraction",
normal_args = ( "__iterator", ),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
)
from .ReformulationTryExceptStatements import makeTryExceptSingleHandlerNode
from .ReformulationAssignmentStatements import buildAssignmentStatements
from .ReformulationBooleanExpressions import buildAndNode
from .Helpers import (
makeStatementsSequenceFromStatement,
mergeStatements,
buildNodeList,
buildNode,
getKind
)
def buildListContractionNode(provider, node, source_ref):
# List contractions are dealt with by general code.
return _buildContractionNode(
provider = provider,
node = node,
name = "",
emit_class = ExpressionListOperationAppend,
start_value = ExpressionConstantRef(
constant = [],
source_ref = source_ref
),
# Note: For Python3, the list contractions no longer assign to the outer
# scope.
assign_provider = Utils.python_version < 300,
source_ref = source_ref
)
def buildSetContractionNode(provider, node, source_ref):
# Set contractions are dealt with by general code.
return _buildContractionNode(
provider = provider,
node = node,
name = "",
emit_class = ExpressionSetOperationAdd,
start_value = ExpressionConstantRef(
constant = set(),
source_ref = source_ref
),
assign_provider = False,
source_ref = source_ref
)
def buildDictContractionNode(provider, node, source_ref):
# Dict contractions are dealt with by general code.
return _buildContractionNode(
provider = provider,
node = node,
name = "",
emit_class = ExpressionDictOperationSet,
start_value = ExpressionConstantRef(
constant = {},
source_ref = source_ref
),
assign_provider = False,
source_ref = source_ref
)
def buildGeneratorExpressionNode(provider, node, source_ref):
# Generator expressions are dealt with by general code.
assert getKind( node ) == "GeneratorExp"
return _buildContractionNode(
provider = provider,
node = node,
name = "",
emit_class = ExpressionYield,
start_value = None,
assign_provider = False,
source_ref = source_ref
)
def _buildContractionNode(provider, node, name, emit_class, start_value,
assign_provider, source_ref):
# The contraction nodes are reformulated to function bodies, with loops as
# described in the developer manual. They use a lot of temporary names,
# nested blocks, etc. and so a lot of variable names. There is no good way
# around that, and we deal with many cases, due to having generator
# expressions sharing this code, pylint: disable=R0912,R0914
# Note: The assign_provider is only to cover Python2 list contractions,
# assigning one of the loop variables to the outside scope.
assert provider.isParentVariableProvider(), provider
function_body = ExpressionFunctionBody(
provider = provider,
name = name,
doc = None,
parameters = make_contraction_parameters,
source_ref = source_ref
)
if start_value is not None:
container_tmp = function_body.allocateTempVariable( None, "result" )
statements = [
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = container_tmp.makeReference( function_body ),
source_ref = source_ref
),
source = start_value,
source_ref = source_ref.atInternal()
)
]
else:
statements = []
if hasattr( node, "elt" ):
if start_value is not None:
current_body = emit_class(
ExpressionTempVariableRef(
variable = container_tmp.makeReference( function_body ),
source_ref = source_ref
),
buildNode(
provider = function_body,
node = node.elt,
source_ref = source_ref
),
source_ref = source_ref
)
else:
assert emit_class is ExpressionYield
function_body.markAsGenerator()
current_body = emit_class(
buildNode(
provider = function_body,
node = node.elt,
source_ref = source_ref
),
source_ref = source_ref
)
else:
assert emit_class is ExpressionDictOperationSet
current_body = emit_class(
ExpressionTempVariableRef(
variable = container_tmp.makeReference( function_body ),
source_ref = source_ref
),
key = buildNode(
provider = function_body,
node = node.key,
source_ref = source_ref,
),
value = buildNode(
provider = function_body,
node = node.value,
source_ref = source_ref,
),
source_ref = source_ref
)
current_body = StatementExpressionOnly(
expression = current_body,
source_ref = source_ref
)
for count, qual in enumerate(reversed( node.generators)):
tmp_iter_variable = function_body.allocateTempVariable(
temp_scope = None,
name = "contraction_iter_%d" % count
)
tmp_value_variable = function_body.allocateTempVariable(
temp_scope = None,
name = "iter_value_%d" % count
)
# The first iterated value is to be calculated outside of the function
# and will be given as a parameter "_iterated", the others are built
# inside the function.
if qual is node.generators[0]:
value_iterator = ExpressionVariableRef(
variable_name = "__iterator",
source_ref = source_ref
)
else:
value_iterator = ExpressionBuiltinIter1(
value = buildNode(
provider = function_body,
node = qual.iter,
source_ref = source_ref
),
source_ref = source_ref
)
# First create the iterator and store it, next should be loop body
nested_statements = [
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_iter_variable.makeReference(
function_body
),
source_ref = source_ref
),
source = value_iterator,
source_ref = source_ref
)
]
loop_statements = [
makeTryExceptSingleHandlerNode(
tried = makeStatementsSequenceFromStatement(
statement = StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_value_variable.makeReference(
function_body
),
source_ref = source_ref
),
source = ExpressionBuiltinNext1(
value = ExpressionTempVariableRef(
variable = tmp_iter_variable.makeReference(
function_body
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
),
exception_name = "StopIteration",
handler_body = makeStatementsSequenceFromStatement(
statement = StatementBreakLoop(
source_ref = source_ref.atInternal()
)
),
source_ref = source_ref
),
buildAssignmentStatements(
provider = provider if assign_provider else function_body,
temp_provider = function_body,
node = qual.target,
source = ExpressionTempVariableRef(
variable = tmp_value_variable.makeReference(
function_body
),
source_ref = source_ref
),
source_ref = source_ref
)
]
conditions = buildNodeList(
provider = function_body,
nodes = qual.ifs,
source_ref = source_ref
)
if len( conditions ) == 1:
loop_statements.append(
StatementConditional(
condition = conditions[0],
yes_branch = makeStatementsSequenceFromStatement(
statement = current_body
),
no_branch = None,
source_ref = source_ref
)
)
elif len( conditions ) > 1:
loop_statements.append(
StatementConditional(
condition = buildAndNode(
provider = function_body,
values = conditions,
source_ref = source_ref
),
yes_branch = makeStatementsSequenceFromStatement(
statement = current_body
),
no_branch = None,
source_ref = source_ref
)
)
else:
loop_statements.append(current_body)
nested_statements.append(
StatementLoop(
body = StatementsSequence(
statements = mergeStatements(loop_statements),
source_ref = source_ref
),
source_ref = source_ref
)
)
nested_statements.append(
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_iter_variable.makeReference(
function_body
),
source_ref = source_ref
),
tolerant = False,
source_ref = source_ref
)
)
current_body = StatementsSequence(
statements = nested_statements,
source_ref = source_ref
)
statements.append(current_body)
if start_value is not None:
statements.append(
StatementReturn(
expression = ExpressionTempVariableRef(
variable = container_tmp.makeReference( function_body ),
source_ref = source_ref
),
source_ref = source_ref
)
)
function_body.setBody(
StatementsFrame(
statements = mergeStatements(statements),
guard_mode = "pass_through"
if emit_class is not ExpressionYield else
"generator",
var_names = (),
arg_count = 0,
kw_only_count = 0,
has_starlist = False,
has_stardict = False,
code_name = "contraction",
source_ref = source_ref
)
)
return ExpressionFunctionCall(
function = ExpressionFunctionCreation(
function_ref = ExpressionFunctionRef(
function_body = function_body,
source_ref = source_ref
),
defaults = (),
kw_defaults = None,
annotations = None,
source_ref = source_ref
),
values = (
ExpressionBuiltinIter1(
value = buildNode(
provider = provider,
node = node.generators[0].iter,
source_ref = source_ref
),
source_ref = source_ref
),
),
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/tree/ReformulationLambdaExpressions.py 0000644 0001750 0001750 00000013774 12265264105 024355 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka import Utils
from nuitka.nodes.VariableRefNodes import (
ExpressionTargetTempVariableRef,
ExpressionTempVariableRef
)
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.FunctionNodes import (
ExpressionFunctionCreation,
ExpressionFunctionBody,
ExpressionFunctionRef
)
from nuitka.nodes.StatementNodes import (
StatementExpressionOnly,
StatementsSequence,
StatementsFrame
)
from nuitka.nodes.ComparisonNodes import ExpressionComparisonIsNOT
from nuitka.nodes.ConditionalNodes import StatementConditional
from nuitka.nodes.YieldNodes import ExpressionYield
from nuitka.nodes.ReturnNodes import StatementReturn
from nuitka.nodes.AssignNodes import StatementAssignmentVariable
from .ReformulationFunctionStatements import (
buildParameterAnnotations,
buildParameterKwDefaults,
buildParameterSpec
)
from .Helpers import (
makeStatementsSequenceFromStatement,
mergeStatements,
buildNodeList,
buildNode,
getKind
)
def buildLambdaNode(provider, node, source_ref):
assert getKind( node ) == "Lambda"
parameters = buildParameterSpec( "", node, source_ref )
function_body = ExpressionFunctionBody(
provider = provider,
name = "",
doc = None,
parameters = parameters,
source_ref = source_ref,
)
defaults = buildNodeList(provider, node.args.defaults, source_ref)
kw_defaults = buildParameterKwDefaults(
provider = provider,
node = node,
function_body = function_body,
source_ref = source_ref
)
body = buildNode(
provider = function_body,
node = node.body,
source_ref = source_ref,
)
if function_body.isGenerator():
if Utils.python_version < 270:
tmp_return_value = function_body.allocateTempVariable(
temp_scope = None,
name = "yield_return"
)
statements = (
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_return_value.makeReference(
function_body
),
source_ref = source_ref,
),
source = body,
source_ref = source_ref
),
StatementConditional(
condition = ExpressionComparisonIsNOT(
left = ExpressionTempVariableRef(
variable = tmp_return_value.makeReference(
function_body
),
source_ref = source_ref,
),
right = ExpressionConstantRef(
constant = None,
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = makeStatementsSequenceFromStatement(
statement = StatementExpressionOnly(
expression = ExpressionYield(
expression = ExpressionTempVariableRef(
variable = tmp_return_value.makeReference(
function_body
),
source_ref = source_ref,
),
source_ref = source_ref
),
source_ref = source_ref
)
),
no_branch = None,
source_ref = source_ref
)
)
body = StatementsSequence(
statements = statements,
source_ref = source_ref
)
else:
body = StatementExpressionOnly(
expression = body,
source_ref = source_ref
)
else:
body = StatementReturn(
expression = body,
source_ref = source_ref
)
body = StatementsFrame(
statements = mergeStatements(
(body,)
),
guard_mode = "generator" if function_body.isGenerator() else "full",
var_names = parameters.getCoArgNames(),
arg_count = parameters.getArgumentCount(),
kw_only_count = parameters.getKwOnlyParameterCount(),
has_starlist = parameters.getStarListArgumentName() is not None,
has_stardict = parameters.getStarDictArgumentName() is not None,
code_name = "",
source_ref = body.getSourceReference()
)
function_body.setBody(body)
annotations = buildParameterAnnotations(provider, node, source_ref)
return ExpressionFunctionCreation(
function_ref = ExpressionFunctionRef(
function_body = function_body,
source_ref = source_ref
),
defaults = defaults,
kw_defaults = kw_defaults,
annotations = annotations,
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/tree/ReformulationPrintStatements.py 0000644 0001750 0001750 00000003376 12265264105 024073 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka.nodes.BuiltinTypeNodes import ExpressionBuiltinStr
from nuitka.nodes.PrintNodes import StatementPrint
from .Helpers import (
buildNodeList,
buildNode
)
def buildPrintNode(provider, node, source_ref):
# "print" statements, should only occur with Python2.
values = buildNodeList( provider, node.values, source_ref )
def wrapValue(value):
if value.isExpressionConstantRef():
return value.getStrValue()
else:
return ExpressionBuiltinStr(
value = value,
source_ref = value.getSourceReference()
)
values = [
wrapValue( value )
for value in
values
]
return StatementPrint(
newline = node.nl,
dest = buildNode(
provider = provider,
node = node.dest,
source_ref = source_ref,
allow_none = True
),
values = values,
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/tree/ReformulationBooleanExpressions.py 0000644 0001750 0001750 00000007101 12265264105 024537 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka.nodes.OperatorNodes import ExpressionOperationNOT
from nuitka.nodes.ConditionalNodes import ExpressionConditional
from nuitka.nodes.KeeperNodes import (
ExpressionAssignmentTempKeeper,
ExpressionTempKeeperRef
)
from .Helpers import (
buildNodeList,
buildNode,
getKind
)
def buildBoolOpNode(provider, node, source_ref):
bool_op = getKind( node.op )
if bool_op == "Or":
# The "or" may be short circuit and is therefore not a plain operation
return buildOrNode(
provider = provider,
values = buildNodeList( provider, node.values, source_ref ),
source_ref = source_ref
)
elif bool_op == "And":
# The "and" may be short circuit and is therefore not a plain operation
return buildAndNode(
provider = provider,
values = buildNodeList( provider, node.values, source_ref ),
source_ref = source_ref
)
elif bool_op == "Not":
# The "not" is really only a unary operation and no special.
return ExpressionOperationNOT(
operand = buildNode( provider, node.operand, source_ref ),
source_ref = source_ref
)
else:
assert False, bool_op
def buildOrNode(provider, values, source_ref):
result = values[ -1 ]
del values[ -1 ]
while True:
keeper_variable = provider.allocateTempKeeperVariable()
tmp_assign = ExpressionAssignmentTempKeeper(
variable = keeper_variable.makeReference( provider ),
source = values[ -1 ],
source_ref = source_ref
)
del values[ -1 ]
result = ExpressionConditional(
condition = tmp_assign,
yes_expression = ExpressionTempKeeperRef(
variable = keeper_variable.makeReference( provider ),
source_ref = source_ref
),
no_expression = result,
source_ref = source_ref
)
if not values:
break
return result
def buildAndNode(provider, values, source_ref):
result = values[ -1 ]
del values[ -1 ]
while values:
keeper_variable = provider.allocateTempKeeperVariable()
tmp_assign = ExpressionAssignmentTempKeeper(
variable = keeper_variable.makeReference( provider ),
source = values[ -1 ],
source_ref = source_ref
)
del values[ -1 ]
result = ExpressionConditional(
condition = tmp_assign,
no_expression = ExpressionTempKeeperRef(
variable = keeper_variable.makeReference( provider ),
source_ref = source_ref
),
yes_expression = result,
source_ref = source_ref
)
return result
Nuitka-0.5.0.1/nuitka/tree/ReformulationCallExpressions.py 0000644 0001750 0001750 00000014731 12265264105 024042 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.CallNodes import ExpressionCall
from nuitka.nodes.FunctionNodes import (
ExpressionFunctionCreation,
ExpressionFunctionCall,
ExpressionFunctionRef
)
from .Helpers import (
makeSequenceCreationOrConstant,
makeDictCreationOrConstant,
buildNodeList,
buildNode
)
def buildCallNode(provider, node, source_ref):
positional_args = buildNodeList( provider, node.args, source_ref )
# Only the values of keyword pairs have a real source ref, and those only
# really matter, so that makes sense.
keys = []
values = []
for keyword in node.keywords:
keys.append(
ExpressionConstantRef(
constant = keyword.arg,
source_ref = source_ref,
user_provided = True
)
)
values.append(
buildNode( provider, keyword.value, source_ref )
)
list_star_arg = buildNode( provider, node.starargs, source_ref, True )
dict_star_arg = buildNode( provider, node.kwargs, source_ref, True )
return _makeCallNode(
called = buildNode( provider, node.func, source_ref ),
positional_args = positional_args,
keys = keys,
values = values,
list_star_arg = list_star_arg,
dict_star_arg = dict_star_arg,
source_ref = source_ref,
)
def _makeCallNode( called, positional_args, keys, values, list_star_arg,
dict_star_arg, source_ref ):
# Many variables, but only to cover the many complex call cases.
# pylint: disable=R0914
if list_star_arg is None and dict_star_arg is None:
return ExpressionCall(
called = called,
args = makeSequenceCreationOrConstant(
sequence_kind = "tuple",
elements = positional_args,
source_ref = source_ref
),
kw = makeDictCreationOrConstant(
keys = keys,
values = values,
lazy_order = True,
source_ref = source_ref
),
source_ref = source_ref,
)
else:
# Dispatch to complex helper function for each case. These do
# re-formulation of complex calls according to developer manual.
key = (
len( positional_args ) > 0,
len( keys ) > 0,
list_star_arg is not None,
dict_star_arg is not None
)
from .ComplexCallHelperFunctions import (
getFunctionCallHelperPosKeywordsStarList,
getFunctionCallHelperPosStarList,
getFunctionCallHelperKeywordsStarList,
getFunctionCallHelperStarList,
getFunctionCallHelperPosKeywordsStarDict,
getFunctionCallHelperPosStarDict,
getFunctionCallHelperKeywordsStarDict,
getFunctionCallHelperStarDict,
getFunctionCallHelperPosKeywordsStarListStarDict,
getFunctionCallHelperPosStarListStarDict,
getFunctionCallHelperKeywordsStarListStarDict,
getFunctionCallHelperStarListStarDict,
)
table = {
( True, True, True, False ) :
getFunctionCallHelperPosKeywordsStarList,
( True, False, True, False ) :
getFunctionCallHelperPosStarList,
( False, True, True, False ) :
getFunctionCallHelperKeywordsStarList,
( False, False, True, False ) :
getFunctionCallHelperStarList,
( True, True, False, True ) :
getFunctionCallHelperPosKeywordsStarDict,
( True, False, False, True ) :
getFunctionCallHelperPosStarDict,
( False, True, False, True ) :
getFunctionCallHelperKeywordsStarDict,
( False, False, False, True ) :
getFunctionCallHelperStarDict,
( True, True, True, True ) :
getFunctionCallHelperPosKeywordsStarListStarDict,
( True, False, True, True ) :
getFunctionCallHelperPosStarListStarDict,
( False, True, True, True ) :
getFunctionCallHelperKeywordsStarListStarDict,
( False, False, True, True ) :
getFunctionCallHelperStarListStarDict,
}
get_helper = table[ key ]
helper_args = [ called ]
if positional_args:
helper_args.append(
makeSequenceCreationOrConstant(
sequence_kind = "tuple",
elements = positional_args,
source_ref = source_ref
)
)
if keys:
helper_args.append(
makeDictCreationOrConstant(
keys = keys,
values = values,
lazy_order = True,
source_ref = source_ref
)
)
if list_star_arg is not None:
helper_args.append( list_star_arg )
if dict_star_arg is not None:
helper_args.append( dict_star_arg )
return ExpressionFunctionCall(
function = ExpressionFunctionCreation(
function_ref = ExpressionFunctionRef(
function_body = get_helper(),
source_ref = source_ref
),
defaults = (),
kw_defaults = None,
annotations = None,
source_ref = source_ref
),
values = helper_args,
source_ref = source_ref,
)
Nuitka-0.5.0.1/nuitka/tree/ReformulationExecStatements.py 0000644 0001750 0001750 00000016543 12265264105 023663 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Reformulation of exec statements
Consult the developmer manual for information. TODO: Add ability to sync
source code comments with developer manual sections.
"""
from nuitka.nodes.ExceptionNodes import StatementRaiseException
from nuitka.nodes.BuiltinRefNodes import ExpressionBuiltinExceptionRef
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.ExecEvalNodes import StatementExec
from nuitka.nodes.ConditionalNodes import ExpressionConditional
from nuitka.nodes.KeeperNodes import (
ExpressionAssignmentTempKeeper,
ExpressionTempKeeperRef
)
from nuitka.nodes.GlobalsLocalsNodes import (
ExpressionBuiltinGlobals,
ExpressionBuiltinLocals,
)
from nuitka.nodes.ComparisonNodes import ExpressionComparisonIs
from .Helpers import (
buildNode,
getKind
)
def wrapEvalGlobalsAndLocals( provider, globals_node, locals_node, exec_mode,
source_ref ):
""" Wrap the locals and globals arguments for eval and exec.
For eval, this is called from the outside, and when the node tree
already exists.
"""
if globals_node is not None:
global_keeper_variable = provider.allocateTempKeeperVariable()
tmp_global_assign = ExpressionAssignmentTempKeeper(
variable = global_keeper_variable.makeReference( provider ),
source = globals_node,
source_ref = source_ref
)
globals_wrap = ExpressionConditional(
condition = ExpressionComparisonIs(
left = tmp_global_assign,
right = ExpressionConstantRef(
constant = None,
source_ref = source_ref
),
source_ref = source_ref
),
no_expression = ExpressionTempKeeperRef(
variable = global_keeper_variable.makeReference(
provider
),
source_ref = source_ref
),
yes_expression = ExpressionBuiltinGlobals(
source_ref = source_ref
),
source_ref = source_ref
)
else:
globals_wrap = ExpressionBuiltinGlobals(
source_ref = source_ref
)
if locals_node is not None:
local_keeper_variable = provider.allocateTempKeeperVariable()
tmp_local_assign = ExpressionAssignmentTempKeeper(
variable = local_keeper_variable.makeReference( provider ),
source = locals_node,
source_ref = source_ref
)
if exec_mode:
locals_fallback = ExpressionBuiltinLocals(
source_ref = source_ref
)
else:
locals_fallback = ExpressionConditional(
condition = ExpressionComparisonIs(
left = ExpressionTempKeeperRef(
variable = global_keeper_variable.makeReference(
provider
),
source_ref = source_ref
),
right = ExpressionConstantRef(
constant = None,
source_ref = source_ref
),
source_ref = source_ref
),
no_expression = ExpressionTempKeeperRef(
variable = global_keeper_variable.makeReference(
provider
),
source_ref = source_ref
),
yes_expression = ExpressionBuiltinLocals(
source_ref = source_ref
),
source_ref = source_ref
)
locals_wrap = ExpressionConditional(
condition = ExpressionComparisonIs(
left = tmp_local_assign,
right = ExpressionConstantRef(
constant = None,
source_ref = source_ref
),
source_ref = source_ref
),
no_expression = ExpressionTempKeeperRef(
variable = local_keeper_variable.makeReference(
provider
),
source_ref = source_ref
),
yes_expression = locals_fallback,
source_ref = source_ref
)
else:
if globals_node is None:
locals_wrap = ExpressionBuiltinLocals(
source_ref = source_ref
)
else:
locals_wrap = ExpressionTempKeeperRef(
variable = global_keeper_variable.makeReference(
provider
),
source_ref = source_ref
)
return globals_wrap, locals_wrap
def buildExecNode(provider, node, source_ref):
# "exec" statements, should only occur with Python2.
exec_globals = node.globals
exec_locals = node.locals
body = node.body
orig_globals = exec_globals
# Handle exec(a,b,c) to be same as exec a, b, c
if exec_locals is None and exec_globals is None and \
getKind( body ) == "Tuple":
parts = body.elts
body = parts[0]
if len( parts ) > 1:
exec_globals = parts[1]
if len( parts ) > 2:
exec_locals = parts[2]
else:
return StatementRaiseException(
exception_type = ExpressionBuiltinExceptionRef(
exception_name = "TypeError",
source_ref = source_ref
),
exception_value = ExpressionConstantRef(
constant = "exec: arg 1 must be a string, file, or code object",
source_ref = source_ref
),
exception_trace = None,
exception_cause = None,
source_ref = source_ref
)
if provider.isExpressionFunctionBody():
provider.markAsExecContaining()
if orig_globals is None:
provider.markAsUnqualifiedExecContaining( source_ref )
globals_wrap, locals_wrap = wrapEvalGlobalsAndLocals(
provider = provider,
globals_node = buildNode( provider, exec_globals, source_ref, True ),
locals_node = buildNode( provider, exec_locals, source_ref, True ),
exec_mode = True,
source_ref = source_ref
)
return StatementExec(
source_code = buildNode( provider, body, source_ref ),
globals_arg = globals_wrap,
locals_arg = locals_wrap,
source_ref = source_ref
)
# This is here, to make sure it can register, pylint: disable=W0611
import nuitka.optimizations.OptimizeBuiltinCalls
Nuitka-0.5.0.1/nuitka/tree/Extractions.py 0000644 0001750 0001750 00000003112 12265264105 020447 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Extracting visitors.
This is used for look-aheads supporting abstract execution. We need to e.g.
know the variables written by a piece of code ahead of abstractly executing a
loop.
"""
from .Operations import VisitorNoopMixin, visitTree
class VariableWriteExtractor(VisitorNoopMixin):
""" Extract variables written to.
"""
def __init__(self):
self.written_to = set()
def onEnterNode(self, node):
if node.isExpressionTargetVariableRef() or \
node.isExpressionAssignmentTempKeeper():
key = node.getVariable(), node.getVariableVersion()
self.written_to.add( key )
def getResult(self):
return self.written_to
def getVariablesWritten(node):
visitor = VariableWriteExtractor()
visitTree( node, visitor )
return visitor.getResult()
Nuitka-0.5.0.1/nuitka/tree/ReformulationLoopStatements.py 0000644 0001750 0001750 00000030320 12265264105 023675 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka.nodes.VariableRefNodes import (
ExpressionTargetTempVariableRef,
ExpressionTempVariableRef
)
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.BuiltinIteratorNodes import (
ExpressionBuiltinNext1,
ExpressionBuiltinIter1
)
from nuitka.nodes.ComparisonNodes import ExpressionComparisonIs
from nuitka.nodes.StatementNodes import StatementsSequence
from nuitka.nodes.LoopNodes import (
StatementBreakLoop,
StatementLoop
)
from nuitka.nodes.ConditionalNodes import StatementConditional
from nuitka.nodes.TryNodes import StatementTryFinally
from nuitka.nodes.AssignNodes import (
StatementAssignmentVariable,
StatementDelVariable
)
from .Helpers import (
makeStatementsSequenceFromStatement,
makeStatementsSequence,
buildStatementsNode,
buildNode
)
from .ReformulationAssignmentStatements import buildAssignmentStatements
from .ReformulationTryExceptStatements import makeTryExceptSingleHandlerNode
def buildForLoopNode(provider, node, source_ref):
# The for loop is re-formulated according to developer manual. An iterator
# is created, and looped until it gives StopIteration. The else block is
# taken if a for loop exits normally, i.e. because of iterator
# exhaustion. We do this by introducing an indicator variable.
source = buildNode( provider, node.iter, source_ref )
temp_scope = provider.allocateTempScope( "for_loop" )
tmp_iter_variable = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "for_iterator"
)
tmp_value_variable = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "iter_value"
)
else_block = buildStatementsNode(
provider = provider,
nodes = node.orelse if node.orelse else None,
source_ref = source_ref
)
if else_block is not None:
tmp_break_indicator_variable = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "break_indicator"
)
statements = [
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_break_indicator_variable.makeReference(
provider
),
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = True,
source_ref = source_ref
),
source_ref = source_ref
)
]
else:
statements = []
statements.append(
StatementBreakLoop(
source_ref = source_ref.atInternal()
)
)
handler_body = makeStatementsSequence(
statements = statements,
allow_none = False,
source_ref = source_ref
)
statements = (
makeTryExceptSingleHandlerNode(
tried = makeStatementsSequenceFromStatement(
statement = StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_value_variable.makeReference(
provider
),
source_ref = source_ref
),
source = ExpressionBuiltinNext1(
value = ExpressionTempVariableRef(
variable = tmp_iter_variable.makeReference(
provider
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
),
exception_name = "StopIteration",
handler_body = handler_body,
source_ref = source_ref
),
buildAssignmentStatements(
provider = provider,
node = node.target,
source = ExpressionTempVariableRef(
variable = tmp_value_variable.makeReference(
provider
),
source_ref = source_ref
),
source_ref = source_ref
)
)
statements += (
buildStatementsNode(
provider = provider,
nodes = node.body,
source_ref = source_ref
),
)
loop_body = makeStatementsSequence(
statements = statements,
allow_none = True,
source_ref = source_ref
)
if else_block is not None:
statements = [
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_break_indicator_variable.makeReference(
provider
),
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = False,
source_ref = source_ref
),
source_ref = source_ref
)
]
else:
statements = []
cleanup_statements = (
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_value_variable.makeReference( provider ),
source_ref = source_ref
),
tolerant = True,
source_ref = source_ref.atInternal()
),
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_iter_variable.makeReference( provider ),
source_ref = source_ref
),
tolerant = False,
source_ref = source_ref.atInternal()
)
)
statements += [
# First create the iterator and store it.
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_iter_variable.makeReference(
provider
),
source_ref = source_ref
),
source = ExpressionBuiltinIter1(
value = source,
source_ref = source.getSourceReference()
),
source_ref = source_ref
),
StatementTryFinally(
tried = makeStatementsSequenceFromStatement(
statement = StatementLoop(
body = loop_body,
source_ref = source_ref
)
),
final = StatementsSequence(
statements = cleanup_statements,
source_ref = source_ref.atInternal()
),
source_ref = source_ref.atInternal()
)
]
if else_block is not None:
statements += [
StatementConditional(
condition = ExpressionComparisonIs(
left = ExpressionTempVariableRef(
variable = tmp_break_indicator_variable.makeReference( provider ),
source_ref = source_ref
),
right = ExpressionConstantRef(
constant = True,
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = else_block,
no_branch = None,
source_ref = source_ref
)
]
return StatementsSequence(
statements = statements,
source_ref = source_ref
)
def buildWhileLoopNode(provider, node, source_ref):
# The while loop is re-formulated according to developer manual. The
# condition becomes an early condition to break the loop. The else block is
# taken if a while loop exits normally, i.e. because of condition not being
# true. We do this by introducing an indicator variable.
else_block = buildStatementsNode(
provider = provider,
nodes = node.orelse if node.orelse else None,
source_ref = source_ref
)
if else_block is not None:
temp_scope = provider.allocateTempScope( "while_loop" )
tmp_break_indicator_variable = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "break_indicator"
)
statements = (
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_break_indicator_variable.makeReference(
provider
),
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = True,
source_ref = source_ref
),
source_ref = source_ref
),
StatementBreakLoop(
source_ref = source_ref
)
)
else:
statements = (
StatementBreakLoop(
source_ref = source_ref
),
)
# The loop body contains a conditional statement at the start that breaks
# the loop if it fails.
loop_body = makeStatementsSequence(
statements = (
StatementConditional(
condition = buildNode( provider, node.test, source_ref ),
no_branch = StatementsSequence(
statements = statements,
source_ref = source_ref
),
yes_branch = None,
source_ref = source_ref
),
buildStatementsNode(
provider = provider,
nodes = node.body,
source_ref = source_ref
)
),
allow_none = True,
source_ref = source_ref
)
loop_statement = StatementLoop(
body = loop_body,
source_ref = source_ref
)
if else_block is None:
return loop_statement
else:
statements = (
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_break_indicator_variable.makeReference(
provider
),
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = False,
source_ref = source_ref
),
source_ref = source_ref
),
loop_statement,
StatementConditional(
condition = ExpressionComparisonIs(
left = ExpressionTempVariableRef(
variable = tmp_break_indicator_variable.makeReference(
provider
),
source_ref = source_ref
),
right = ExpressionConstantRef(
constant = True,
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = else_block,
no_branch = None,
source_ref = source_ref
)
)
return StatementsSequence(
statements = statements,
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/tree/ReformulationWithStatements.py 0000644 0001750 0001750 00000030432 12265264105 023703 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka import Utils
from nuitka.nodes.VariableRefNodes import (
ExpressionTargetTempVariableRef,
ExpressionTempVariableRef
)
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.ContainerMakingNodes import ExpressionMakeTuple
from nuitka.nodes.ExceptionNodes import (
ExpressionCaughtExceptionTracebackRef,
ExpressionCaughtExceptionValueRef,
ExpressionCaughtExceptionTypeRef,
StatementRaiseException
)
from nuitka.nodes.CallNodes import (
ExpressionCallNoKeywords,
ExpressionCallEmpty
)
from nuitka.nodes.AttributeNodes import (
ExpressionSpecialAttributeLookup,
ExpressionAttributeLookup
)
from nuitka.nodes.StatementNodes import (
StatementExpressionOnly,
StatementsSequence
)
from nuitka.nodes.ConditionalNodes import StatementConditional
from nuitka.nodes.ComparisonNodes import ExpressionComparisonIs
from nuitka.nodes.AssignNodes import StatementAssignmentVariable
from nuitka.nodes.TryNodes import StatementTryFinally
from .ReformulationTryExceptStatements import makeTryExceptSingleHandlerNode
from .ReformulationAssignmentStatements import buildAssignmentStatements
from .Helpers import (
makeStatementsSequenceFromStatement,
makeStatementsSequence,
buildStatementsNode,
buildNode
)
def _buildWithNode(provider, context_expr, assign_target, body, source_ref):
with_source = buildNode( provider, context_expr, source_ref )
temp_scope = provider.allocateTempScope( "with" )
tmp_source_variable = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "source"
)
tmp_exit_variable = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "exit"
)
tmp_enter_variable = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "enter"
)
tmp_indicator_variable = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "indicator"
)
statements = (
buildAssignmentStatements(
provider = provider,
node = assign_target,
allow_none = True,
source = ExpressionTempVariableRef(
variable = tmp_enter_variable.makeReference( provider ),
source_ref = source_ref
),
source_ref = source_ref
),
body
)
with_body = makeStatementsSequence(
statements = statements,
allow_none = True,
source_ref = source_ref
)
# The "__enter__" and "__exit__" were normal attribute lookups under
# CPython2.6, but that changed with CPython2.7.
if Utils.python_version < 270:
attribute_lookup_class = ExpressionAttributeLookup
else:
attribute_lookup_class = ExpressionSpecialAttributeLookup
statements = [
# First assign the with context to a temporary variable.
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_source_variable.makeReference( provider ),
source_ref = source_ref
),
source = with_source,
source_ref = source_ref
),
# Next, assign "__enter__" and "__exit__" attributes to temporary
# variables.
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_exit_variable.makeReference( provider ),
source_ref = source_ref
),
source = attribute_lookup_class(
expression = ExpressionTempVariableRef(
variable = tmp_source_variable.makeReference( provider ),
source_ref = source_ref
),
attribute_name = "__exit__",
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_enter_variable.makeReference( provider ),
source_ref = source_ref
),
source = ExpressionCallEmpty(
called = attribute_lookup_class(
expression = ExpressionTempVariableRef(
variable = tmp_source_variable.makeReference(
provider
),
source_ref = source_ref
),
attribute_name = "__enter__",
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_indicator_variable.makeReference( provider ),
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = True,
source_ref = source_ref
),
source_ref = source_ref
),
]
source_ref = source_ref.atInternal()
statements += [
StatementTryFinally(
tried = makeStatementsSequenceFromStatement(
statement = makeTryExceptSingleHandlerNode(
tried = with_body,
exception_name = "BaseException",
handler_body = StatementsSequence(
statements = (
# Prevents final block from calling __exit__ as
# well.
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_indicator_variable.makeReference( provider ),
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = False,
source_ref = source_ref
),
source_ref = source_ref
),
StatementConditional(
condition = ExpressionCallNoKeywords(
called = ExpressionTempVariableRef(
variable = tmp_exit_variable.makeReference( provider ),
source_ref = source_ref
),
args = ExpressionMakeTuple(
elements = (
ExpressionCaughtExceptionTypeRef(
source_ref = source_ref
),
ExpressionCaughtExceptionValueRef(
source_ref = source_ref
),
ExpressionCaughtExceptionTracebackRef(
source_ref = source_ref
),
),
source_ref = source_ref
),
source_ref = source_ref
),
no_branch = makeStatementsSequenceFromStatement(
statement = StatementRaiseException(
exception_type = None,
exception_value = None,
exception_trace = None,
exception_cause = None,
source_ref = source_ref
)
),
yes_branch = None,
source_ref = source_ref
),
),
source_ref = source_ref
),
source_ref = source_ref
),
),
final = makeStatementsSequenceFromStatement(
statement = StatementConditional(
condition = ExpressionComparisonIs(
left = ExpressionTempVariableRef(
variable = tmp_indicator_variable.makeReference(
provider
),
source_ref = source_ref
),
right = ExpressionConstantRef(
constant = True,
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = makeStatementsSequenceFromStatement(
statement = StatementExpressionOnly(
expression = ExpressionCallNoKeywords(
called = ExpressionTempVariableRef(
variable = tmp_exit_variable.makeReference( provider ),
source_ref = source_ref
),
args = ExpressionConstantRef(
constant = ( None, None, None ),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
),
no_branch = None,
source_ref = source_ref
)
),
source_ref = source_ref
)
]
return StatementsSequence(
statements = statements,
source_ref = source_ref
)
def buildWithNode(provider, node, source_ref):
# "with" statements are re-formulated as described in the developer
# manual. Catches exceptions, and provides them to "__exit__", while making
# the "__enter__" value available under a given name.
# Before Python3.3, multiple context managers are not visible in the parse
# tree, now we need to handle it ourselves.
if hasattr( node, "items" ):
context_exprs = [ item.context_expr for item in node.items ]
assign_targets = [ item.optional_vars for item in node.items ]
else:
# Make it a list for before Python3.3
context_exprs = [ node.context_expr ]
assign_targets = [ node.optional_vars ]
# The body for the first context manager is the other things.
body = buildStatementsNode( provider, node.body, source_ref )
assert len( context_exprs ) > 0 and len( context_exprs ) == len( assign_targets )
for context_expr, assign_target in zip( reversed( context_exprs ), reversed( assign_targets ) ):
body = _buildWithNode(
provider = provider,
body = body,
context_expr = context_expr,
assign_target = assign_target,
source_ref = source_ref
)
return body
Nuitka-0.5.0.1/nuitka/tree/ReformulationYieldExpressions.py 0000644 0001750 0001750 00000004376 12265264105 024241 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka.nodes.YieldNodes import ExpressionYield, ExpressionYieldFrom
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.BuiltinIteratorNodes import ExpressionBuiltinIter1
from nuitka import Utils, SyntaxErrors
from .Helpers import buildNode
def _markAsGenerator(provider, node, source_ref):
if provider.isPythonModule():
SyntaxErrors.raiseSyntaxError(
"'yield' outside function",
source_ref,
None if Utils.python_version < 300 else node.col_offset
)
provider.markAsGenerator()
def buildYieldNode(provider, node, source_ref):
_markAsGenerator( provider, node, source_ref )
if node.value is not None:
return ExpressionYield(
expression = buildNode( provider, node.value, source_ref ),
source_ref = source_ref
)
else:
return ExpressionYield(
expression = ExpressionConstantRef(
constant = None,
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
)
def buildYieldFromNode(provider, node, source_ref):
assert Utils.python_version >= 330
_markAsGenerator( provider, node, source_ref )
iter_arg = ExpressionBuiltinIter1(
value = buildNode( provider, node.value, source_ref ),
source_ref = source_ref
)
return ExpressionYieldFrom(
expression = iter_arg,
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/tree/ReformulationTryExceptStatements.py 0000644 0001750 0001750 00000022352 12265264105 024721 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka import Utils, SyntaxErrors, Options
from nuitka.nodes.VariableRefNodes import (
ExpressionTargetTempVariableRef,
ExpressionTempVariableRef
)
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.ExceptionNodes import ExpressionCaughtExceptionValueRef
from nuitka.nodes.BuiltinRefNodes import ExpressionBuiltinExceptionRef
from nuitka.nodes.ComparisonNodes import ExpressionComparisonIs
from nuitka.nodes.StatementNodes import StatementsSequence
from nuitka.nodes.ConditionalNodes import StatementConditional
from nuitka.nodes.AssignNodes import StatementAssignmentVariable
from nuitka.nodes.TryNodes import (
StatementExceptHandler,
StatementTryExcept
)
from .ReformulationAssignmentStatements import (
buildDeleteStatementFromDecoded,
buildAssignmentStatements,
decodeAssignTarget
)
from .Helpers import (
makeStatementsSequence,
buildStatementsNode,
buildNode
)
def makeTryExceptNoRaise(provider, temp_scope, tried, handlers, no_raise,
source_ref):
# This helper executes the core re-formulation of "no_raise" blocks, which
# are the "else" blocks of "try"/"except" statements. In order to limit the
# execution, we use an indicator variable instead, which will signal that
# the tried block executed up to the end. And then we make the else block be
# a conditional statement checking that.
# This is a separate function, so it can be re-used in other
# re-formulations, e.g. with statements.
assert no_raise is not None
assert len(handlers) > 0
tmp_handler_indicator_variable = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "unhandled_indicator"
)
for handler in handlers:
statements = (
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_handler_indicator_variable.makeReference(
provider
),
source_ref = source_ref.atInternal()
),
source = ExpressionConstantRef(
constant = False,
source_ref = source_ref
),
source_ref = no_raise.getSourceReference().atInternal()
),
handler.getExceptionBranch()
)
handler.setExceptionBranch(
makeStatementsSequence(
statements = statements,
allow_none = True,
source_ref = source_ref
)
)
statements = (
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_handler_indicator_variable.makeReference(
provider
),
source_ref = source_ref.atInternal()
),
source = ExpressionConstantRef(
constant = True,
source_ref = source_ref
),
source_ref = source_ref
),
StatementTryExcept(
tried = tried,
handlers = handlers,
source_ref = source_ref
),
StatementConditional(
condition = ExpressionComparisonIs(
left = ExpressionTempVariableRef(
variable = tmp_handler_indicator_variable.makeReference(
provider
),
source_ref = source_ref
),
right = ExpressionConstantRef(
constant = True,
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = no_raise,
no_branch = None,
source_ref = source_ref
)
)
return StatementsSequence(
statements = statements,
source_ref = source_ref
)
def makeTryExceptSingleHandlerNode(tried, exception_name, handler_body,
source_ref):
return StatementTryExcept(
tried = tried,
handlers = (
StatementExceptHandler(
exception_types = (
ExpressionBuiltinExceptionRef(
exception_name = exception_name,
source_ref = source_ref
),
),
body = handler_body,
source_ref = source_ref
),
),
source_ref = source_ref
)
def buildTryExceptionNode(provider, node, source_ref):
# Try/except nodes. Re-formulated as described in the developer
# manual. Exception handlers made the assignment to variables explicit. Same
# for the "del" as done for Python3. Also catches always work a tuple of
# exception types and hides away that they may be built or not.
# Many variables, due to the re-formulation that is going on here, which
# just has the complexity, pylint: disable=R0914
handlers = []
for handler in node.handlers:
exception_expression, exception_assign, exception_block = (
handler.type,
handler.name,
handler.body
)
statements = [
buildAssignmentStatements(
provider = provider,
node = exception_assign,
allow_none = True,
source = ExpressionCaughtExceptionValueRef(
source_ref = source_ref.atInternal()
),
source_ref = source_ref.atInternal()
),
buildStatementsNode(
provider = provider,
nodes = exception_block,
source_ref = source_ref
)
]
if Utils.python_version >= 300:
target_info = decodeAssignTarget(
provider = provider,
node = exception_assign,
source_ref = source_ref,
allow_none = True
)
if target_info is not None:
kind, detail = target_info
assert kind == "Name", kind
kind = "Name_Exception"
statements.append(
buildDeleteStatementFromDecoded(
kind = kind,
detail = detail,
source_ref = source_ref
)
)
handler_body = makeStatementsSequence(
statements = statements,
allow_none = True,
source_ref = source_ref
)
exception_types = buildNode(
provider = provider,
node = exception_expression,
source_ref = source_ref,
allow_none = True
)
# The exception types should be a tuple, so as to be most general.
if exception_types is None:
exception_types = ()
if handler is not node.handlers[-1]:
SyntaxErrors.raiseSyntaxError(
reason = "default 'except:' must be last",
source_ref = source_ref.atLineNumber(
handler.lineno-1
if Options.isFullCompat() else
handler.lineno
)
)
elif exception_types.isExpressionMakeSequence():
exception_types = exception_types.getElements()
else:
exception_types = (exception_types,)
handlers.append(
StatementExceptHandler(
exception_types = exception_types,
body = handler_body,
source_ref = source_ref
)
)
tried = buildStatementsNode(
provider = provider,
nodes = node.body,
source_ref = source_ref
)
no_raise = buildStatementsNode(
provider = provider,
nodes = node.orelse,
source_ref = source_ref
)
if no_raise is None:
return StatementTryExcept(
handlers = handlers,
tried = tried,
source_ref = source_ref
)
else:
return makeTryExceptNoRaise(
provider = provider,
temp_scope = provider.allocateTempScope("try_except"),
handlers = handlers,
tried = tried,
no_raise = no_raise,
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/tree/__init__.py 0000644 0001750 0001750 00000001501 12265264105 017703 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Dummy file to make this directory a package. """
Nuitka-0.5.0.1/nuitka/tree/Helpers.py 0000644 0001750 0001750 00000025756 12265264105 017570 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Helper functions for parsing the AST nodes and building the Nuitka node tree.
"""
from nuitka.nodes.StatementNodes import (
StatementsSequence,
StatementsFrame
)
from nuitka.nodes.NodeBases import NodeBase
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.ContainerMakingNodes import (
ExpressionKeyValuePair,
ExpressionMakeTuple,
ExpressionMakeList,
ExpressionMakeDict,
ExpressionMakeSet
)
from nuitka import Tracing, Constants
from logging import warning
import ast
def dump(node):
Tracing.printLine( ast.dump( node ) )
def getKind(node):
return node.__class__.__name__.split(".")[-1]
def extractDocFromBody(node):
# Work around ast.get_docstring breakage.
if len( node.body ) > 0 and getKind( node.body[0] ) == "Expr" and getKind( node.body[0].value ) == "Str":
return node.body[1:], node.body[0].value.s
else:
return node.body, None
build_nodes_args3 = None
build_nodes_args2 = None
build_nodes_args1 = None
def setBuildDispatchers(path_args3, path_args2, path_args1):
# Using global here, as this is really a singleton, in the form of a module,
# and this is to break the cyclic dependency it has, pylint: disable=W0603
global build_nodes_args3, build_nodes_args2, build_nodes_args1
build_nodes_args3 = path_args3
build_nodes_args2 = path_args2
build_nodes_args1 = path_args1
def buildNode(provider, node, source_ref, allow_none = False):
if node is None and allow_none:
return None
try:
kind = getKind( node )
if hasattr( node, "lineno" ):
source_ref = source_ref.atLineNumber( node.lineno )
else:
source_ref = source_ref
if kind in build_nodes_args3:
result = build_nodes_args3[ kind ](
provider = provider,
node = node,
source_ref = source_ref
)
elif kind in build_nodes_args2:
result = build_nodes_args2[ kind ](
node = node,
source_ref = source_ref
)
elif kind in build_nodes_args1:
result = build_nodes_args1[ kind ](
source_ref = source_ref
)
elif kind == "Pass":
result = None
else:
assert False, kind
if result is None and allow_none:
return None
assert isinstance( result, NodeBase ), result
return result
except SyntaxError:
raise
except:
warning( "Problem at '%s' with %s." % ( source_ref, ast.dump( node ) ) )
raise
def buildNodeList(provider, nodes, source_ref, allow_none = False):
if nodes is not None:
result = []
for node in nodes:
if hasattr( node, "lineno" ):
node_source_ref = source_ref.atLineNumber( node.lineno )
else:
node_source_ref = source_ref
entry = buildNode( provider, node, node_source_ref, allow_none )
if entry is not None:
result.append( entry )
return result
else:
return []
def makeModuleFrame(module, statements, source_ref):
assert module.isPythonModule()
if module.isMainModule():
code_name = ""
else:
code_name = module.getName()
return StatementsFrame(
statements = statements,
guard_mode = "once",
var_names = (),
arg_count = 0,
kw_only_count = 0,
code_name = code_name,
has_starlist = False,
has_stardict = False,
source_ref = source_ref
)
def buildStatementsNode(provider, nodes, source_ref, frame = False):
# We are not creating empty statement sequences.
if nodes is None:
return None
# Build as list of statements, throw away empty ones, and remove useless
# nesting.
statements = buildNodeList(provider, nodes, source_ref, allow_none = True)
statements = mergeStatements(statements)
# We are not creating empty statement sequences. Might be empty, because
# e.g. a global node generates not really a statement, or pass statements.
if not statements:
return None
# In case of a frame is desired, build it instead.
if frame:
if provider.isExpressionFunctionBody():
parameters = provider.getParameters()
arg_names = parameters.getCoArgNames()
kw_only_count = parameters.getKwOnlyParameterCount()
code_name = provider.getFunctionName()
guard_mode = "generator" if provider.isGenerator() else "full"
has_starlist = parameters.getStarListArgumentName() is not None
has_stardict = parameters.getStarDictArgumentName() is not None
return StatementsFrame(
statements = statements,
guard_mode = guard_mode,
var_names = arg_names,
arg_count = len( arg_names ),
kw_only_count = kw_only_count,
code_name = code_name,
has_starlist = has_starlist,
has_stardict = has_stardict,
source_ref = source_ref
)
else:
return makeModuleFrame(
module = provider,
statements = statements,
source_ref = source_ref
)
else:
return StatementsSequence(
statements = statements,
source_ref = source_ref
)
def mergeStatements(statements):
""" Helper function that merges nested statement sequences. """
merged_statements = []
for statement in statements:
if statement.isStatement() or statement.isStatementsFrame():
merged_statements.append(statement)
elif statement.isStatementsSequence():
merged_statements.extend(mergeStatements(statement.getStatements()))
else:
assert False, statement
return merged_statements
def makeStatementsSequenceOrStatement(statements, source_ref):
""" Make a statement sequence, but only if more than one statement
Useful for when we can unroll constructs already here, but are not sure if
we actually did that. This avoids the branch or the pollution of doing it
always.
"""
if len(statements) > 1:
return StatementsSequence(
statements = mergeStatements(statements),
source_ref = source_ref
)
else:
return statements[0]
def makeStatementsSequence(statements, allow_none, source_ref):
if allow_none:
statements = tuple(
statement
for statement in
statements
if statement is not None
)
if statements:
return StatementsSequence(
statements = mergeStatements(statements),
source_ref = source_ref
)
else:
return None
def makeStatementsSequenceFromStatement(statement):
return StatementsSequence(
statements = mergeStatements(
(statement,)
),
source_ref = statement.getSourceReference()
)
def makeSequenceCreationOrConstant(sequence_kind, elements, source_ref):
# Sequence creation. Tries to avoid creations with only constant
# elements. Would be caught by optimization, but would be useless churn. For
# mutable constants we cannot do it though.
for element in elements:
if not element.isExpressionConstantRef():
constant = False
break
else:
constant = True
sequence_kind = sequence_kind.upper()
# Note: This would happen in optimization instead, but lets just do it
# immediately to save some time.
if constant:
if sequence_kind == "TUPLE":
const_type = tuple
elif sequence_kind == "LIST":
const_type = list
elif sequence_kind == "SET":
const_type = set
else:
assert False, sequence_kind
return ExpressionConstantRef(
constant = const_type(
element.getConstant()
for element in
elements
),
source_ref = source_ref,
user_provided = True
)
else:
if sequence_kind == "TUPLE":
return ExpressionMakeTuple(
elements = elements,
source_ref = source_ref
)
elif sequence_kind == "LIST":
return ExpressionMakeList(
elements = elements,
source_ref = source_ref
)
elif sequence_kind == "SET":
return ExpressionMakeSet(
elements = elements,
source_ref = source_ref
)
else:
assert False, sequence_kind
def makeDictCreationOrConstant(keys, values, lazy_order, source_ref):
# Create dictionary node. Tries to avoid it for constant values that are not
# mutable.
assert len( keys ) == len( values )
for key, value in zip( keys, values ):
if not key.isExpressionConstantRef():
constant = False
break
if not value.isExpressionConstantRef():
constant = False
break
else:
constant = True
# Note: This would happen in optimization instead, but lets just do it
# immediately to save some time.
if constant:
# Unless tolder otherwise, create the dictionary in its full size, so
# that no growing occurs and the constant becomes as similar as possible
# before being marshalled.
return ExpressionConstantRef(
constant = Constants.createConstantDict(
lazy_order = not lazy_order,
keys = [ key.getConstant() for key in keys ],
values = [ value.getConstant() for value in values ]
),
source_ref = source_ref,
user_provided = True
)
else:
return ExpressionMakeDict(
pairs = [
ExpressionKeyValuePair( key, value, key.getSourceReference() )
for key, value in
zip( keys, values )
],
lazy_order = lazy_order,
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/tree/Building.py 0000644 0001750 0001750 00000116336 12265264105 017716 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Build the internal node tree from source code.
Does all the Python parsing and puts it into a tree structure for use in later
stages of the compilation process.
At the bottom of the file, the dispatching is happening. One function deals
with every node kind as found in the AST. The parsing is centered around the
module "ast" output.
Many higher level language features and translated into lower level ones.
Inplace assignments, for loops, while loops, classes, complex calls, with
statements, and even or/and etc. are all translated to simpler constructs.
The output of this module is a node tree, which contains only relatively low
level operations. A property of the output module is also an overlaid tree
of provider structure that indicates variable provision.
"""
# pylint: disable=W0622
from nuitka.__past__ import long, unicode
# pylint: enable=W0622
from nuitka import (
SourceCodeReferences,
SyntaxErrors,
Importing,
Options,
Utils
)
from nuitka.nodes.FutureSpecs import FutureSpec
from nuitka.nodes.VariableRefNodes import (
ExpressionTargetVariableRef,
ExpressionVariableRef
)
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.ExceptionNodes import StatementRaiseException
from nuitka.nodes.AttributeNodes import ExpressionAttributeLookup
from nuitka.nodes.SubscriptNodes import ExpressionSubscriptLookup
from nuitka.nodes.SliceNodes import (
ExpressionSliceLookup,
ExpressionSliceObject
)
from nuitka.nodes.StatementNodes import (
StatementExpressionOnly,
StatementsSequence
)
from nuitka.nodes.ImportNodes import (
ExpressionImportModule,
ExpressionImportName,
StatementImportStar,
)
from nuitka.nodes.OperatorNodes import (
ExpressionOperationBinary,
ExpressionOperationUnary
)
from nuitka.nodes.LoopNodes import (
StatementContinueLoop,
StatementBreakLoop,
)
from nuitka.nodes.ConditionalNodes import (
ExpressionConditional,
StatementConditional
)
from nuitka.nodes.ReturnNodes import StatementReturn
from nuitka.nodes.AssignNodes import StatementAssignmentVariable
from nuitka.nodes.ModuleNodes import (
PythonShlibModule,
PythonMainModule,
PythonPackage,
PythonModule
)
from nuitka.nodes.TryNodes import StatementTryFinally
from .VariableClosure import completeVariableClosures
# Classes are handled in a separate file. They are re-formulated into functions
# producing dictionaries used to call the metaclass with.
from .ReformulationClasses import buildClassNode
# Try/except/else statements are handled in a separate file. They are
# re-formulated into using a temporary variable to track if the else branch
# should execute.
from .ReformulationTryExceptStatements import buildTryExceptionNode
# With statements are handled in a separate file. They are re-formulated into
# special attribute lookups for "__enter__" and "__exit__", calls of them,
# catching and passing in exceptions raised.
from .ReformulationWithStatements import buildWithNode
from .ReformulationAssignmentStatements import (
buildInplaceAssignNode,
buildExtSliceNode,
buildAssignNode,
buildDeleteNode
)
from .ReformulationLoopStatements import (
buildWhileLoopNode,
buildForLoopNode
)
from .ReformulationAssertStatements import buildAssertNode
from .ReformulationPrintStatements import buildPrintNode
from .ReformulationFunctionStatements import buildFunctionNode
from .ReformulationLambdaExpressions import buildLambdaNode
from .ReformulationBooleanExpressions import buildBoolOpNode
from .ReformulationComparisonExpressions import buildComparisonNode
from .ReformulationContractionExpressions import (
buildGeneratorExpressionNode,
buildListContractionNode,
buildDictContractionNode,
buildSetContractionNode
)
from .ReformulationCallExpressions import buildCallNode
from .ReformulationExecStatements import buildExecNode
from .ReformulationYieldExpressions import buildYieldNode, buildYieldFromNode
from .ReformulationNamespacePackages import createNamespacePackage
# Some helpers.
from .Helpers import (
makeStatementsSequenceOrStatement,
makeSequenceCreationOrConstant,
makeDictCreationOrConstant,
buildStatementsNode,
setBuildDispatchers,
extractDocFromBody,
makeModuleFrame,
mergeStatements,
buildNodeList,
buildNode,
getKind
)
from .SourceReading import readSourceCodeFromFilename
from .ImportCache import addImportedModule
import ast, sys
def buildVariableReferenceNode(provider, node, source_ref):
# Python3 is influenced by the mere use of a variable name. So we need to
# remember it, esp. for cases, where it is optimized away.
if Utils.python_version >= 300 and \
node.id == "super" and \
provider.isExpressionFunctionBody():
provider.markAsClassClosureTaker()
return ExpressionVariableRef(
variable_name = node.id,
source_ref = source_ref
)
# Python3.4 only, True and False, are not given as variables anymore.
def buildNamedConstantNode(node, source_ref):
return ExpressionConstantRef(
constant = node.value,
source_ref = source_ref
)
def buildSequenceCreationNode(provider, node, source_ref):
return makeSequenceCreationOrConstant(
sequence_kind = getKind( node ).upper(),
elements = buildNodeList( provider, node.elts, source_ref ),
source_ref = source_ref
)
def buildDictionaryNode(provider, node, source_ref):
return makeDictCreationOrConstant(
keys = buildNodeList( provider, node.keys, source_ref ),
values = buildNodeList( provider, node.values, source_ref ),
lazy_order = False,
source_ref = source_ref
)
def buildConditionNode(provider, node, source_ref):
# Conditional statements may have one or two branches. We will never see an
# "elif", because that's already dealt with by module "ast", which turns it
# into nested conditional statements.
return StatementConditional(
condition = buildNode( provider, node.test, source_ref ),
yes_branch = buildStatementsNode(
provider = provider,
nodes = node.body,
source_ref = source_ref
),
no_branch = buildStatementsNode(
provider = provider,
nodes = node.orelse if node.orelse else None,
source_ref = source_ref
),
source_ref = source_ref
)
def buildTryFinallyNode(provider, node, source_ref):
# Try/finally node statements.
return StatementTryFinally(
tried = buildStatementsNode(
provider = provider,
nodes = node.body,
source_ref = source_ref
),
final = buildStatementsNode(
provider = provider,
nodes = node.finalbody,
source_ref = source_ref
),
source_ref = source_ref
)
def buildTryNode(provider, node, source_ref):
# Note: This variant is used for Python3.3 or higher only, older stuff uses
# the above ones, this one merges try/except with try/finally in the
# "ast". We split it up again, as it's logically separated of course.
return StatementTryFinally(
tried = StatementsSequence(
statements = mergeStatements(
(
buildTryExceptionNode(
provider = provider,
node = node,
source_ref = source_ref
),
)
),
source_ref = source_ref
),
final = buildStatementsNode(
provider = provider,
nodes = node.finalbody,
source_ref = source_ref
),
source_ref = source_ref
)
def buildRaiseNode(provider, node, source_ref):
# Raise statements. Under Python2 they may have type, value and traceback
# attached, for Python3, you can only give type (actually value) and cause.
if Utils.python_version < 300:
return StatementRaiseException(
exception_type = buildNode( provider, node.type, source_ref, allow_none = True ),
exception_value = buildNode( provider, node.inst, source_ref, allow_none = True ),
exception_trace = buildNode( provider, node.tback, source_ref, allow_none = True ),
exception_cause = None,
source_ref = source_ref
)
else:
return StatementRaiseException(
exception_type = buildNode( provider, node.exc, source_ref, allow_none = True ),
exception_value = None,
exception_trace = None,
exception_cause = buildNode( provider, node.cause, source_ref, allow_none = True ),
source_ref = source_ref
)
def buildSubscriptNode(provider, node, source_ref):
# Subscript expression nodes.
assert getKind( node.ctx ) == "Load", source_ref
# The subscribt "[]" operator is one of many different things. This is
# expressed by this kind, there are "slice" lookups (two values, even if one
# is using default), and then "index" lookups. The form with three argument
# is really an "index" lookup, with a slice object. And the "..." lookup is
# also an index loopup, with it as the argument. So this splits things into
# two different operations, "subscript" with a single "subscript" object. Or
# a slice lookup with a lower and higher boundary. These things should
# behave similar, but they are different slots.
kind = getKind( node.slice )
if kind == "Index":
return ExpressionSubscriptLookup(
expression = buildNode( provider, node.value, source_ref ),
subscript = buildNode( provider, node.slice.value, source_ref ),
source_ref = source_ref
)
elif kind == "Slice":
lower = buildNode( provider, node.slice.lower, source_ref, True )
upper = buildNode( provider, node.slice.upper, source_ref, True )
if node.slice.step is not None:
step = buildNode( provider, node.slice.step, source_ref )
return ExpressionSubscriptLookup(
expression = buildNode( provider, node.value, source_ref ),
subscript = ExpressionSliceObject(
lower = lower,
upper = upper,
step = step,
source_ref = source_ref
),
source_ref = source_ref
)
else:
return ExpressionSliceLookup(
expression = buildNode( provider, node.value, source_ref ),
lower = lower,
upper = upper,
source_ref = source_ref
)
elif kind == "ExtSlice":
return ExpressionSubscriptLookup(
expression = buildNode( provider, node.value, source_ref ),
subscript = buildExtSliceNode( provider, node, source_ref ),
source_ref = source_ref
)
elif kind == "Ellipsis":
return ExpressionSubscriptLookup(
expression = buildNode( provider, node.value, source_ref ),
subscript = ExpressionConstantRef(
constant = Ellipsis,
source_ref = source_ref
),
source_ref = source_ref
)
else:
assert False, kind
def buildImportModulesNode(node, source_ref):
# Import modules statement. As described in the developer manual, these
# statements can be treated as several ones.
import_names = [
( import_desc.name, import_desc.asname )
for import_desc in
node.names
]
import_nodes = []
for import_desc in import_names:
module_name, local_name = import_desc
module_topname = module_name.split(".")[0]
# Note: The "level" of import is influenced by the future absolute
# imports.
level = 0 if source_ref.getFutureSpec().isAbsoluteImport() else -1
if local_name:
import_node = ExpressionImportModule(
module_name = module_name,
import_list = None,
level = level,
source_ref = source_ref
)
for import_name in module_name.split(".")[1:]:
import_node = ExpressionImportName(
module = import_node,
import_name = import_name,
source_ref = source_ref
)
else:
import_node = ExpressionImportModule(
module_name = module_name,
import_list = None,
level = level,
source_ref = source_ref
)
# If a name was given, use the one provided, otherwise the import gives
# the top level package name given for assignment of the imported
# module.
import_nodes.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = local_name
if local_name is not None else
module_topname,
source_ref = source_ref
),
source = import_node,
source_ref = source_ref
)
)
# Note: Each import is sequential. It will potentially succeed, and the
# failure of a later one is not changing that one bit . We can therefore
# have a sequence of imports that only import one thing therefore.
return makeStatementsSequenceOrStatement(
statements = import_nodes,
source_ref = source_ref
)
def enableFutureFeature(object_name, future_spec, source_ref):
if object_name == "unicode_literals":
future_spec.enableUnicodeLiterals()
elif object_name == "absolute_import":
future_spec.enableAbsoluteImport()
elif object_name == "division":
future_spec.enableFutureDivision()
elif object_name == "print_function":
future_spec.enableFuturePrint()
elif object_name == "barry_as_FLUFL" and Utils.python_version >= 300:
future_spec.enableBarry()
elif object_name == "braces":
SyntaxErrors.raiseSyntaxError(
"not a chance",
source_ref
)
elif object_name in ( "nested_scopes", "generators", "with_statement" ):
# These are enabled in all cases already.
pass
else:
SyntaxErrors.raiseSyntaxError(
"future feature %s is not defined" % object_name,
source_ref
)
# For checking afterwards, if it was at the beginning of the file.
_future_import_nodes = []
def buildImportFromNode(provider, node, source_ref):
# "from .. import .." statements. This may trigger a star import, or
# multiple names being looked up from the given module variable name.
module_name = node.module if node.module is not None else ""
level = node.level
# Importing from "__future__" module may enable flags.
if module_name == "__future__":
if not provider.isPythonModule() and not source_ref.isExecReference():
SyntaxErrors.raiseSyntaxError(
reason = """\
from __future__ imports must occur at the beginning of the file""",
col_offset = 8
if Utils.python_version >= 300 or \
not Options.isFullCompat()
else None,
source_ref = source_ref
)
for import_desc in node.names:
object_name, _local_name = import_desc.name, import_desc.asname
enableFutureFeature(
object_name = object_name,
future_spec = source_ref.getFutureSpec(),
source_ref = source_ref
)
# Remember it for checks to be applied once module is complete.
node.source_ref = source_ref
_future_import_nodes.append( node )
target_names = []
import_names = []
for import_desc in node.names:
object_name, local_name = import_desc.name, import_desc.asname
if object_name == "*":
target_names.append( None )
else:
target_names.append(
local_name
if local_name is not None else
object_name
)
import_names.append( object_name )
if None in target_names:
# More than "*" is a syntax error in Python, need not care about this at
# all, it's only allowed value for import list in this case.
assert target_names == [ None ]
# Python3 made this a syntax error unfortunately.
if not provider.isPythonModule() and Utils.python_version >= 300:
SyntaxErrors.raiseSyntaxError(
"import * only allowed at module level",
provider.getSourceReference()
)
if provider.isExpressionFunctionBody():
provider.markAsStarImportContaining()
return StatementImportStar(
module_import = ExpressionImportModule(
module_name = module_name,
import_list = ( "*", ),
level = level,
source_ref = source_ref
),
source_ref = source_ref
)
else:
import_nodes = []
for target_name, import_name in zip( target_names, import_names ):
import_nodes.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = target_name,
source_ref = source_ref
),
source = ExpressionImportName(
module = ExpressionImportModule(
module_name = module_name,
import_list = import_names,
level = level,
source_ref = source_ref
),
import_name = import_name,
source_ref = source_ref
),
source_ref = source_ref
)
)
# Note: Each import is sequential. It can succeed, and the failure of a
# later one is not changing one. We can therefore have a sequence of
# imports that only import one thing therefore.
return StatementsSequence(
statements = import_nodes,
source_ref = source_ref
)
def handleGlobalDeclarationNode(provider, node, source_ref):
if not source_ref.isExecReference():
# On the module level, there is nothing to do.
if provider.isPythonModule():
return None
# Need to catch the error of declaring a parameter variable as global
# ourselves here. The AST parsing doesn't catch it.
try:
parameters = provider.getParameters()
for variable_name in node.names:
if variable_name in parameters.getParameterNames():
SyntaxErrors.raiseSyntaxError(
reason = "name '%s' is %s and global" % (
variable_name,
"local"
if Utils.python_version < 300 else
"parameter"
),
source_ref = provider.getSourceReference()
)
except AttributeError:
pass
module = provider.getParentModule()
for variable_name in node.names:
closure_variable = None
# Re-use already taken global variables, in order to avoid creating yet
# another instance, esp. as the markups could then potentially not be
# shared.
if provider.hasTakenVariable( variable_name ):
closure_variable = provider.getTakenVariable( variable_name )
if not closure_variable.isModuleVariableReference():
closure_variable = None
if closure_variable is None:
module_variable = module.getVariableForAssignment(
variable_name = variable_name
)
closure_variable = provider.addClosureVariable(
variable = module_variable
)
assert closure_variable.isModuleVariableReference()
closure_variable.markFromGlobalStatement()
if source_ref.isExecReference():
closure_variable.markFromExecStatement()
provider.registerProvidedVariable(
variable = closure_variable
)
return None
def handleNonlocalDeclarationNode(provider, node, source_ref):
# The source reference of the nonlocal really doesn't matter.
# pylint: disable=W0613
# Need to catch the error of declaring a parameter variable as global
# ourselves here. The AST parsing doesn't catch it, but we can do it here.
parameters = provider.getParameters()
for variable_name in node.names:
if variable_name in parameters.getParameterNames():
SyntaxErrors.raiseSyntaxError(
reason = "name '%s' is parameter and nonlocal" % (
variable_name
),
source_ref = None if Options.isFullCompat() else source_ref,
display_file = not Options.isFullCompat(),
display_line = not Options.isFullCompat()
)
provider.addNonlocalsDeclaration( node.names, source_ref )
return None
def buildStringNode(node, source_ref):
assert type( node.s ) in ( str, unicode )
return ExpressionConstantRef(
constant = node.s,
source_ref = source_ref,
user_provided = True
)
def buildNumberNode(node, source_ref):
assert type( node.n ) in ( int, long, float, complex ), type( node.n )
return ExpressionConstantRef(
constant = node.n,
source_ref = source_ref,
user_provided = True
)
def buildBytesNode(node, source_ref):
return ExpressionConstantRef(
constant = node.s,
source_ref = source_ref,
user_provided = True
)
def buildEllipsisNode(source_ref):
return ExpressionConstantRef(
constant = Ellipsis,
source_ref = source_ref,
user_provided = True
)
def buildAttributeNode(provider, node, source_ref):
return ExpressionAttributeLookup(
expression = buildNode( provider, node.value, source_ref ),
attribute_name = node.attr,
source_ref = source_ref
)
def buildReturnNode(provider, node, source_ref):
if not provider.isExpressionFunctionBody() or \
provider.isClassDictCreation():
SyntaxErrors.raiseSyntaxError(
"'return' outside function",
source_ref,
None if Utils.python_version < 300 else (
node.col_offset
if provider.isPythonModule() else
node.col_offset+4
)
)
if node.value is not None:
return StatementReturn(
expression = buildNode( provider, node.value, source_ref ),
source_ref = source_ref
)
else:
return StatementReturn(
expression = ExpressionConstantRef(
constant = None,
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
)
def buildExprOnlyNode(provider, node, source_ref):
return StatementExpressionOnly(
expression = buildNode( provider, node.value, source_ref ),
source_ref = source_ref
)
def buildUnaryOpNode(provider, node, source_ref):
if getKind( node.op ) == "Not":
return buildBoolOpNode(
provider = provider,
node = node,
source_ref = source_ref
)
else:
return ExpressionOperationUnary(
operator = getKind( node.op ),
operand = buildNode( provider, node.operand, source_ref ),
source_ref = source_ref
)
def buildBinaryOpNode(provider, node, source_ref):
operator = getKind( node.op )
if operator == "Div" and source_ref.getFutureSpec().isFutureDivision():
operator = "TrueDiv"
return ExpressionOperationBinary(
operator = operator,
left = buildNode( provider, node.left, source_ref ),
right = buildNode( provider, node.right, source_ref ),
source_ref = source_ref
)
def buildReprNode(provider, node, source_ref):
return ExpressionOperationUnary(
operator = "Repr",
operand = buildNode( provider, node.value, source_ref ),
source_ref = source_ref
)
def buildConditionalExpressionNode(provider, node, source_ref):
return ExpressionConditional(
condition = buildNode( provider, node.test, source_ref ),
yes_expression = buildNode( provider, node.body, source_ref ),
no_expression = buildNode( provider, node.orelse, source_ref ),
source_ref = source_ref
)
setBuildDispatchers(
path_args3 = {
"Name" : buildVariableReferenceNode,
"Assign" : buildAssignNode,
"Delete" : buildDeleteNode,
"Lambda" : buildLambdaNode,
"GeneratorExp" : buildGeneratorExpressionNode,
"If" : buildConditionNode,
"While" : buildWhileLoopNode,
"For" : buildForLoopNode,
"Compare" : buildComparisonNode,
"ListComp" : buildListContractionNode,
"DictComp" : buildDictContractionNode,
"SetComp" : buildSetContractionNode,
"Dict" : buildDictionaryNode,
"Set" : buildSequenceCreationNode,
"Tuple" : buildSequenceCreationNode,
"List" : buildSequenceCreationNode,
"Global" : handleGlobalDeclarationNode,
"Nonlocal" : handleNonlocalDeclarationNode,
"TryExcept" : buildTryExceptionNode,
"TryFinally" : buildTryFinallyNode,
"Try" : buildTryNode,
"Raise" : buildRaiseNode,
"ImportFrom" : buildImportFromNode,
"Assert" : buildAssertNode,
"Exec" : buildExecNode,
"With" : buildWithNode,
"FunctionDef" : buildFunctionNode,
"ClassDef" : buildClassNode,
"Print" : buildPrintNode,
"Call" : buildCallNode,
"Subscript" : buildSubscriptNode,
"BoolOp" : buildBoolOpNode,
"Attribute" : buildAttributeNode,
"Return" : buildReturnNode,
"Yield" : buildYieldNode,
"YieldFrom" : buildYieldFromNode,
"Expr" : buildExprOnlyNode,
"UnaryOp" : buildUnaryOpNode,
"BinOp" : buildBinaryOpNode,
"Repr" : buildReprNode,
"AugAssign" : buildInplaceAssignNode,
"IfExp" : buildConditionalExpressionNode,
},
path_args2 = {
"NameConstant" : buildNamedConstantNode,
"Import" : buildImportModulesNode,
"Str" : buildStringNode,
"Num" : buildNumberNode,
"Bytes" : buildBytesNode,
},
path_args1 = {
"Ellipsis" : buildEllipsisNode,
"Continue" : StatementContinueLoop,
"Break" : StatementBreakLoop,
}
)
def buildParseTree( provider, source_code, source_ref, is_module,
is_main ):
# Workaround: ast.parse cannot cope with some situations where a file is not
# terminated by a new line.
if not source_code.endswith( "\n" ):
source_code = source_code + "\n"
body = ast.parse( source_code, source_ref.getFilename() )
assert getKind( body ) == "Module"
line_offset = source_ref.getLineNumber() - 1
if line_offset > 0:
for created_node in ast.walk( body ):
if hasattr( created_node, "lineno" ):
created_node.lineno += line_offset
body, doc = extractDocFromBody( body )
result = buildStatementsNode(
provider = provider,
nodes = body,
source_ref = source_ref
)
# Check if a __future__ imports really were at the beginning of the file.
for node in body:
if node in _future_import_nodes:
_future_import_nodes.remove( node )
else:
if _future_import_nodes:
SyntaxErrors.raiseSyntaxError(
reason = """\
from __future__ imports must occur at the beginning of the file""",
col_offset = 1
if Utils.python_version >= 300 or \
not Options.isFullCompat() else
None,
source_ref = _future_import_nodes[0].source_ref
)
internal_source_ref = source_ref.atInternal()
statements = []
if is_module:
# Add import of "site" module of main programs visibly in the node tree,
# so recursion and optimization can pick it up, checking its effects.
if is_main and not sys.flags.no_site:
statements.append(
StatementExpressionOnly(
expression = ExpressionImportModule(
module_name = "site",
import_list = (),
level = 0,
source_ref = source_ref,
),
source_ref = source_ref
)
)
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__doc__",
source_ref = internal_source_ref
),
source = ExpressionConstantRef(
constant = doc,
source_ref = internal_source_ref,
user_provided = True
),
source_ref = internal_source_ref
)
)
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__file__",
source_ref = internal_source_ref
),
source = ExpressionConstantRef(
constant = source_ref.getFilename(),
source_ref = internal_source_ref,
user_provided = True
),
source_ref = internal_source_ref
)
)
if provider.isPythonPackage():
# TODO: __package__ is not set here, but automatically, which makes
# it invisible though
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__path__",
source_ref = internal_source_ref
),
source = ExpressionConstantRef(
constant = [
Utils.dirname( source_ref.getFilename() )
],
source_ref = internal_source_ref,
user_provided = True
),
source_ref = internal_source_ref
)
)
if Utils.python_version >= 300:
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__cached__",
source_ref = internal_source_ref
),
source = ExpressionConstantRef(
constant = None,
source_ref = internal_source_ref,
user_provided = True
),
source_ref = internal_source_ref
)
)
if Utils.python_version >= 330:
# For Python3.3, it's set for both packages and non-packages.
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__package__",
source_ref = internal_source_ref
),
source = ExpressionConstantRef(
constant = provider.getFullName()
if provider.isPythonPackage() else
provider.getPackage(),
source_ref = internal_source_ref,
user_provided = True
),
source_ref = internal_source_ref
)
)
if Utils.python_version >= 330 and not provider.isMainModule():
# Set initialzing at the beginning to True
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__initializing__",
source_ref = internal_source_ref
),
source = ExpressionConstantRef(
constant = True,
source_ref = internal_source_ref,
user_provided = True
),
source_ref = internal_source_ref
)
)
# Now the module body if there is any at all.
if result is not None:
statements.extend( result.getStatements() )
if Utils.python_version >= 330 and not provider.isMainModule():
# Set initialzing at the beginning to True
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetVariableRef(
variable_name = "__initializing__",
source_ref = internal_source_ref
),
source = ExpressionConstantRef(
constant = False,
source_ref = internal_source_ref,
user_provided = True
),
source_ref = internal_source_ref
)
)
if is_module:
return makeModuleFrame(
module = provider,
statements = statements,
source_ref = source_ref
)
else:
assert False
def decideModuleTree(filename, package, is_shlib, is_top, is_main):
# Many variables, branches, due to the many cases, pylint: disable=R0912
assert package is None or type( package ) is str
if is_main and Utils.isDir( filename ):
source_filename = Utils.joinpath( filename, "__main__.py" )
if not Utils.isFile( source_filename ):
sys.stderr.write(
"%s: can't find '__main__' module in '%s'\n" % (
Utils.basename( sys.argv[0] ),
filename
)
)
sys.exit( 2 )
filename = source_filename
main_added = True
else:
main_added = False
if Utils.isFile( filename ):
source_filename = filename
source_ref = SourceCodeReferences.fromFilename(
filename = filename,
future_spec = FutureSpec()
)
if is_main:
module_name = "__main__"
else:
module_name = Utils.basename( filename )
if module_name.endswith( ".py" ):
module_name = module_name[:-3]
if is_shlib:
module_name = module_name.split(".")[0]
if "." in module_name:
sys.stderr.write(
"Error, '%s' is not a proper python module name.\n" % (
module_name
)
)
sys.exit( 2 )
if is_shlib:
result = PythonShlibModule(
name = module_name,
source_ref = source_ref,
package_name = package,
)
elif is_main:
result = PythonMainModule(
source_ref = source_ref,
main_added = main_added
)
else:
result = PythonModule(
name = module_name,
package_name = package,
source_ref = source_ref
)
elif Importing.isPackageDir(filename):
if is_top:
package_name = Utils.splitpath(filename)[-1]
else:
package_name = Utils.basename(filename)
source_filename = Utils.joinpath(filename, "__init__.py")
if not Utils.isFile( source_filename ):
assert Utils.python_version >= 330, source_filename
source_ref, result = createNamespacePackage(
package_name = package_name,
module_relpath = filename
)
source_filename = None
else:
source_ref = SourceCodeReferences.fromFilename(
filename = Utils.abspath( source_filename ),
future_spec = FutureSpec()
)
result = PythonPackage(
name = package_name,
package_name = package,
source_ref = source_ref
)
else:
sys.stderr.write(
"%s: can't open file '%s'.\n" % (
Utils.basename( sys.argv[0] ),
filename
)
)
sys.exit( 2 )
if not Options.shallHaveStatementLines():
source_ref = source_ref.atInternal()
return result, source_ref, source_filename
def createModuleTree(module, source_ref, source_filename, is_main):
source_code = readSourceCodeFromFilename( source_filename )
module_body = buildParseTree(
provider = module,
source_code = source_code,
source_ref = source_ref,
is_module = True,
is_main = is_main
)
module.setBody( module_body )
completeVariableClosures( module )
def buildModuleTree(filename, package, is_top, is_main):
module, source_ref, source_filename = decideModuleTree(
filename = filename,
package = package,
is_top = is_top,
is_main = is_main,
is_shlib = False
)
addImportedModule( Utils.relpath( filename ), module )
# If there is source code associated (not the case for namespace packages of
# Python3.3 or higher, then read it.
if source_filename is not None:
createModuleTree(
module = module,
source_ref = source_ref,
source_filename = source_filename,
is_main = is_main
)
return module
Nuitka-0.5.0.1/nuitka/tree/ComplexCallHelperFunctions.py 0000644 0001750 0001750 00000223751 12265264105 023415 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" This module is providing helper functions for complex call re-formulations.
One for each type of call. """
from nuitka.nodes.FunctionNodes import (
ExpressionFunctionBody,
ExpressionFunctionCreation,
ExpressionFunctionCall,
ExpressionFunctionRef
)
from nuitka.nodes.StatementNodes import (
StatementsSequence,
StatementsFrame
)
from nuitka.nodes.LoopNodes import (
StatementLoop,
StatementBreakLoop
)
from nuitka.nodes.TypeNodes import (
ExpressionBuiltinIsinstance,
ExpressionBuiltinType1
)
from nuitka.nodes.BuiltinRefNodes import (
ExpressionBuiltinAnonymousRef,
ExpressionBuiltinRef
)
from nuitka.nodes.ConditionalNodes import StatementConditional
from nuitka.nodes.ComparisonNodes import ExpressionComparison
from nuitka.nodes.VariableRefNodes import (
ExpressionTargetTempVariableRef,
ExpressionTargetVariableRef,
ExpressionTempVariableRef,
ExpressionVariableRef
)
from nuitka.nodes.CallNodes import (
ExpressionCallKeywordsOnly,
ExpressionCallNoKeywords,
ExpressionCallEmpty,
ExpressionCall
)
from nuitka.nodes.ReturnNodes import StatementReturn
from nuitka.nodes.AssignNodes import (
StatementAssignmentVariable,
StatementAssignmentSubscript
)
from nuitka.nodes.ExceptionNodes import (
StatementRaiseException,
ExpressionBuiltinMakeException
)
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.AttributeNodes import ExpressionAttributeLookup
from nuitka.nodes.ContainerMakingNodes import ExpressionMakeTuple
from nuitka.nodes.BuiltinTypeNodes import ExpressionBuiltinTuple
from nuitka.nodes.OperatorNodes import (
ExpressionOperationBinary,
ExpressionOperationNOT
)
from nuitka.nodes.BuiltinIteratorNodes import (
ExpressionBuiltinIter1,
ExpressionBuiltinNext1
)
from nuitka.nodes.SubscriptNodes import ExpressionSubscriptLookup
from nuitka.nodes.BuiltinDictNodes import ExpressionBuiltinDict
from nuitka.nodes.ModuleNodes import PythonInternalModule
from nuitka.nodes.ParameterSpecs import ParameterSpec
from nuitka.nodes.FutureSpecs import FutureSpec
from nuitka.SourceCodeReferences import fromFilename
from .ReformulationTryExceptStatements import makeTryExceptSingleHandlerNode
from .VariableClosure import completeVariableClosures
from .Helpers import makeStatementsSequenceFromStatement
source_ref = fromFilename( "internal", FutureSpec() ).atInternal()
from nuitka.Utils import python_version
# Cache result. TODO: no more as special as it used to be, maybe can be found in
# stdlib.
def once_decorator(func):
func.cached_value = None
def replacement():
if func.cached_value is None:
func.cached_value = func()
return func.cached_value
return replacement
internal_module = None
def getInternalModule():
# Using global here, as this is really a about the internal module as a
# singleton, pylint: disable=W0603
global internal_module
if internal_module is None:
internal_module = PythonInternalModule()
return internal_module
def makeCalledVariableRef():
variable_ref = ExpressionVariableRef(
variable_name = "called",
source_ref = source_ref
)
return variable_ref
def makeArgsVariableRef():
variable_ref = ExpressionVariableRef(
variable_name = "args",
source_ref = source_ref
)
return variable_ref
def makeKwVariableRef(assign):
variable_ref_class = ExpressionTargetVariableRef if assign else ExpressionVariableRef
variable_ref = variable_ref_class(
variable_name = "kw",
source_ref = source_ref
)
return variable_ref
def makeStarListArgVariableRef(assign):
variable_ref_class = ( ExpressionTargetVariableRef
if assign else
ExpressionVariableRef )
variable_ref = variable_ref_class(
variable_name = "star_arg_list",
source_ref = source_ref
)
return variable_ref
def makeStarDictArgVariableRef(assign):
variable_ref_class = ( ExpressionTargetVariableRef
if assign else
ExpressionVariableRef )
variable_ref = variable_ref_class(
variable_name = "star_arg_dict",
source_ref = source_ref
)
return variable_ref
# TODO: Code generation should become capable of not generating actual
# exceptions for the TypeError caught immediately and then unused, then the
# frame will be unnecessary.
def makePseudoFrame(parameters, statements):
return StatementsFrame(
code_name = "unused",
guard_mode = "pass_through",
var_names = parameters.getParameterNames(),
arg_count = parameters.getArgumentCount(),
has_starlist = parameters.getStarListArgumentName() is not None,
has_stardict = parameters.getStarDictArgumentName() is not None,
kw_only_count = 0,
statements = statements,
source_ref = source_ref
)
@once_decorator
def getCallableNameDescBody():
helper_name = "get_callable_name_desc"
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = ( "called", ),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
# Equivalent of:
#
# Note: The "called_type" is a temporary variable.
#
# called_type = type( BuiltinFunctionType )
#
# if ininstance( called, (FunctionType, MethodType, BuiltinFunctionType) ):
# return called.__name__
# elif python_version < 3 and isinstance( called, ClassType ):
# return called_type.__name__ + " constructor"
# elif python_version < 3 and isinstance( called, InstanceType ):
# return called_type.__name__ + " instance"
# else:
# return called_type.__name__ + " object"
def makeNameAttributeLookup(node, attribute_name = "__name__"):
return ExpressionAttributeLookup(
expression = node,
attribute_name = attribute_name,
source_ref = source_ref
)
functions_case = makeStatementsSequenceFromStatement(
statement = (
StatementReturn(
expression = ExpressionOperationBinary(
operator = "Add",
right = ExpressionConstantRef(
constant = "()",
source_ref = source_ref,
user_provided = True
),
left = makeNameAttributeLookup(
makeCalledVariableRef()
),
source_ref = source_ref
),
source_ref = source_ref
)
)
)
no_branch = makeStatementsSequenceFromStatement(
statement = StatementReturn(
expression = ExpressionOperationBinary(
operator = "Add",
right = ExpressionConstantRef(
constant = " object",
source_ref = source_ref,
user_provided = True
),
left = makeNameAttributeLookup(
ExpressionBuiltinType1(
value = makeCalledVariableRef(),
source_ref = source_ref
)
),
source_ref = source_ref
),
source_ref = source_ref
),
)
if python_version < 300:
instance_case = makeStatementsSequenceFromStatement(
statement = StatementReturn(
expression = ExpressionOperationBinary(
operator = "Add",
right = ExpressionConstantRef(
constant = " instance",
source_ref = source_ref,
user_provided = True
),
left = makeNameAttributeLookup(
makeNameAttributeLookup(
makeCalledVariableRef(),
attribute_name = "__class__",
)
),
source_ref = source_ref
),
source_ref = source_ref
)
)
no_branch = makeStatementsSequenceFromStatement(
statement = StatementConditional(
condition = ExpressionBuiltinIsinstance(
instance = makeCalledVariableRef(),
cls = ExpressionBuiltinAnonymousRef(
builtin_name = "instance",
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = instance_case,
no_branch = no_branch,
source_ref = source_ref
)
)
class_case = makeStatementsSequenceFromStatement(
statement = StatementReturn(
expression = ExpressionOperationBinary(
operator = "Add",
right = ExpressionConstantRef(
constant = " constructor",
source_ref = source_ref,
user_provided = True
),
left = makeNameAttributeLookup(
makeCalledVariableRef(),
),
source_ref = source_ref
),
source_ref = source_ref
)
)
no_branch = makeStatementsSequenceFromStatement(
statement = StatementConditional(
condition = ExpressionBuiltinIsinstance(
instance = makeCalledVariableRef(),
cls = ExpressionBuiltinAnonymousRef(
builtin_name = "classobj",
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = class_case,
no_branch = no_branch,
source_ref = source_ref
)
)
if python_version < 300:
normal_cases = (
"function", "builtin_function_or_method", "instancemethod"
)
else:
normal_cases = (
"function", "builtin_function_or_method"
)
statements = (
StatementConditional(
condition = ExpressionBuiltinIsinstance(
instance = makeCalledVariableRef(),
cls = ExpressionMakeTuple(
elements = tuple(
ExpressionBuiltinAnonymousRef(
builtin_name = builtin_name,
source_ref = source_ref
)
for builtin_name in
normal_cases
),
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = functions_case,
no_branch = no_branch,
source_ref = source_ref
),
)
result.setBody(
StatementsSequence(
statements = statements,
source_ref = source_ref
)
)
completeVariableClosures( result )
return result
def _makeStarListArgumentToTupleStatement( called_variable_ref,
star_list_target_variable_ref,
star_list_variable_ref ):
raise_statement = StatementRaiseException(
exception_type = ExpressionBuiltinMakeException(
exception_name = "TypeError",
args = (
ExpressionOperationBinary(
operator = "Mod",
left = ExpressionConstantRef(
constant = """\
%s argument after * must be a sequence, not %s""",
source_ref = source_ref,
user_provided = True
),
right = ExpressionMakeTuple(
elements = (
ExpressionFunctionCall(
function = ExpressionFunctionCreation(
function_ref = ExpressionFunctionRef(
function_body = getCallableNameDescBody(),
source_ref = source_ref
),
defaults = (),
kw_defaults = None,
annotations = None,
source_ref = source_ref
),
values = (
called_variable_ref,
),
source_ref = source_ref
),
ExpressionAttributeLookup(
expression = ExpressionBuiltinType1(
value = star_list_variable_ref.makeCloneAt( source_ref ),
source_ref = source_ref
),
attribute_name = "__name__",
source_ref = source_ref
)
),
source_ref = source_ref
),
source_ref = source_ref
),
),
source_ref = source_ref
),
exception_value = None,
exception_trace = None,
exception_cause = None,
source_ref = source_ref
)
handler_body = makeStatementsSequenceFromStatement(
statement = raise_statement
)
return StatementConditional(
condition = ExpressionOperationNOT(
operand = ExpressionBuiltinIsinstance(
instance = star_list_variable_ref.makeCloneAt( source_ref ),
cls = ExpressionBuiltinRef(
builtin_name = "tuple",
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = makeStatementsSequenceFromStatement(
statement = makeTryExceptSingleHandlerNode(
tried = makeStatementsSequenceFromStatement(
statement = StatementAssignmentVariable(
variable_ref = star_list_target_variable_ref.makeCloneAt( source_ref ),
source = ExpressionBuiltinTuple(
value = star_list_variable_ref.makeCloneAt( source_ref ),
source_ref = source_ref
),
source_ref = source_ref
)
),
exception_name = "TypeError",
handler_body = handler_body,
source_ref = source_ref
),
),
no_branch = None,
source_ref = source_ref
)
def _makeStarDictArgumentToDictStatement( result, called_variable_ref,
star_dict_target_variable_ref,
star_dict_variable_ref ):
raise_statement = StatementRaiseException(
exception_type = ExpressionBuiltinMakeException(
exception_name = "TypeError",
args = (
ExpressionOperationBinary(
operator = "Mod",
left = ExpressionConstantRef(
constant = """\
%s argument after ** must be a mapping, not %s""",
source_ref = source_ref,
user_provided = True
),
right = ExpressionMakeTuple(
elements = (
ExpressionFunctionCall(
function = ExpressionFunctionCreation(
function_ref = ExpressionFunctionRef(
function_body = getCallableNameDescBody(),
source_ref = source_ref
),
defaults = (),
kw_defaults = None,
annotations = None,
source_ref = source_ref
),
values = (
called_variable_ref,
),
source_ref = source_ref
),
ExpressionAttributeLookup(
expression = ExpressionBuiltinType1(
value = star_dict_variable_ref.makeCloneAt( source_ref ),
source_ref = source_ref
),
attribute_name = "__name__",
source_ref = source_ref
)
),
source_ref = source_ref
),
source_ref = source_ref
),
),
source_ref = source_ref
),
exception_value = None,
exception_trace = None,
exception_cause = None,
source_ref = source_ref
)
temp_scope = result.allocateTempScope( "mapping" )
tmp_dict_variable = result.allocateTempVariable( temp_scope, "dict" )
tmp_iter_variable = result.allocateTempVariable( temp_scope, "iter" )
tmp_keys_variable = result.allocateTempVariable( temp_scope, "keys" )
tmp_key_variable = result.allocateTempVariable( temp_scope, "key" )
statements = (
makeTryExceptSingleHandlerNode(
tried = makeStatementsSequenceFromStatement(
statement = StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_key_variable.makeReference(
result
),
source_ref = source_ref
),
source = ExpressionBuiltinNext1(
value = ExpressionTempVariableRef(
variable = tmp_iter_variable.makeReference(
result
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
),
exception_name = "StopIteration",
handler_body = makeStatementsSequenceFromStatement(
statement = StatementBreakLoop(
source_ref = source_ref
)
),
source_ref = source_ref
),
StatementAssignmentSubscript(
expression = ExpressionTempVariableRef(
variable = tmp_dict_variable.makeReference( result ),
source_ref = source_ref
),
subscript = ExpressionTempVariableRef(
variable = tmp_key_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionSubscriptLookup(
expression = star_dict_variable_ref.makeCloneAt( source_ref ),
subscript = ExpressionTempVariableRef(
variable = tmp_key_variable.makeReference( result ),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
)
loop_body = StatementsSequence(
statements = statements,
source_ref = source_ref
)
statements = (
# Initializing the temp variable outside of try/except, because code
# generation does not yet detect that case properly. TODO: Can be
# removed once code generation is apt enough.
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_keys_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = None,
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
),
makeTryExceptSingleHandlerNode(
tried = makeStatementsSequenceFromStatement(
statement = StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_keys_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionCallEmpty(
called = ExpressionAttributeLookup(
expression = star_dict_variable_ref.makeCloneAt( source_ref ),
attribute_name = "keys",
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
),
exception_name = "AttributeError",
handler_body = makeStatementsSequenceFromStatement(
statement = raise_statement
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_iter_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionBuiltinIter1(
value = ExpressionTempVariableRef(
variable = tmp_keys_variable.makeReference( result ),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_dict_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = {},
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
),
StatementLoop(
body = loop_body,
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = star_dict_target_variable_ref.makeCloneAt(
source_ref = source_ref
),
source = ExpressionTempVariableRef(
variable = tmp_dict_variable.makeReference( result ),
source_ref = source_ref
),
source_ref = source_ref
),
)
mapping_case = StatementsSequence(
statements = statements,
source_ref = source_ref
)
return StatementConditional(
condition = ExpressionOperationNOT(
operand = ExpressionBuiltinIsinstance(
instance = star_dict_variable_ref.makeCloneAt(
source_ref = source_ref
),
cls = ExpressionBuiltinRef(
builtin_name = "dict",
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = mapping_case,
no_branch = None,
source_ref = source_ref
)
def _makeStarDictArgumentMergeToKwStatement( result, called_variable_ref,
kw_target_variable_ref,
kw_variable_ref,
star_dict_variable_ref ):
# This is plain terribly complex, pylint: disable=R0914
raise_statement = StatementRaiseException(
exception_type = ExpressionBuiltinMakeException(
exception_name = "TypeError",
args = (
ExpressionOperationBinary(
operator = "Mod",
left = ExpressionConstantRef(
constant = """\
%s argument after ** must be a mapping, not %s""",
source_ref = source_ref,
user_provided = True
),
right = ExpressionMakeTuple(
elements = (
ExpressionFunctionCall(
function = ExpressionFunctionCreation(
function_ref = ExpressionFunctionRef(
function_body = getCallableNameDescBody(),
source_ref = source_ref
),
defaults = (),
kw_defaults = None,
annotations = None,
source_ref = source_ref
),
values = (
called_variable_ref.makeCloneAt(
source_ref
),
),
source_ref = source_ref
),
ExpressionAttributeLookup(
expression = ExpressionBuiltinType1(
value = star_dict_variable_ref.makeCloneAt( source_ref ),
source_ref = source_ref
),
attribute_name = "__name__",
source_ref = source_ref
)
),
source_ref = source_ref
),
source_ref = source_ref
),
),
source_ref = source_ref
),
exception_value = None,
exception_trace = None,
exception_cause = None,
source_ref = source_ref
)
temp_scope = result.allocateTempScope( "dict" )
tmp_dict_variable = result.allocateTempVariable( temp_scope, "dict" )
tmp_keys_variable = result.allocateTempVariable( temp_scope, "keys" )
tmp_key_variable = result.allocateTempVariable( temp_scope, "key_xxx" )
tmp_iter_variable = result.allocateTempVariable( temp_scope, "iter" )
raise_duplicate = StatementRaiseException(
exception_type = ExpressionBuiltinMakeException(
exception_name = "TypeError",
args = (
ExpressionOperationBinary(
operator = "Mod",
left = ExpressionConstantRef(
constant = """\
%s got multiple values for keyword argument '%s'""",
source_ref = source_ref,
user_provided = True
),
right = ExpressionMakeTuple(
elements = (
ExpressionFunctionCall(
function = ExpressionFunctionCreation(
function_ref = ExpressionFunctionRef(
function_body = getCallableNameDescBody(
),
source_ref = source_ref
),
defaults = (),
kw_defaults = None,
annotations = None,
source_ref = source_ref
),
values = (
called_variable_ref.makeCloneAt(
source_ref
),
),
source_ref = source_ref
),
ExpressionTempVariableRef(
variable = tmp_key_variable.makeReference(
result
),
source_ref = source_ref
)
),
source_ref = source_ref
),
source_ref = source_ref
),
),
source_ref = source_ref
),
exception_value = None,
exception_trace = None,
exception_cause = None,
source_ref = source_ref
)
statements = (
makeTryExceptSingleHandlerNode(
tried = makeStatementsSequenceFromStatement(
statement = StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_key_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionBuiltinNext1(
value = ExpressionTempVariableRef(
variable = tmp_iter_variable.makeReference(
result
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
),
exception_name = "StopIteration",
handler_body = makeStatementsSequenceFromStatement(
statement = StatementBreakLoop(
source_ref = source_ref
)
),
source_ref = source_ref
),
StatementConditional(
condition = ExpressionComparison(
comparator = "In",
left = ExpressionTempVariableRef(
variable = tmp_key_variable.makeReference( result ),
source_ref = source_ref
),
right = kw_variable_ref.makeCloneAt( source_ref ),
source_ref = source_ref
),
yes_branch = makeStatementsSequenceFromStatement(
statement = raise_duplicate
),
no_branch = None,
source_ref = source_ref
),
StatementAssignmentSubscript(
expression = kw_variable_ref.makeCloneAt( source_ref ),
subscript = ExpressionTempVariableRef(
variable = tmp_key_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionSubscriptLookup(
expression = star_dict_variable_ref.makeCloneAt( source_ref ),
subscript = ExpressionTempVariableRef(
variable = tmp_key_variable.makeReference( result ),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
)
mapping_loop_body = StatementsSequence(
statements = statements,
source_ref = source_ref
)
statements = (
# Initializing the temp variable outside of try/except, because code
# generation does not yet detect that case properly. TODO: Can be
# removed once code generation is apt enough.
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_keys_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = None,
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
),
makeTryExceptSingleHandlerNode(
tried = makeStatementsSequenceFromStatement(
statement = StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_keys_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionCallEmpty(
called = ExpressionAttributeLookup(
expression = star_dict_variable_ref.makeCloneAt(
source_ref
),
attribute_name = "keys",
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
),
exception_name = "AttributeError",
handler_body = makeStatementsSequenceFromStatement(
statement = raise_statement
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_iter_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionBuiltinIter1(
value = ExpressionTempVariableRef(
variable = tmp_keys_variable.makeReference( result ),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_dict_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionConstantRef(
constant = {},
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
),
StatementLoop(
body = mapping_loop_body,
source_ref = source_ref
),
)
mapping_case = StatementsSequence(
statements = statements,
source_ref = source_ref
)
temp_scope = result.allocateTempScope( "dict" )
tmp_iter_variable = result.allocateTempVariable( temp_scope, "iter" )
tmp_item_variable = result.allocateTempVariable( temp_scope, "item" )
tmp_key_variable = result.allocateTempVariable( temp_scope, "key" )
# TODO: Duplication from above, just so the other temp is used.
raise_duplicate = StatementRaiseException(
exception_type = ExpressionBuiltinMakeException(
exception_name = "TypeError",
args = (
ExpressionOperationBinary(
operator = "Mod",
left = ExpressionConstantRef(
constant = """\
%s got multiple values for keyword argument '%s'""",
source_ref = source_ref,
user_provided = True
),
right = ExpressionMakeTuple(
elements = (
ExpressionFunctionCall(
function = ExpressionFunctionCreation(
function_ref = ExpressionFunctionRef(
function_body = getCallableNameDescBody(
),
source_ref = source_ref
),
defaults = (),
kw_defaults = None,
annotations = None,
source_ref = source_ref
),
values = (
called_variable_ref.makeCloneAt(
source_ref
),
),
source_ref = source_ref
),
ExpressionTempVariableRef(
variable = tmp_key_variable.makeReference(
result
),
source_ref = source_ref
)
),
source_ref = source_ref
),
source_ref = source_ref
),
),
source_ref = source_ref
),
exception_value = None,
exception_trace = None,
exception_cause = None,
source_ref = source_ref
)
statements = (
makeTryExceptSingleHandlerNode(
tried = makeStatementsSequenceFromStatement(
statement = StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_item_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionBuiltinNext1(
value = ExpressionTempVariableRef(
variable = tmp_iter_variable.makeReference(
result
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
),
exception_name = "StopIteration",
handler_body = makeStatementsSequenceFromStatement(
statement = StatementBreakLoop( source_ref )
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_key_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionSubscriptLookup(
expression = ExpressionTempVariableRef(
variable = tmp_item_variable.makeReference( result ),
source_ref = source_ref
),
subscript = ExpressionConstantRef(
constant = 0,
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
),
source_ref = source_ref
),
StatementConditional(
condition = ExpressionComparison(
comparator = "In",
left = ExpressionTempVariableRef(
variable = tmp_key_variable.makeReference( result ),
source_ref = source_ref
),
right = kw_variable_ref.makeCloneAt( source_ref ),
source_ref = source_ref
),
yes_branch = makeStatementsSequenceFromStatement(
statement = raise_duplicate,
),
no_branch = None,
source_ref = source_ref
),
StatementAssignmentSubscript(
expression = kw_variable_ref.makeCloneAt( source_ref ),
subscript = ExpressionTempVariableRef(
variable = tmp_key_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionSubscriptLookup(
expression = ExpressionTempVariableRef(
variable = tmp_item_variable.makeReference( result ),
source_ref = source_ref
),
subscript = ExpressionConstantRef(
constant = 1,
source_ref = source_ref,
user_provided = True
),
source_ref = source_ref
),
source_ref = source_ref
)
)
dict_loop_body = StatementsSequence(
statements = statements,
source_ref = source_ref
)
statements = (
StatementAssignmentVariable(
variable_ref = kw_target_variable_ref.makeCloneAt( source_ref ),
source = ExpressionBuiltinDict(
pos_arg = kw_variable_ref.makeCloneAt( source_ref ),
pairs = (),
source_ref = source_ref
),
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_iter_variable.makeReference( result ),
source_ref = source_ref
),
source = ExpressionBuiltinIter1(
value = ExpressionCallEmpty(
called = ExpressionAttributeLookup(
expression = star_dict_variable_ref.makeCloneAt(
source_ref
),
attribute_name = "iteritems"
if python_version < 300 else
"items",
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
StatementLoop(
body = dict_loop_body,
source_ref = source_ref
),
)
dict_case = StatementsSequence(
statements = statements,
source_ref = source_ref
)
statements = (
StatementConditional(
condition = star_dict_variable_ref.makeCloneAt( source_ref ),
yes_branch = dict_case,
no_branch = None,
source_ref = source_ref
),
)
dict_case = StatementsSequence(
statements = statements,
source_ref = source_ref
)
return StatementConditional(
condition = ExpressionOperationNOT(
operand = ExpressionBuiltinIsinstance(
instance = star_dict_variable_ref.makeCloneAt( source_ref ),
cls = ExpressionBuiltinRef(
builtin_name = "dict",
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = mapping_case,
no_branch = dict_case,
source_ref = source_ref
)
@once_decorator
def getFunctionCallHelperStarList():
helper_name = "complex_call_helper_star_list"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = ( "called", "star_arg_list" ),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
# Equivalent of:
#
# Note: Call in here is not the same, as it can go without checks directly
# to PyObject_Call.
#
# if not isinstance( star_arg_list, tuple ):
# try:
# star_arg_list = tuple( star_arg_list )
# except TypeError:
# raise TypeError, "%s argument after * must be a sequence, not %s" % (
# get_callable_name_desc( function ),
# type( star_arg_list ).__name__
# )
#
# return called( *star_arg_list )
statements = (
_makeStarListArgumentToTupleStatement(
called_variable_ref = makeCalledVariableRef(),
star_list_variable_ref = makeStarListArgVariableRef(
assign = False
),
star_list_target_variable_ref = makeStarListArgVariableRef(
assign = True
)
),
StatementReturn(
expression = ExpressionCallNoKeywords(
called = makeCalledVariableRef(),
args = makeStarListArgVariableRef( assign = False ),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
@once_decorator
def getFunctionCallHelperKeywordsStarList():
helper_name = "complex_call_helper_keywords_star_list"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = ( "called", "kw", "star_arg_list" ),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
# Equivalent of:
#
# Note: Call in here is not the same, as it can go without checks directly
# to PyObject_Call.
#
# if not isinstance( star_arg_list, tuple ):
# try:
# star_arg_list = tuple( star_arg_list )
# except TypeError:
# raise TypeError, "%s argument after * must be a sequence, not %s" % (
# get_callable_name_desc( function ),
# type( star_arg_list ).__name__
# )
#
# return called( *star_arg_list )
statements = (
_makeStarListArgumentToTupleStatement(
called_variable_ref = makeCalledVariableRef(),
star_list_variable_ref = makeStarListArgVariableRef(
assign = False
),
star_list_target_variable_ref = makeStarListArgVariableRef(
assign = True
)
),
StatementReturn(
expression = ExpressionCall(
called = makeCalledVariableRef(),
args = makeStarListArgVariableRef( assign = False ),
kw = makeKwVariableRef( assign = False),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
@once_decorator
def getFunctionCallHelperPosStarList():
helper_name = "complex_call_helper_pos_star_list"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = ( "called", "args", "star_arg_list" ),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
# Equivalent of:
#
# Note: Call in here is not the same, as it can go without checks directly
# to PyObject_Call.
#
# if not isinstance( star_arg_list, tuple ):
# try:
# star_arg_list = tuple( star_arg_list )
# except TypeError:
# raise TypeError, "%s argument after * must be a sequence, not %s" % (
# get_callable_name_desc( function ),
# type( star_arg_list ).__name__
# )
#
# return called( *star_arg_list )
statements = (
_makeStarListArgumentToTupleStatement(
called_variable_ref = makeCalledVariableRef(),
star_list_variable_ref = makeStarListArgVariableRef(
assign = False
),
star_list_target_variable_ref = makeStarListArgVariableRef(
assign = True
)
),
StatementReturn(
expression = ExpressionCallNoKeywords(
called = makeCalledVariableRef(),
args = ExpressionOperationBinary(
operator = "Add",
left = makeArgsVariableRef(),
right = makeStarListArgVariableRef( assign = False ),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
@once_decorator
def getFunctionCallHelperPosKeywordsStarList():
helper_name = "complex_call_helper_pos_keywords_star_list"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = ( "called", "args", "kw", "star_arg_list" ),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
# Equivalent of:
#
# Note: Call in here is not the same, as it can go without checks directly
# to PyObject_Call.
#
# if not isinstance( star_arg_list, tuple ):
# try:
# star_arg_list = tuple( star_arg_list )
# except TypeError:
# raise TypeError, "%s argument after * must be a sequence, not %s" % (
# get_callable_name_desc( function ),
# type( star_arg_list ).__name__
# )
#
# return called( *star_arg_list )
statements = (
_makeStarListArgumentToTupleStatement(
called_variable_ref = makeCalledVariableRef(),
star_list_variable_ref = makeStarListArgVariableRef(
assign = False
),
star_list_target_variable_ref = makeStarListArgVariableRef(
assign = True
)
),
StatementReturn(
expression = ExpressionCall(
called = makeCalledVariableRef(),
args = ExpressionOperationBinary(
operator = "Add",
left = makeArgsVariableRef(),
right = makeStarListArgVariableRef( assign = False ),
source_ref = source_ref
),
kw = makeKwVariableRef( assign = False ),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
@once_decorator
def getFunctionCallHelperStarDict():
helper_name = "complex_call_helper_star_dict"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = ( "called", "star_arg_dict" ),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
# Equivalent of:
#
# Note: Call in here is not the same, as it can go without checks directly
# to PyObject_Call.
#
# if not isinstance( star_arg_dict, dict ):
# try:
# tmp_keys = star_arg_dict.keys()
# except AttributeError:
# raise TypeError, ""%s argument after ** must be a mapping, not %s" % (
# get_callable_name_desc( function ),
# type( star_arg_dict ).__name__
# )
#
# tmp_iter = iter( keys )
# tmp_dict = {}
#
# while 1:
# try:
# tmp_key = tmp_iter.next()
# except StopIteration:
# break
#
# tmp_dict[ tmp_key ] = star_dict_arg[ tmp_key )
#
# star_arg_dict = new
#
# return called( **star_arg_dict )
statements = (
_makeStarDictArgumentToDictStatement(
result = result,
called_variable_ref = makeCalledVariableRef(),
star_dict_variable_ref = makeStarDictArgVariableRef(
assign = False
),
star_dict_target_variable_ref = makeStarDictArgVariableRef(
assign = True
)
),
StatementReturn(
expression = ExpressionCallKeywordsOnly(
called = makeCalledVariableRef(),
kw = makeStarDictArgVariableRef( assign = False ),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
@once_decorator
def getFunctionCallHelperPosStarDict():
helper_name = "complex_call_helper_pos_star_dict"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = ( "called", "args", "star_arg_dict" ),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
# Equivalent of:
#
# Note: Call in here is not the same, as it can go without checks directly
# to PyObject_Call.
#
# if not isinstance( star_arg_dict, dict ):
# try:
# tmp_keys = star_arg_dict.keys()
# except AttributeError:
# raise TypeError, ""%s argument after ** must be a mapping, not %s" % (
# get_callable_name_desc( function ),
# type( star_arg_dict ).__name__
# )
#
# tmp_iter = iter( keys )
# tmp_dict = {}
#
# while 1:
# try:
# tmp_key = tmp_iter.next()
# except StopIteration:
# break
#
# tmp_dict[ tmp_key ] = star_dict_arg[ tmp_key )
#
# star_arg_dict = new
#
# return called( args, **star_arg_dict )
statements = (
_makeStarDictArgumentToDictStatement(
result = result,
called_variable_ref = makeCalledVariableRef(),
star_dict_variable_ref = makeStarDictArgVariableRef(
assign = False
),
star_dict_target_variable_ref = makeStarDictArgVariableRef(
assign = True
)
),
StatementReturn(
expression = ExpressionCall(
called = makeCalledVariableRef(),
args = makeArgsVariableRef(),
kw = makeStarDictArgVariableRef( assign = False ),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
@once_decorator
def getFunctionCallHelperKeywordsStarDict():
helper_name = "complex_call_helper_keywords_star_dict"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = ( "called", "kw", "star_arg_dict" ),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
# Equivalent of:
#
# Note: Call in here is not the same, as it can go without checks directly
# to PyObject_Call. One goal is to avoid copying "kw" unless really
# necessary, and to take the slow route only for non-dictionaries.
#
# if not isinstance( star_arg_dict, dict ):
# try:
# tmp_keys = star_arg_dict.keys()
# except AttributeError:
# raise TypeError, ""%s argument after ** must be a mapping, not %s" % (
# get_callable_name_desc( function ),
# type( star_arg_dict ).__name__
# )
#
# if keys:
# kw = dict( kw )
#
# tmp_iter = iter( keys )
# tmp_dict = {}
#
# while 1:
# try:
# tmp_key = tmp_iter.next()
# except StopIteration:
# break
#
# if tmp_key in kw:
# raise TypeError, "%s got multiple values for keyword argument '%s'" % (
# get_callable_name_desc( function ),
# tmp_key
# )
#
# kw[ tmp_key ] = star_dict_arg[ tmp_key )
#
# elif star_arg_dict:
# tmp_iter = star_arg_dict.iteritems()
#
# kw = dict( kw )
# while 1:
# try:
# tmp_key, tmp_value = tmp_iter.next()
# except StopIteration:
# break
#
# if tmp_key in kw:
# raise TypeError, "%s got multiple values for keyword argument '%s'" % (
# get_callable_name_desc( function ),
# tmp_key
# )
#
# kw[ tmp_key ] = tmp_value
#
# return called( **kw )
statements = (
_makeStarDictArgumentMergeToKwStatement(
result = result,
called_variable_ref = makeCalledVariableRef(),
kw_variable_ref = makeKwVariableRef( assign = False ),
kw_target_variable_ref = makeKwVariableRef( assign = True ),
star_dict_variable_ref = makeStarDictArgVariableRef(
assign = False
)
),
StatementReturn(
expression = ExpressionCallKeywordsOnly(
called = makeCalledVariableRef(),
kw = makeKwVariableRef( assign = False ),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
@once_decorator
def getFunctionCallHelperPosKeywordsStarDict():
helper_name = "complex_call_helper_pos_keywords_star_dict"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = ( "called", "args", "kw", "star_arg_dict" ),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
# Equivalent of:
#
# Note: Call in here is not the same, as it can go without checks directly
# to PyObject_Call. One goal is to avoid copying "kw" unless really
# necessary, and to take the slow route only for non-dictionaries.
#
# if not isinstance( star_arg_dict, dict ):
# try:
# tmp_keys = star_arg_dict.keys()
# except AttributeError:
# raise TypeError, ""%s argument after ** must be a mapping, not %s" % (
# get_callable_name_desc( function ),
# type( star_arg_dict ).__name__
# )
#
# if keys:
# kw = dict( kw )
#
# tmp_iter = iter( keys )
# tmp_dict = {}
#
# while 1:
# try:
# tmp_key = tmp_iter.next()
# except StopIteration:
# break
#
# if tmp_key in kw:
# raise TypeError, "%s got multiple values for keyword argument '%s'" % (
# get_callable_name_desc( function ),
# tmp_key
# )
#
# kw[ tmp_key ] = star_dict_arg[ tmp_key )
#
# elif star_arg_dict:
# tmp_iter = star_arg_dict.iteritems()
#
# kw = dict( kw )
# while 1:
# try:
# tmp_key, tmp_value = tmp_iter.next()
# except StopIteration:
# break
#
# if tmp_key in kw:
# raise TypeError, "%s got multiple values for keyword argument '%s'" % (
# get_callable_name_desc( function ),
# tmp_key
# )
#
# kw[ tmp_key ] = tmp_value
#
# return called( **kw )
statements = (
_makeStarDictArgumentMergeToKwStatement(
result = result,
called_variable_ref = makeCalledVariableRef(),
kw_variable_ref = makeKwVariableRef( assign = False ),
kw_target_variable_ref = makeKwVariableRef( assign = True ),
star_dict_variable_ref = makeStarDictArgVariableRef(
assign = False
)
),
StatementReturn(
expression = ExpressionCall(
called = makeCalledVariableRef(),
args = makeArgsVariableRef(),
kw = makeKwVariableRef( assign = False ),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
@once_decorator
def getFunctionCallHelperStarListStarDict():
helper_name = "complex_call_helper_star_list_star_dict"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = ( "called", "star_arg_list", "star_arg_dict" ),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
statements = (
_makeStarDictArgumentToDictStatement(
result = result,
called_variable_ref = makeCalledVariableRef(),
star_dict_variable_ref = makeStarDictArgVariableRef(
assign = False
),
star_dict_target_variable_ref = makeStarDictArgVariableRef(
assign = True
)
),
_makeStarListArgumentToTupleStatement(
called_variable_ref = makeCalledVariableRef(),
star_list_variable_ref = makeStarListArgVariableRef(
assign = False
),
star_list_target_variable_ref = makeStarListArgVariableRef(
assign = True
)
),
StatementReturn(
expression = ExpressionCall(
called = makeCalledVariableRef(),
args = makeStarListArgVariableRef( assign = False ),
kw = makeStarDictArgVariableRef( assign = False ),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
@once_decorator
def getFunctionCallHelperPosStarListStarDict():
helper_name = "complex_call_helper_pos_star_list_star_dict"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = (
"called", "args", "star_arg_list", "star_arg_dict"
),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
statements = (
_makeStarDictArgumentToDictStatement(
result = result,
called_variable_ref = makeCalledVariableRef(),
star_dict_variable_ref = makeStarDictArgVariableRef(
assign = False
),
star_dict_target_variable_ref = makeStarDictArgVariableRef(
assign = True
)
),
_makeStarListArgumentToTupleStatement(
called_variable_ref = makeCalledVariableRef(),
star_list_variable_ref = makeStarListArgVariableRef(
assign = False
),
star_list_target_variable_ref = makeStarListArgVariableRef(
assign = True
)
),
StatementReturn(
expression = ExpressionCall(
called = makeCalledVariableRef(),
args = ExpressionOperationBinary(
operator = "Add",
left = makeArgsVariableRef(),
right = makeStarListArgVariableRef( assign = False ),
source_ref = source_ref
),
kw = makeStarDictArgVariableRef( assign = False ),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
@once_decorator
def getFunctionCallHelperKeywordsStarListStarDict():
helper_name = "complex_call_helper_keywords_star_list_star_dict"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = (
"called", "kw", "star_arg_list", "star_arg_dict"
),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
statements = (
_makeStarDictArgumentMergeToKwStatement(
result = result,
called_variable_ref = makeCalledVariableRef(),
kw_variable_ref = makeKwVariableRef( assign = False ),
kw_target_variable_ref = makeKwVariableRef( assign = True ),
star_dict_variable_ref = makeStarDictArgVariableRef(
assign = False
)
),
_makeStarListArgumentToTupleStatement(
called_variable_ref = makeCalledVariableRef(),
star_list_variable_ref = makeStarListArgVariableRef(
assign = False
),
star_list_target_variable_ref = makeStarListArgVariableRef(
assign = True
)
),
StatementReturn(
expression = ExpressionCall(
called = makeCalledVariableRef(),
args = makeStarListArgVariableRef( assign = False ),
kw = makeKwVariableRef( assign = False ),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
@once_decorator
def getFunctionCallHelperPosKeywordsStarListStarDict():
helper_name = "complex_call_helper_pos_keywords_star_list_star_dict"
# Only need to check if the star argument value is a sequence and then
# convert to tuple.
result = ExpressionFunctionBody(
provider = getInternalModule(),
name = helper_name,
doc = None,
parameters = ParameterSpec(
name = helper_name,
normal_args = (
"called", "args", "kw", "star_arg_list", "star_arg_dict"
),
list_star_arg = None,
dict_star_arg = None,
default_count = 0,
kw_only_args = ()
),
source_ref = source_ref,
is_class = False
)
statements = (
_makeStarDictArgumentMergeToKwStatement(
result = result,
called_variable_ref = makeCalledVariableRef(),
kw_variable_ref = makeKwVariableRef( assign = False ),
kw_target_variable_ref = makeKwVariableRef( assign = True ),
star_dict_variable_ref = makeStarDictArgVariableRef(
assign = False
)
),
_makeStarListArgumentToTupleStatement(
called_variable_ref = makeCalledVariableRef(),
star_list_variable_ref = makeStarListArgVariableRef(
assign = False
),
star_list_target_variable_ref = makeStarListArgVariableRef(
assign = True
)
),
StatementReturn(
expression = ExpressionCall(
called = makeCalledVariableRef(),
args = ExpressionOperationBinary(
operator = "Add",
left = makeArgsVariableRef(),
right = makeStarListArgVariableRef( assign = False ),
source_ref = source_ref
),
kw = makeKwVariableRef( assign = False ),
source_ref = source_ref
),
source_ref = source_ref
)
)
result.setBody(
makePseudoFrame(
parameters = result.getParameters(),
statements = statements
)
)
completeVariableClosures( result )
return result
Nuitka-0.5.0.1/nuitka/tree/ReformulationAssignmentStatements.py 0000644 0001750 0001750 00000102345 12265264105 025103 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from nuitka.nodes.VariableRefNodes import (
ExpressionTargetTempVariableRef,
ExpressionTargetVariableRef,
ExpressionTempVariableRef,
ExpressionVariableRef
)
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.TryNodes import StatementTryFinally
from nuitka.nodes.BuiltinIteratorNodes import (
StatementSpecialUnpackCheck,
ExpressionSpecialUnpack,
ExpressionBuiltinIter1,
)
from nuitka.nodes.BuiltinTypeNodes import ExpressionBuiltinList
from nuitka.nodes.SliceNodes import ExpressionSliceObject
from nuitka.nodes.AttributeNodes import ExpressionAttributeLookup
from nuitka.nodes.StatementNodes import StatementsSequence
from nuitka.nodes.ConditionalNodes import StatementConditional
from nuitka.nodes.AssignNodes import (
StatementAssignmentVariable,
StatementAssignmentAttribute,
StatementAssignmentSubscript,
StatementAssignmentSlice,
StatementDelAttribute,
StatementDelSubscript,
StatementDelVariable,
StatementDelSlice,
)
from nuitka.nodes.OperatorNodes import ExpressionOperationBinaryInplace
from nuitka.nodes.ComparisonNodes import ExpressionComparisonIsNOT
from nuitka.nodes.SubscriptNodes import ExpressionSubscriptLookup
from nuitka.nodes.SliceNodes import ExpressionSliceLookup
from .Helpers import (
makeStatementsSequenceFromStatement,
makeStatementsSequenceOrStatement,
makeSequenceCreationOrConstant,
buildNode,
getKind
)
def buildExtSliceNode(provider, node, source_ref):
elements = []
for dim in node.slice.dims:
dim_kind = getKind( dim )
if dim_kind == "Slice":
lower = buildNode( provider, dim.lower, source_ref, True )
upper = buildNode( provider, dim.upper, source_ref, True )
step = buildNode( provider, dim.step, source_ref, True )
element = ExpressionSliceObject(
lower = lower,
upper = upper,
step = step,
source_ref = source_ref
)
elif dim_kind == "Ellipsis":
element = ExpressionConstantRef(
constant = Ellipsis,
source_ref = source_ref,
user_provided = True
)
elif dim_kind == "Index":
element = buildNode(
provider = provider,
node = dim.value,
source_ref = source_ref
)
else:
assert False, dim
elements.append( element )
return makeSequenceCreationOrConstant(
sequence_kind = "tuple",
elements = elements,
source_ref = source_ref
)
def buildAssignmentStatementsFromDecoded( provider, kind, detail, source,
source_ref ):
# This is using many variable names on purpose, so as to give names to the
# unpacked detail values, pylint: disable=R0914
if kind == "Name":
variable_ref = detail
return StatementAssignmentVariable(
variable_ref = variable_ref,
source = source,
source_ref = source_ref
)
elif kind == "Attribute":
lookup_source, attribute_name = detail
return StatementAssignmentAttribute(
expression = lookup_source,
attribute_name = attribute_name,
source = source,
source_ref = source_ref
)
elif kind == "Subscript":
subscribed, subscript = detail
return StatementAssignmentSubscript(
expression = subscribed,
subscript = subscript,
source = source,
source_ref = source_ref
)
elif kind == "Slice":
lookup_source, lower, upper = detail
return StatementAssignmentSlice(
expression = lookup_source,
lower = lower,
upper = upper,
source = source,
source_ref = source_ref
)
elif kind == "Tuple":
temp_scope = provider.allocateTempScope( "tuple_unpack" )
source_iter_var = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "source_iter"
)
statements = [
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = source_iter_var.makeReference( provider ),
source_ref = source_ref
),
source = ExpressionBuiltinIter1(
value = source,
source_ref = source_ref
),
source_ref = source_ref
)
]
element_vars = [
provider.allocateTempVariable(
temp_scope = temp_scope,
name = "element_%d" % ( element_index + 1 )
)
for element_index in
range( len( detail ) )
]
starred = False
for element_index, element in enumerate( detail ):
element_var = element_vars[ element_index ]
if element[0] != "Starred":
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = element_var.makeReference( provider ),
source_ref = source_ref
),
source = ExpressionSpecialUnpack(
value = ExpressionTempVariableRef(
variable = source_iter_var.makeReference(
provider
),
source_ref = source_ref
),
count = element_index + 1,
source_ref = source_ref
),
source_ref = source_ref
)
)
else:
starred = True
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = element_var.makeReference( provider ),
source_ref = source_ref
),
source = ExpressionBuiltinList(
value = ExpressionTempVariableRef(
variable = source_iter_var.makeReference(
provider
),
source_ref = source_ref
),
source_ref = source_ref
),
source_ref = source_ref
)
)
if not starred:
statements.append(
StatementSpecialUnpackCheck(
iterator = ExpressionTempVariableRef(
variable = source_iter_var.makeReference( provider ),
source_ref = source_ref
),
count = len( detail ),
source_ref = source_ref
)
)
for element_index, element in enumerate( detail ):
if element[0] == "Starred":
element = element[1]
element_var = element_vars[ element_index ]
statements.append(
buildAssignmentStatementsFromDecoded(
provider = provider,
kind = element[0],
detail = element[1],
source = ExpressionTempVariableRef(
variable = element_var.makeReference( provider ),
source_ref = source_ref
),
source_ref = source_ref
)
)
final_statements = []
final_statements.append(
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = source_iter_var.makeReference( provider ),
source_ref = source_ref
),
tolerant = True,
source_ref = source_ref
)
)
# TODO: In that order, or reversed.
for element_var in element_vars:
final_statements.append(
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = element_var.makeReference( provider ),
source_ref = source_ref
),
tolerant = True,
source_ref = source_ref
)
)
return StatementTryFinally(
tried = StatementsSequence(
statements = statements,
source_ref = source_ref
),
final = StatementsSequence(
statements = final_statements,
source_ref = source_ref
),
source_ref = source_ref
)
else:
assert False, ( kind, source_ref, detail )
def buildAssignmentStatements( provider, node, source, source_ref,
allow_none = False, temp_provider = None ):
if node is None and allow_none:
return None
if temp_provider is None:
temp_provider = provider
kind, detail = decodeAssignTarget(
provider = provider,
node = node,
source_ref = source_ref
)
return buildAssignmentStatementsFromDecoded(
provider = temp_provider,
kind = kind,
detail = detail,
source = source,
source_ref = source_ref
)
def decodeAssignTarget(provider, node, source_ref, allow_none = False):
# Many cases to deal with, because of the different assign targets,
# pylint: disable=R0911,R0912
if node is None and allow_none:
return None
if hasattr( node, "ctx" ):
assert getKind( node.ctx ) in ( "Store", "Del" )
kind = getKind( node )
if type( node ) is str:
return "Name", ExpressionTargetVariableRef(
variable_name = node,
source_ref = source_ref
)
elif kind == "Name":
return kind, ExpressionTargetVariableRef(
variable_name = node.id,
source_ref = source_ref
)
elif kind == "Attribute":
return kind, (
buildNode( provider, node.value, source_ref ),
node.attr
)
elif kind == "Subscript":
slice_kind = getKind( node.slice )
if slice_kind == "Index":
return "Subscript", (
buildNode( provider, node.value, source_ref ),
buildNode( provider, node.slice.value, source_ref )
)
elif slice_kind == "Slice":
lower = buildNode( provider, node.slice.lower, source_ref, True )
upper = buildNode( provider, node.slice.upper, source_ref, True )
if node.slice.step is not None:
step = buildNode( provider, node.slice.step, source_ref )
return "Subscript", (
buildNode( provider, node.value, source_ref ),
ExpressionSliceObject(
lower = lower,
upper = upper,
step = step,
source_ref = source_ref
)
)
else:
return "Slice", (
buildNode( provider, node.value, source_ref ),
lower,
upper
)
elif slice_kind == "ExtSlice":
return "Subscript", (
buildNode( provider, node.value, source_ref ),
buildExtSliceNode( provider, node, source_ref )
)
elif slice_kind == "Ellipsis":
return "Subscript", (
buildNode( provider, node.value, source_ref ),
ExpressionConstantRef(
constant = Ellipsis,
source_ref = source_ref
)
)
else:
assert False, slice_kind
elif kind in ( "Tuple", "List" ):
return "Tuple", tuple(
decodeAssignTarget(
provider = provider,
node = sub_node,
source_ref = source_ref,
allow_none = False
)
for sub_node in
node.elts
)
elif kind == "Starred":
return "Starred", decodeAssignTarget(
provider = provider,
node = node.value,
source_ref = source_ref,
allow_none = False
)
else:
assert False, ( source_ref, kind )
def buildAssignNode(provider, node, source_ref):
assert len( node.targets ) >= 1, source_ref
# Evaluate the right hand side first, so it can get names provided
# before the left hand side exists.
source = buildNode( provider, node.value, source_ref )
if len( node.targets ) == 1:
# Simple assignment case, one source, one target.
return buildAssignmentStatements(
provider = provider,
node = node.targets[0],
source = source,
source_ref = source_ref
)
else:
# Complex assignment case, one source, but multiple targets. We keep the
# source in a temporary variable, and then assign from it multiple
# times.
temp_scope = provider.allocateTempScope( "assign_unpack" )
tmp_source = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "assign_source"
)
statements = [
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_source.makeReference( provider ),
source_ref = source_ref
),
source = source,
source_ref = source_ref
)
]
for target in node.targets:
statements.append(
buildAssignmentStatements(
provider = provider,
node = target,
source = ExpressionTempVariableRef(
variable = tmp_source.makeReference( provider ),
source_ref = source_ref
),
source_ref = source_ref
)
)
statements.append(
StatementDelVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_source.makeReference( provider ),
source_ref = source_ref
),
tolerant = False,
source_ref = source_ref
)
)
return StatementsSequence(
statements = statements,
source_ref = source_ref
)
def buildDeleteStatementFromDecoded(kind, detail, source_ref):
if kind in ("Name", "Name_Exception"):
# Note: Name_Exception is a "del" for exception handlers that doesn't
# insist on the variable being defined, user code may do it too, and
# that will be fine, so make that tolerant.
variable_ref = detail
return StatementDelVariable(
variable_ref = variable_ref,
tolerant = kind == "Name_Exception",
source_ref = source_ref
)
elif kind == "Attribute":
lookup_source, attribute_name = detail
return StatementDelAttribute(
expression = lookup_source,
attribute_name = attribute_name,
source_ref = source_ref
)
elif kind == "Subscript":
subscribed, subscript = detail
return StatementDelSubscript(
expression = subscribed,
subscript = subscript,
source_ref = source_ref
)
elif kind == "Slice":
lookup_source, lower, upper = detail
return StatementDelSlice(
expression = lookup_source,
lower = lower,
upper = upper,
source_ref = source_ref
)
elif kind == "Tuple":
result = []
for sub_node in detail:
result.append(
buildDeleteStatementFromDecoded(
kind = sub_node[0],
detail = sub_node[1],
source_ref = source_ref
)
)
return makeStatementsSequenceOrStatement(
statements = result,
source_ref = source_ref
)
else:
assert False, ( kind, detail, source_ref )
def buildDeleteNode(provider, node, source_ref):
# Build del statements.
# Note: Each delete is sequential. It can succeed, and the failure of a
# later one does not prevent the former to succeed. We can therefore have a
# simple sequence of del statements that each only delete one thing
# therefore. In output tree for optimization "del" therefore only ever has
# single arguments.
statements = []
for target in node.targets:
kind, detail = decodeAssignTarget(
provider = provider,
node = target,
source_ref = source_ref
)
statements.append(
buildDeleteStatementFromDecoded(
kind = kind,
detail = detail,
source_ref = source_ref
)
)
return makeStatementsSequenceOrStatement(
statements = statements,
source_ref = source_ref
)
def _buildInplaceAssignVariableNode( provider, variable_ref, tmp_variable1,
tmp_variable2, operator, expression,
source_ref ):
assert variable_ref.isExpressionTargetVariableRef(), variable_ref
return (
# First assign the target value to a temporary variable.
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
source = ExpressionVariableRef(
variable_name = variable_ref.getVariableName(),
source_ref = source_ref
),
source_ref = source_ref
),
# Second assign the inplace result to a temporary variable.
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
),
source = ExpressionOperationBinaryInplace(
operator = operator,
left = ExpressionTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
right = expression,
source_ref = source_ref
),
source_ref = source_ref
),
# Copy it over, if the reference values change, i.e. IsNot is true.
StatementConditional(
condition = ExpressionComparisonIsNOT(
left = ExpressionTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
right = ExpressionTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = makeStatementsSequenceFromStatement(
statement = StatementAssignmentVariable(
variable_ref = variable_ref.makeCloneAt( source_ref ),
source = ExpressionTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
),
source_ref = source_ref
)
),
no_branch = None,
source_ref = source_ref
)
)
def _buildInplaceAssignAttributeNode( provider, lookup_source, attribute_name,
tmp_variable1, tmp_variable2, operator,
expression, source_ref ):
return (
# First assign the target value to a temporary variable.
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
source = ExpressionAttributeLookup(
expression = lookup_source.makeCloneAt( source_ref ),
attribute_name = attribute_name,
source_ref = source_ref
),
source_ref = source_ref
),
# Second assign the inplace result to a temporary variable.
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
),
source = ExpressionOperationBinaryInplace(
operator = operator,
left = ExpressionTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
right = expression,
source_ref = source_ref
),
source_ref = source_ref
),
# Copy it over, if the reference values change, i.e. IsNot is true.
StatementConditional(
condition = ExpressionComparisonIsNOT(
left = ExpressionTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
right = ExpressionTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
),
source_ref = source_ref
),
yes_branch = makeStatementsSequenceFromStatement(
statement = StatementAssignmentAttribute(
expression = lookup_source.makeCloneAt( source_ref ),
attribute_name = attribute_name,
source = ExpressionTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
),
source_ref = source_ref
)
),
no_branch = None,
source_ref = source_ref
)
)
def _buildInplaceAssignSubscriptNode( provider, subscribed, subscript,
tmp_variable1, tmp_variable2, operator,
expression, source_ref ):
return (
# First assign the target value and subscript to temporary variables.
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
source = subscribed,
source_ref = source_ref
),
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
),
source = subscript,
source_ref = source_ref
),
# Second assign the inplace result over the original value.
StatementAssignmentSubscript(
expression = ExpressionTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
subscript = ExpressionTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
),
source = ExpressionOperationBinaryInplace(
operator = operator,
left = ExpressionSubscriptLookup(
expression = ExpressionTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
subscript = ExpressionTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
),
source_ref = source_ref
),
right = expression,
source_ref = source_ref
),
source_ref = source_ref
)
)
def _buildInplaceAssignSliceNode( provider, lookup_source, lower, upper,
tmp_variable1, tmp_variable2, tmp_variable3,
operator, expression, source_ref ):
# First assign the target value, lower and upper to temporary variables.
statements = [
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
source = lookup_source,
source_ref = source_ref
)
]
if lower is not None:
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
),
source = lower,
source_ref = source_ref
)
)
lower_ref1 = ExpressionTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
)
lower_ref2 = ExpressionTempVariableRef(
variable = tmp_variable2.makeReference( provider ),
source_ref = source_ref
)
else:
assert tmp_variable2 is None
lower_ref1 = lower_ref2 = None
if upper is not None:
statements.append(
StatementAssignmentVariable(
variable_ref = ExpressionTargetTempVariableRef(
variable = tmp_variable3.makeReference( provider ),
source_ref = source_ref
),
source = upper,
source_ref = source_ref
)
)
upper_ref1 = ExpressionTempVariableRef(
variable = tmp_variable3.makeReference( provider ),
source_ref = source_ref
)
upper_ref2 = ExpressionTempVariableRef(
variable = tmp_variable3.makeReference( provider ),
source_ref = source_ref
)
else:
assert tmp_variable3 is None
upper_ref1 = upper_ref2 = None
# Second assign the inplace result over the original value.
statements.append(
StatementAssignmentSlice(
expression = ExpressionTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
lower = lower_ref1,
upper = upper_ref1,
source = ExpressionOperationBinaryInplace(
operator = operator,
left = ExpressionSliceLookup(
expression = ExpressionTempVariableRef(
variable = tmp_variable1.makeReference( provider ),
source_ref = source_ref
),
lower = lower_ref2,
upper = upper_ref2,
source_ref = source_ref
),
right = expression,
source_ref = source_ref
),
source_ref = source_ref
)
)
return statements
def buildInplaceAssignNode(provider, node, source_ref):
# There are many inplace assignment variables, and the detail is unpacked
# into names, so we end up with a lot of variables, which is on purpose,
# pylint: disable=R0914
operator = getKind( node.op )
if operator == "Div" and source_ref.getFutureSpec().isFutureDivision():
operator = "TrueDiv"
expression = buildNode( provider, node.value, source_ref )
kind, detail = decodeAssignTarget(
provider = provider,
node = node.target,
source_ref = source_ref
)
temp_scope = provider.allocateTempScope( "inplace_assign" )
if kind == "Name":
variable_ref = detail
tmp_variable1 = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "inplace_start"
)
tmp_variable2 = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "inplace_end"
)
statements = _buildInplaceAssignVariableNode(
provider = provider,
variable_ref = variable_ref,
tmp_variable1 = tmp_variable1,
tmp_variable2 = tmp_variable2,
operator = operator,
expression = expression,
source_ref = source_ref
)
elif kind == "Attribute":
lookup_source, attribute_name = detail
tmp_variable1 = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "inplace_start"
)
tmp_variable2 = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "inplace_end"
)
statements = _buildInplaceAssignAttributeNode(
provider = provider,
lookup_source = lookup_source,
attribute_name = attribute_name,
tmp_variable1 = tmp_variable1,
tmp_variable2 = tmp_variable2,
operator = operator,
expression = expression,
source_ref = source_ref
)
elif kind == "Subscript":
subscribed, subscript = detail
tmp_variable1 = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "inplace_target"
)
tmp_variable2 = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "inplace_subscript"
)
statements = _buildInplaceAssignSubscriptNode(
provider = provider,
subscribed = subscribed,
subscript = subscript,
tmp_variable1 = tmp_variable1,
tmp_variable2 = tmp_variable2,
operator = operator,
expression = expression,
source_ref = source_ref
)
elif kind == "Slice":
lookup_source, lower, upper = detail
tmp_variable1 = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "inplace_target"
)
if lower is not None:
tmp_variable2 = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "inplace_lower"
)
else:
tmp_variable2 = None
if upper is not None:
tmp_variable3 = provider.allocateTempVariable(
temp_scope = temp_scope,
name = "inplace_upper"
)
else:
tmp_variable3 = None
statements = _buildInplaceAssignSliceNode(
provider = provider,
lookup_source = lookup_source,
lower = lower,
upper = upper,
tmp_variable1 = tmp_variable1,
tmp_variable2 = tmp_variable2,
tmp_variable3 = tmp_variable3,
operator = operator,
expression = expression,
source_ref = source_ref
)
else:
assert False, kind
return StatementsSequence(
statements = statements,
source_ref = source_ref
)
Nuitka-0.5.0.1/nuitka/tree/ImportCache.py 0000644 0001750 0001750 00000004457 12265264105 020357 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Import cache.
This is not about caching the search of modules in the file system, but about
maintaining a cache of module trees built.
It can happen that modules become unused, and then droped from active modules,
and then later active again, via another import, and in this case, we should
not start anew.
"""
from nuitka import Utils
from logging import warning
imported_modules = {}
imported_by_name = {}
def addImportedModule(module_relpath, imported_module):
if ( module_relpath, "__main__" ) in imported_modules:
warning( """\
Re-importing __main__ module via its filename duplicates the module."""
)
key = module_relpath, imported_module.getFullName()
if key in imported_modules:
assert imported_module is imported_modules[ key ], key
imported_modules[ key ] = imported_module
imported_by_name[ imported_module.getFullName() ] = imported_module
def isImportedModuleByPath(module_relpath):
module_name = Utils.basename( module_relpath )
if module_name.endswith( ".py" ):
module_name = module_name[:-3]
key = module_relpath, module_name
return key in imported_modules
def isImportedModuleByName(full_name):
return full_name in imported_by_name
def getImportedModuleByName(full_name):
return imported_by_name[ full_name ]
def getImportedModuleByPath(module_relpath):
module_name = Utils.basename( module_relpath )
if module_name.endswith( ".py" ):
module_name = module_name[:-3]
key = module_relpath, module_name
return imported_modules[ key ]
Nuitka-0.5.0.1/nuitka/__past__.py 0000644 0001750 0001750 00000003250 12265270763 016762 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Module like __future__ for things that are no more in CPython3,
but provide compatible fallbacks.
This is required to run the same code easily with both CPython2 and CPython3.
"""
# pylint: disable=W0622
# Work around for CPython 3.x renaming long to int.
try:
long = long # lint:ok
except NameError:
long = int # lint:ok
# Work around for CPython 3.x renaming unicode to str.
try:
unicode = unicode # lint:ok
except NameError:
unicode = str # lint:ok
# Work around for CPython 3.x removal of commands
try:
import commands
except ImportError:
# false alarm, no re-import, just another try if above fails, which it will
# on Python3 pylint: disable=W0404
import subprocess as commands # lint:ok
def iterItems(d):
try:
return d.iteritems()
except AttributeError:
return d.items()
# For PyLint to be happy.
assert long
assert unicode
assert commands
Nuitka-0.5.0.1/nuitka/__init__.py 0000644 0001750 0001750 00000001501 12265264105 016744 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Dummy file to make this directory a package. """
Nuitka-0.5.0.1/nuitka/PythonOperators.py 0000644 0001750 0001750 00000005436 12265264105 020400 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Python operator tables
These are mostly used to resolve the operator in the module operator and to know the list
of operations allowed.
"""
from .Utils import python_version
import operator
if python_version >= 300:
operator.div = operator.truediv
operator.idiv = operator.itruediv
binary_operator_functions = {
"Add" : operator.add,
"Sub" : operator.sub,
"Pow" : operator.pow,
"Mult" : operator.mul,
"Div" : operator.div,
"FloorDiv" : operator.floordiv,
"TrueDiv" : operator.truediv,
"Mod" : operator.mod,
"LShift" : operator.lshift,
"RShift" : operator.rshift,
"BitAnd" : operator.and_,
"BitOr" : operator.or_,
"BitXor" : operator.xor,
"IAdd" : operator.iadd,
"ISub" : operator.isub,
"IPow" : operator.ipow,
"IMult" : operator.imul,
"IDiv" : operator.idiv,
"IFloorDiv" : operator.ifloordiv,
"ITrueDiv" : operator.itruediv,
"IMod" : operator.imod,
"ILShift" : operator.ilshift,
"IRShift" : operator.irshift,
"IBitAnd" : operator.iand,
"IBitOr" : operator.ior,
"IBitXor" : operator.ixor,
}
unary_operator_functions = {
"UAdd" : operator.pos,
"USub" : operator.neg,
"Invert" : operator.invert,
"Repr" : repr,
# Boolean not is treated an unary operator.
"Not" : operator.not_,
}
rich_comparison_functions = {
"Lt" : operator.lt,
"LtE" : operator.le,
"Eq" : operator.eq,
"NotEq" : operator.ne,
"Gt" : operator.gt,
"GtE" : operator.ge
}
other_comparison_functions = {
"Is" : operator.is_,
"IsNot" : operator.is_not,
"In" : lambda value1, value2: value1 in value2,
"NotIn" : lambda value1, value2: value1 not in value2
}
comparison_inversions = {
"Is" : "IsNot",
"IsNot" : "Is",
"In" : "NotIn",
"NotIn" : "In"
}
all_comparison_functions = dict( rich_comparison_functions)
all_comparison_functions.update( other_comparison_functions )
Nuitka-0.5.0.1/nuitka/optimizations/ 0000755 0001750 0001750 00000000000 12265271051 017545 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/optimizations/Tags.py 0000644 0001750 0001750 00000004075 12265264105 021025 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Tags and set of it.
Used by optimization to keep track of the current state of optimization, these
tags trigger the execution of optimization steps, which in turn may emit these
tags to execute other steps.
"""
allowed_tags = (
# New code means new statements.
# Could be a new module, or an inlined exec statement.
"new_code",
# Added new import.
"new_import",
# New statements added.
"new_statements",
# New expression added.
"new_expression",
# TODO: A bit unclear what this it, potentially a changed variable.
"var_usage",
# Detected module variable to be read only.
"read_only_mvar",
# New builtin function detected.
"new_builtin",
# New raise statement detected.
"new_raise",
# New constant introduced.
"new_constant",
)
class TagSet(set):
# false alarm, pylint: disable=R0924
def onSignal(self, signal):
if type(signal) is str:
signal = signal.split()
for tag in signal:
self.add(tag)
def check(self, tags):
for tag in tags.split():
assert tag in allowed_tags, tag
if tag in self:
return True
else:
return False
def add(self, tag):
assert tag in allowed_tags, tag
set.add(self, tag)
Nuitka-0.5.0.1/nuitka/optimizations/OptimizeBuiltinCalls.py 0000644 0001750 0001750 00000067442 12265264105 024244 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Optimize calls to builtins reference builtin nodes.
For builtin name references, we check if it's one of the supported builtin
types.
"""
from nuitka.Utils import python_version
from nuitka.Options import isDebug, shallMakeModule
from nuitka.nodes.BuiltinIteratorNodes import (
ExpressionBuiltinNext1,
ExpressionBuiltinNext2,
ExpressionBuiltinIter1,
ExpressionBuiltinIter2,
ExpressionBuiltinLen
)
from nuitka.nodes.BuiltinTypeNodes import (
ExpressionBuiltinFloat,
ExpressionBuiltinTuple,
ExpressionBuiltinList,
ExpressionBuiltinBool,
ExpressionBuiltinInt,
ExpressionBuiltinStr,
ExpressionBuiltinSet
)
from nuitka.nodes.BuiltinFormatNodes import (
ExpressionBuiltinBin,
ExpressionBuiltinOct,
ExpressionBuiltinHex,
)
from nuitka.nodes.BuiltinDecodingNodes import (
ExpressionBuiltinChr,
ExpressionBuiltinOrd,
ExpressionBuiltinOrd0
)
from nuitka.nodes.ExecEvalNodes import ExpressionBuiltinEval
from nuitka.nodes.VariableRefNodes import (
ExpressionTempVariableRef,
ExpressionVariableRef
)
from nuitka.nodes.GlobalsLocalsNodes import (
ExpressionBuiltinGlobals,
ExpressionBuiltinLocals,
ExpressionBuiltinDir0,
ExpressionBuiltinDir1
)
from nuitka.nodes.OperatorNodes import ExpressionOperationUnary
from nuitka.nodes.ConstantRefNodes import ExpressionConstantRef
from nuitka.nodes.BuiltinDictNodes import ExpressionBuiltinDict
from nuitka.nodes.BuiltinOpenNodes import ExpressionBuiltinOpen
from nuitka.nodes.BuiltinRangeNodes import (
ExpressionBuiltinRange0,
ExpressionBuiltinRange1,
ExpressionBuiltinRange2,
ExpressionBuiltinRange3
)
from nuitka.nodes.BuiltinVarsNodes import ExpressionBuiltinVars
from nuitka.nodes.ImportNodes import ExpressionBuiltinImport
from nuitka.nodes.TypeNodes import (
ExpressionBuiltinSuper,
ExpressionBuiltinType1,
ExpressionBuiltinIsinstance
)
from nuitka.nodes.ClassNodes import ExpressionBuiltinType3
from nuitka.nodes.CallNodes import ExpressionCallEmpty
from nuitka.nodes.AttributeNodes import (
ExpressionAttributeLookup,
ExpressionBuiltinGetattr,
ExpressionBuiltinSetattr,
ExpressionBuiltinHasattr
)
from nuitka.nodes.ConditionalNodes import ExpressionConditional
from nuitka.nodes.ComparisonNodes import ExpressionComparisonIs
from nuitka.tree.ReformulationExecStatements import wrapEvalGlobalsAndLocals
from . import BuiltinOptimization
def dir_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinDir1,
builtin_spec = BuiltinOptimization.builtin_dir_spec,
empty_special_class = ExpressionBuiltinDir0
)
def vars_extractor(node):
def selectVarsEmptyClass(source_ref):
if node.getParentVariableProvider().isPythonModule():
return ExpressionBuiltinGlobals(
source_ref = source_ref
)
else:
return ExpressionBuiltinLocals(
source_ref = source_ref
)
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinVars,
builtin_spec = BuiltinOptimization.builtin_vars_spec,
empty_special_class = selectVarsEmptyClass
)
def import_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinImport,
builtin_spec = BuiltinOptimization.builtin_import_spec
)
def type_extractor(node):
args = node.getCallArgs()
length = args.getIterationLength()
if length == 1:
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinType1,
builtin_spec = BuiltinOptimization.builtin_type1_spec
)
else:
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinType3,
builtin_spec = BuiltinOptimization.builtin_type3_spec
)
def iter_extractor(node):
# Note: Iter in fact names its first argument if the default applies
# "collection", but it won't matter much, fixed up in a wrapper. The
# "callable" is part of the API, pylint: disable=W0622
def wrapIterCreation(callable, sentinel, source_ref):
if sentinel is None:
return ExpressionBuiltinIter1(
value = callable,
source_ref = source_ref
)
else:
return ExpressionBuiltinIter2(
callable = callable,
sentinel = sentinel,
source_ref = source_ref
)
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = wrapIterCreation,
builtin_spec = BuiltinOptimization.builtin_iter_spec
)
def next_extractor(node):
# Split up next with and without defaults, they are not going to behave
# really very similar.
def selectNextBuiltinClass(iterator, default, source_ref):
if default is None:
return ExpressionBuiltinNext1(
value = iterator,
source_ref = source_ref
)
else:
return ExpressionBuiltinNext2(
iterator = iterator,
default = default,
source_ref = source_ref
)
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = selectNextBuiltinClass,
builtin_spec = BuiltinOptimization.builtin_iter_spec
)
def dict_extractor(node):
# The dict is a bit strange in that it accepts a position parameter, or not,
# but won't have a default.
def wrapExpressionBuiltinDictCreation( positional_args, dict_star_arg,
source_ref ):
if len( positional_args ) > 1:
from nuitka.nodes.NodeMakingHelpers import (
makeRaiseExceptionReplacementExpressionFromInstance,
wrapExpressionWithSideEffects
)
result = makeRaiseExceptionReplacementExpressionFromInstance(
expression = node,
exception = TypeError(
"dict expected at most 1 arguments, got %d" % (
len( positional_args )
)
)
)
result = wrapExpressionWithSideEffects(
side_effects = positional_args,
old_node = node,
new_node = result
)
if dict_star_arg:
result = wrapExpressionWithSideEffects(
side_effects = dict_star_arg,
old_node = node,
new_node = result
)
return result
return ExpressionBuiltinDict(
pos_arg = positional_args[0] if positional_args else None,
pairs = dict_star_arg,
source_ref = source_ref
)
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = wrapExpressionBuiltinDictCreation,
builtin_spec = BuiltinOptimization.builtin_dict_spec
)
def chr_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinChr,
builtin_spec = BuiltinOptimization.builtin_chr_spec
)
def ord_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinOrd,
builtin_spec = BuiltinOptimization.builtin_ord_spec,
empty_special_class = ExpressionBuiltinOrd0
)
def bin_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinBin,
builtin_spec = BuiltinOptimization.builtin_bin_spec
)
def oct_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinOct,
builtin_spec = BuiltinOptimization.builtin_oct_spec
)
def hex_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinHex,
builtin_spec = BuiltinOptimization.builtin_hex_spec
)
def repr_extractor(node):
def makeReprOperator(operand, source_ref):
return ExpressionOperationUnary(
operator = "Repr",
operand = operand,
source_ref = source_ref
)
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = makeReprOperator,
builtin_spec = BuiltinOptimization.builtin_repr_spec
)
def range_extractor(node):
def selectRangeBuiltin(low, high, step, source_ref):
if high is None:
return ExpressionBuiltinRange1( low, source_ref )
elif step is None:
return ExpressionBuiltinRange2( low, high, source_ref )
else:
return ExpressionBuiltinRange3( low, high, step, source_ref )
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = selectRangeBuiltin,
builtin_spec = BuiltinOptimization.builtin_range_spec,
empty_special_class = ExpressionBuiltinRange0
)
if python_version < 300:
from nuitka.nodes.BuiltinRangeNodes import ExpressionBuiltinXrange
def xrange_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinXrange,
builtin_spec = BuiltinOptimization.builtin_xrange_spec
)
def len_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinLen,
builtin_spec = BuiltinOptimization.builtin_len_spec
)
def tuple_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinTuple,
builtin_spec = BuiltinOptimization.builtin_tuple_spec
)
def list_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinList,
builtin_spec = BuiltinOptimization.builtin_list_spec
)
def set_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinSet,
builtin_spec = BuiltinOptimization.builtin_set_spec
)
def float_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinFloat,
builtin_spec = BuiltinOptimization.builtin_float_spec
)
def str_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinStr,
builtin_spec = BuiltinOptimization.builtin_str_spec
)
if python_version < 300:
from nuitka.nodes.BuiltinTypeNodes import ExpressionBuiltinUnicode
def unicode_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinUnicode,
builtin_spec = BuiltinOptimization.builtin_unicode_spec
)
def bool_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinBool,
builtin_spec = BuiltinOptimization.builtin_bool_spec
)
def int_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinInt,
builtin_spec = BuiltinOptimization.builtin_int_spec
)
if python_version < 300:
from nuitka.nodes.BuiltinTypeNodes import ExpressionBuiltinLong
def long_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinLong,
builtin_spec = BuiltinOptimization.builtin_long_spec
)
def globals_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinGlobals,
builtin_spec = BuiltinOptimization.builtin_globals_spec
)
def locals_extractor(node):
# Note: Locals on the module level is really globals.
provider = node.getParentVariableProvider()
if provider.isPythonModule():
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinGlobals,
builtin_spec = BuiltinOptimization.builtin_locals_spec
)
else:
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinLocals,
builtin_spec = BuiltinOptimization.builtin_locals_spec
)
if python_version < 300:
from nuitka.nodes.ExecEvalNodes import ExpressionBuiltinExecfile
def execfile_extractor(node):
# Need to accept globals and local keyword argument, that is just the
# API of execfile, pylint: disable=W0622
def wrapExpressionBuiltinExecfileCreation( filename, globals, locals,
source_ref ):
provider = node.getParentVariableProvider()
# TODO: Can't really be true, can it?
if provider.isExpressionFunctionBody():
provider.markAsExecContaining()
if provider.isClassDictCreation():
provider.markAsUnqualifiedExecContaining( source_ref )
globals_wrap, locals_wrap = wrapEvalGlobalsAndLocals(
provider = provider,
globals_node = globals,
locals_node = locals,
exec_mode = False,
source_ref = source_ref
)
return ExpressionBuiltinExecfile(
source_code = ExpressionCallEmpty(
called = ExpressionAttributeLookup(
expression = ExpressionBuiltinOpen(
filename = filename,
mode = ExpressionConstantRef(
constant = "rU",
source_ref = source_ref
),
buffering = None,
source_ref = source_ref
),
attribute_name = "read",
source_ref = source_ref
),
source_ref = source_ref
),
globals_arg = globals_wrap,
locals_arg = locals_wrap,
source_ref = source_ref
)
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = wrapExpressionBuiltinExecfileCreation,
builtin_spec = BuiltinOptimization.builtin_execfile_spec
)
def eval_extractor(node):
# Need to accept globals and local keyword argument, that is just the API of
# eval, pylint: disable=W0622
def wrapEvalBuiltin(source, globals, locals, source_ref):
globals_wrap, locals_wrap = wrapEvalGlobalsAndLocals(
provider = node.getParentVariableProvider(),
globals_node = globals,
locals_node = locals,
exec_mode = False,
source_ref = source_ref
)
return ExpressionBuiltinEval(
source_code = source,
globals_arg = globals_wrap,
locals_arg = locals_wrap,
source_ref = source_ref
)
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = wrapEvalBuiltin,
builtin_spec = BuiltinOptimization.builtin_eval_spec
)
if python_version >= 300:
from nuitka.nodes.ExecEvalNodes import ExpressionBuiltinExec
def exec_extractor(node):
# Need to accept globals and local keyword argument, that is just the
# API of exec, pylint: disable=W0622
def wrapExpressionBuiltinExecCreation( source, globals, locals,
source_ref ):
provider = node.getParentVariableProvider()
# TODO: Can't really be true, can it?
if provider.isExpressionFunctionBody():
provider.markAsExecContaining()
if provider.isClassDictCreation():
provider.markAsUnqualifiedExecContaining( source_ref )
globals_wrap, locals_wrap = wrapEvalGlobalsAndLocals(
provider = provider,
globals_node = globals,
locals_node = locals,
exec_mode = False,
source_ref = source_ref
)
return ExpressionBuiltinExec(
source_code = source,
globals_arg = globals_wrap,
locals_arg = locals_wrap,
source_ref = source_ref
)
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = wrapExpressionBuiltinExecCreation,
builtin_spec = BuiltinOptimization.builtin_eval_spec
)
def open_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinOpen,
builtin_spec = BuiltinOptimization.builtin_open_spec
)
def super_extractor(node):
# Need to accept type and object as keyword argument, that is just the API
# of super, pylint: disable=W0622
def wrapSuperBuiltin(type, object, source_ref):
if type is None and python_version >= 300:
provider = node.getParentVariableProvider()
if python_version < 340:
type = ExpressionVariableRef(
variable_name = "__class__",
source_ref = source_ref
)
# Ought to be already closure taken.
type.setVariable(
provider.getVariableForClosure(
variable_name = "__class__"
)
)
if not type.getVariable().isClosureReference():
type = None
else:
parent_provider = provider.getParentVariableProvider()
class_var = parent_provider.getTempVariable(
temp_scope = None,
name = "__class__"
)
type = ExpressionTempVariableRef(
variable = class_var.makeReference( parent_provider ).makeReference(provider),
source_ref = source_ref
)
from nuitka.nodes.NodeMakingHelpers import \
makeRaiseExceptionReplacementExpression
if type is None:
return makeRaiseExceptionReplacementExpression(
expression = node,
exception_type = "SystemError"
if python_version < 331 else
"RuntimeError",
exception_value = "super(): __class__ cell not found",
)
if object is None:
if provider.getParameters().getArgumentCount() > 0:
par1_name = provider.getParameters().getArgumentNames()[0]
# TODO: Nested first argument would kill us here, need a
# test for that.
object = ExpressionVariableRef(
variable_name = par1_name,
source_ref = source_ref
)
object.setVariable(
provider.getVariableForReference(
variable_name = par1_name
)
)
if not object.getVariable().isParameterVariable():
return makeRaiseExceptionReplacementExpression(
expression = node,
exception_type = "SystemError"
if python_version < 330 else
"RuntimeError",
exception_value = "super(): __class__ cell not found",
)
else:
return makeRaiseExceptionReplacementExpression(
expression = node,
exception_type = "RuntimeError",
exception_value = "super(): no arguments"
)
return ExpressionBuiltinSuper(
super_type = type,
super_object = object,
source_ref = source_ref
)
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = wrapSuperBuiltin,
builtin_spec = BuiltinOptimization.builtin_super_spec
)
def hasattr_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinHasattr,
builtin_spec = BuiltinOptimization.builtin_hasattr_spec
)
def getattr_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinGetattr,
builtin_spec = BuiltinOptimization.builtin_getattr_spec
)
def setattr_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinSetattr,
builtin_spec = BuiltinOptimization.builtin_setattr_spec
)
def isinstance_extractor(node):
return BuiltinOptimization.extractBuiltinArgs(
node = node,
builtin_class = ExpressionBuiltinIsinstance,
builtin_spec = BuiltinOptimization.builtin_isinstance_spec
)
_dispatch_dict = {
"globals" : globals_extractor,
"locals" : locals_extractor,
"eval" : eval_extractor,
"dir" : dir_extractor,
"vars" : vars_extractor,
"__import__" : import_extractor,
"chr" : chr_extractor,
"ord" : ord_extractor,
"bin" : bin_extractor,
"oct" : oct_extractor,
"hex" : hex_extractor,
"type" : type_extractor,
"iter" : iter_extractor,
"next" : next_extractor,
"range" : range_extractor,
"tuple" : tuple_extractor,
"list" : list_extractor,
"dict" : dict_extractor,
"set" : set_extractor,
"float" : float_extractor,
"str" : str_extractor,
"bool" : bool_extractor,
"int" : int_extractor,
"repr" : repr_extractor,
"len" : len_extractor,
"super" : super_extractor,
"hasattr" : hasattr_extractor,
"getattr" : getattr_extractor,
"setattr" : setattr_extractor,
"isinstance" : isinstance_extractor
}
if python_version < 300:
_dispatch_dict[ "long" ] = long_extractor
_dispatch_dict[ "unicode" ] = unicode_extractor
_dispatch_dict[ "execfile" ] = execfile_extractor
# The handling of 'open' built-in for Python3 is not yet correct.
_dispatch_dict[ "open" ] = open_extractor
else:
_dispatch_dict[ "exec" ] = exec_extractor
def check():
from nuitka.Builtins import builtin_names
for builtin_name in _dispatch_dict:
assert builtin_name in builtin_names, builtin_name
check()
def computeBuiltinCall(call_node, called):
builtin_name = called.getBuiltinName()
if builtin_name in _dispatch_dict:
new_node = _dispatch_dict[builtin_name](call_node)
# Lets just have this contract to return "None" when no change is meant
# to be done.
assert new_node is not call_node
if new_node is None:
return call_node, None, None
# For traces, we are going to ignore side effects, and output traces
# only based on the basis of it.
inspect_node = new_node
if inspect_node.isExpressionSideEffects():
inspect_node = inspect_node.getExpression()
if inspect_node.isExpressionBuiltinImport():
tags = "new_import"
message = """\
Replaced dynamic __import__ %s with static module import.""" % (
inspect_node.kind,
)
elif inspect_node.isExpressionBuiltin() or \
inspect_node.isStatementExec():
tags = "new_builtin"
message = "Replaced call to builtin %s with builtin call %s." % (
builtin_name,
inspect_node.kind,
)
elif inspect_node.isExpressionRaiseException():
tags = "new_raise"
message = """\
Replaced call to builtin %s with exception raising call.""" % (
inspect_node.kind,
)
elif inspect_node.isExpressionOperationUnary():
tags = "new_expression"
message = """\
Replaced call to builtin %s with unary operation %s.""" % (
inspect_node.kind,
inspect_node.getOperator()
)
else:
assert False, ( builtin_name, "->", inspect_node )
# TODO: One day, this should be enabled by default and call either the
# original built-in or the optimized above one. That should be done,
# once we can eliminate the condition for most cases.
if False and isDebug() and not shallMakeModule() and builtin_name:
from nuitka.nodes.BuiltinRefNodes import (
ExpressionBuiltinOriginalRef,
ExpressionBuiltinRef,
)
from nuitka.nodes.NodeMakingHelpers import \
makeRaiseExceptionReplacementExpression
source_ref = called.getSourceReference()
new_node = ExpressionConditional(
condition = ExpressionComparisonIs(
left = ExpressionBuiltinRef(
builtin_name = builtin_name,
source_ref = source_ref
),
right = ExpressionBuiltinOriginalRef(
builtin_name = builtin_name,
source_ref = source_ref
),
source_ref = source_ref
),
yes_expression = new_node,
no_expression = makeRaiseExceptionReplacementExpression(
exception_type = "RuntimeError",
exception_value = "Builtin '%s' was overloaded'" % (
builtin_name
),
expression = call_node
),
source_ref = source_ref
)
assert tags != ""
return new_node, tags, message
else:
# TODO: Consider giving warnings, whitelisted potentially
return call_node, None, None
Nuitka-0.5.0.1/nuitka/optimizations/ConstraintCollections.py 0000644 0001750 0001750 00000072231 12265264105 024451 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Constraint collection
At the core of value propagation there is the collection of constraints that
allow to propagate knowledge forward or not.
This is about collecting these constraints and to manage them.
"""
# Python3 compatibility.
from nuitka.__past__ import iterItems
from nuitka.nodes.NodeMakingHelpers import (
makeStatementExpressionOnlyReplacementNode,
makeStatementsSequenceReplacementNode,
)
from nuitka.nodes.AssignNodes import StatementDelVariable
from nuitka import Options
from logging import debug, warning
from .VariableTraces import (
VariableUnknownTrace,
VariableAssignTrace,
VariableUninitTrace,
VariableMergeTrace
)
# TODO: This will be removed, to be replaced by variable trace information.
class VariableUsageProfile:
def __init__(self, variable):
self.variable = variable
self.written_to = False
self.read_from = False
# Indicator, if the variable may contain a reference.
self.needs_free = None
def markAsWrittenTo(self, assign_source):
self.written_to = True
if assign_source.mayProvideReference():
self.needs_free = True
elif self.needs_free is None:
self.needs_free = False
def markAsReadFrom(self):
self.read_from = True
def isReadOnly(self):
return not self.written_to
def isWriteOnly(self):
return self.written_to and not self.read_from
def getNeedsFree(self):
return self.needs_free
class VariableUsageTrackingMixin:
def __init__(self):
self.variable_usages = {}
# TODO: This will be removed, to be replaced by variable trace information.
def _getVariableUsage(self, variable):
if variable in self.variable_usages:
return self.variable_usages[ variable ]
else:
self.variable_usages[ variable ] = VariableUsageProfile( variable )
return self.variable_usages[ variable ]
# TODO: This will be removed, to be replaced by variable trace information.
def setIndications(self):
for variable, usage in iterItems( self.variable_usages ):
if variable.isTempKeeperVariable():
variable.setNeedsFree( usage.getNeedsFree() )
if usage.isWriteOnly():
variable.setWriteOnly()
def setupVariableTraces(self, owner):
for variable in owner.getVariables():
# print owner.isPythonModule(), variable
if variable.isParameterVariable():
self.initVariableUnknown( variable )
elif variable.isLocalVariable():
self.initVariableUninit( variable )
elif variable.isMaybeLocalVariable():
self.initVariableUnknown( variable )
elif variable.isModuleVariableReference():
pass
elif variable.isModuleVariable():
self.initVariableUnknown( variable.makeReference( owner ) )
elif variable.isClosureReference():
pass
else:
assert False, variable
for variable in owner.getTempVariables():
self.initVariableUninit( variable.makeReference( owner ) )
for variable in owner.getTempKeeperVariables():
self.initVariableUninit( variable.makeReference( owner ) )
if owner.isExpressionFunctionBody():
for variable in owner.taken:
self.initVariableUnknown( variable )
def _makeVariableTraceOptimization(self, owner, variable_trace):
variable = variable_trace.getVariable()
if variable.isTempVariableReference():
referenced_variable = variable.getReferenced()
if referenced_variable.isTempKeeperVariable():
if variable_trace.isAssignTrace() and \
not variable_trace.isMergeTrace() and \
not variable_trace.getPotentialUsages():
assign_node = variable_trace.getAssignNode()
assign_source = assign_node.getAssignSource()
assign_node.replaceWith( assign_source )
owner.removeTempKeeperVariable( referenced_variable )
self.signalChange(
"new_expression",
assign_node.getSourceReference(),
"Removed now useless temporary keeper assignment."
)
elif referenced_variable.isTempVariable():
if referenced_variable.getOwner() is owner:
if variable_trace.isUninitTrace() and \
variable_trace.getVersion() == 0:
if self.getVariableCurrentTrace( variable ) is variable_trace:
# TODO: Removing them now breaks merging, could be
# done not at all before code generation.
# owner.removeTempVariable( variable )
pass
def makeVariableTraceOptimizations(self, owner):
# Reliable trace based optimization goes here:
for variable_trace in self.variable_traces.values():
try:
self._makeVariableTraceOptimization(
owner = owner,
variable_trace = variable_trace
)
except:
print( "Problem with", variable_trace, "in", owner )
raise
class CollectionTracingMixin:
def __init__(self):
# For functions, when we are in here, the currently active one,
self.variable_actives = {}
def getVariableCurrentTrace(self, variable):
return self.getVariableTrace(
variable = variable,
version = self.getCurrentVariableVersion( variable )
)
def markCurrentVariableTrace(self, variable, version):
assert not variable.isModuleVariable() or variable.isReference(), \
variable
self.variable_actives[ variable ] = version
def getCurrentVariableVersion(self, variable):
# TODO: This is only while "eval" built-in enters new variables without
# telling us.
if variable.isTempVariableReference() and \
variable.getReferenced().isTempKeeperVariable() and \
variable not in self.variable_actives:
self.initVariableUninit( variable )
assert variable in self.variable_actives, ( variable, self )
return self.variable_actives[ variable ]
def getActiveVariables(self):
return tuple( self.variable_actives.keys() )
def markActiveVariableAsUnknown(self, variable):
current = self.getVariableCurrentTrace(
variable = variable,
)
if not current.isUnknownTrace():
version = variable.allocateTargetNumber()
self.addVariableTrace(
variable = variable,
version = version,
trace = VariableUnknownTrace(
variable = variable,
version = version
)
)
self.markCurrentVariableTrace( variable, version )
def markActiveVariablesAsUnknown(self):
for variable in self.getActiveVariables():
self.markActiveVariableAsUnknown( variable )
class CollectionStartpointMixin:
def __init__(self):
# Variable assignments performed in here, last issued number, only used
# to determine the next number that should be used for a new assignment.
self.variable_versions = {}
# The full trace of a variable with a version for the function or module
# this is.
self.variable_traces = {}
# Cannot mess with local variables that much, as "locals" and "eval"
# calls may not yet be known.
self.unclear_locals = False
def getVariableTrace(self, variable, version):
return self.variable_traces[ ( variable, version ) ]
def getVariableTraces(self, variable):
result = []
for key, variable_trace in iterItems( self.variable_traces ):
candidate = key[0]
candidate = candidate.getReferenced()
if variable is candidate:
result.append( variable_trace )
return result
def addVariableTrace(self, variable, version, trace):
key = variable, version
assert key not in self.variable_traces, ( key, self )
self.variable_traces[ key ] = trace
def addVariableMergeTrace(self, variable, trace_yes, trace_no):
version = variable.allocateTargetNumber()
trace_merge = VariableMergeTrace(
variable = variable,
version = version,
trace_yes = trace_yes,
trace_no = trace_no
)
self.addVariableTrace( variable, version, trace_merge )
# Merging is using, might imply releasing.
trace_yes.addUsage( trace_merge )
trace_no.addUsage( trace_merge )
def dumpTraces(self):
debug( "Constraint collection state:" )
for variable_desc, variable_trace in iterItems( self.variable_traces ):
debug( "%r: %r", variable_desc, variable_trace )
variable_trace.dump()
def initVariableUnknown(self, variable):
self.addVariableTrace(
variable = variable,
version = 0,
trace = VariableUnknownTrace(
variable = variable,
version = 0
)
)
self.markCurrentVariableTrace( variable, 0 )
def initVariableUninit(self, variable):
self.addVariableTrace(
variable = variable,
version = 0,
trace = VariableUninitTrace(
variable = variable,
version = 0
)
)
self.markCurrentVariableTrace( variable, 0 )
def assumeUnclearLocals(self):
self.unclear_locals = True
# TODO: This code is only here while staging it, will live in a dedicated module
# later on
class ConstraintCollectionBase(CollectionTracingMixin):
def __init__(self, parent, signal_change = None):
CollectionTracingMixin.__init__( self )
assert signal_change is None or parent is None
if signal_change is not None:
self.signalChange = signal_change
else:
self.signalChange = parent.signalChange
self.parent = parent
# Trust variable_traces, should go away later on, for now we use it to
# disable optimization.
self.removes_knowledge = False
def mustAlias(self, a, b):
if a.isExpressionVariableRef() and b.isExpressionVariableRef():
return a.getVariable() is b.getVariable()
return False
def mustNotAlias(self, a, b):
return False
def removeKnowledge(self, node):
assert node.isNode()
def removeAllKnowledge(self):
# Temporary, we don't have to have this anyway, this will just disable
# all uses of variable traces for optimization.
self.removes_knowledge = True
self.markActiveVariablesAsUnknown()
def assumeUnclearLocals(self):
self.parent.assumeUnclearLocals()
def getVariableTrace(self, variable, version):
return self.parent.getVariableTrace( variable, version )
def addVariableTrace(self, variable, version, trace):
assert self.parent is not None, self
self.parent.addVariableTrace( variable, version, trace )
def addVariableMergeTrace(self, variable, trace_yes, trace_no):
assert self.parent is not None, self
self.parent.addVariableMergeTrace( variable, trace_yes, trace_no )
def onVariableSet(self, assign_node):
if assign_node.isStatementAssignmentVariable():
target_node = assign_node.getTargetVariableRef()
else:
target_node = assign_node
# Add a new trace, using the version allocated for the variable, and
# remember the value friend.
variable = target_node.getVariable()
assert not variable.isModuleVariable() or variable.isReference(), \
variable
# print "SET", target_node, target_node.getVariableVersion()
version = target_node.getVariableVersion()
self.addVariableTrace(
variable = variable,
version = version,
trace = VariableAssignTrace(
assign_node = assign_node,
variable = variable,
version = version
)
)
# Make references point to it.
self.markCurrentVariableTrace( variable, version )
def onVariableDel(self, target_node):
# Add a new trace, allocating a new version for the variable, and
# remember the delete of the current
variable = target_node.getVariable()
current = self.getVariableCurrentTrace( variable )
current.addRelease( target_node )
version = target_node.getVariableVersion()
# Assign to uninit again.
self.addVariableTrace(
variable = variable,
version = version,
trace = VariableUninitTrace(
variable = variable,
version = version
)
)
# Make references point to it.
self.markCurrentVariableTrace( variable, version )
def onVariableUsage(self, ref_node):
variable = ref_node.getVariable()
self.getVariableCurrentTrace( variable ).addUsage( ref_node )
def onVariableContentEscapes(self, variable):
self.getVariableCurrentTrace( variable ).onValueEscape()
def onExpression(self, expression, allow_none = False):
if expression is None and allow_none:
return
assert expression.isExpression(), expression
assert expression.parent, expression
# Now compute this expression, allowing it to replace itself with
# something else as part of a local peephole optimization.
r = expression.computeExpressionRaw(
constraint_collection = self
)
assert type(r) is tuple, expression
new_node, change_tags, change_desc = r
if change_tags is not None:
# This is mostly for tracing and indication that a change occured
# and it may be interesting to look again.
self.signalChange(
change_tags,
expression.getSourceReference(),
change_desc
)
if new_node is not expression:
expression.replaceWith(new_node)
if new_node.isExpressionVariableRef():
# OLD:
if not new_node.getVariable().isModuleVariableReference():
self.onLocalVariableRead( new_node.getVariable() )
# Remember this for constraint collection. Any variable that we
# access has a version already that we can query. TODO: May do this
# as a "computeReference".
self.onVariableUsage( new_node )
elif new_node.isExpressionAssignmentTempKeeper():
variable = new_node.getVariable()
assert variable is not None
assign_source = new_node.getAssignSource()
assert assign_source is not None
self.onTempVariableAssigned( variable, assign_source )
elif new_node.isExpressionTempKeeperRef():
variable = new_node.getVariable()
assert variable is not None
self.onVariableUsage( new_node )
self.onTempVariableRead( variable )
return new_node
def onModuleVariableAssigned(self, variable, assign_source):
self.parent.onModuleVariableAssigned( variable, assign_source )
def onLocalVariableAssigned(self, variable, assign_source):
self.parent.onLocalVariableAssigned( variable, assign_source )
def onLocalVariableRead(self, variable):
self.parent.onLocalVariableRead( variable )
def onTempVariableAssigned(self, variable, assign_source):
self.parent.onTempVariableAssigned( variable, assign_source )
def onTempVariableRead(self, variable):
self.parent.onTempVariableRead( variable )
def _onStatementAssignmentVariable(self, statement):
# But now it cannot re-compute anymore:
source = statement.getAssignSource()
if source.willRaiseException( BaseException ):
result = makeStatementExpressionOnlyReplacementNode(
expression = source,
node = statement
)
return result, "new_raise", """\
Removed assignment that has source that will raise."""
variable_ref = statement.getTargetVariableRef()
variable = variable_ref.getVariable()
assert variable is not None
# Assigning from and to the same variable, can be optimized away
# immediately, there is no point in doing it. Exceptions are of course
# module variables that collide with builtin names.
if not variable.isModuleVariableReference() and \
source.isExpressionVariableRef() and \
source.getVariable() == variable:
if source.mayHaveSideEffects():
result = makeStatementExpressionOnlyReplacementNode(
expression = source,
node = statement
)
return result, "new_statements", """\
Reduced assignment of variable from itself to access of it."""
else:
return None, "new_statements", """\
Removed assignment of variable from itself which is known to be defined."""
# If the assignment source has side effects, we can simply evaluate them
# beforehand, we have already visited and evaluated them before.
if source.isExpressionSideEffects():
statements = [
makeStatementExpressionOnlyReplacementNode(
side_effect,
statement
)
for side_effect in
source.getSideEffects()
]
statements.append( statement )
result = makeStatementsSequenceReplacementNode(
statements = statements,
node = statement,
)
source.replaceWith( source.getExpression() )
# Need to update it.
source = statement.getAssignSource()
result = result, "new_statements", """\
Side effects of assignments promoted to statements."""
else:
result = statement, None, None
if variable.isModuleVariableReference():
self.onModuleVariableAssigned( variable, source )
elif variable.isLocalVariable():
self.onLocalVariableAssigned( variable, source )
elif variable.isTempVariableReference():
self.onTempVariableAssigned( variable, source )
return result
def onStatement(self, statement):
try:
assert statement.isStatement(), statement
new_statement, change_tags, change_desc = \
statement.computeStatement(self)
if new_statement is not statement:
self.signalChange(
change_tags,
statement.getSourceReference(),
change_desc
)
return new_statement
except Exception:
warning(
"Problem with statement at %s:",
statement.getSourceReference()
)
raise
def mergeBranches(self, collection_yes, collection_no):
# Refuse to do stupid work
if collection_yes is None and collection_no is None:
pass
elif collection_yes is None or collection_no is None:
# Handle one branch case, we need to merge versions backwards as
# they may make themselves obsolete.
collection = collection_yes or collection_no
for variable in collection.getActiveVariables():
# print "ACTIVE", variable, self.getCurrentVariableVersion( variable )
trace_old = self.getVariableCurrentTrace( variable )
trace_new = collection.getVariableCurrentTrace( variable )
assert trace_old is not None
assert trace_new is not None
if trace_old is not trace_new:
self.addVariableMergeTrace(
variable = variable,
trace_yes = trace_new,
trace_no = trace_old
)
return
else:
for variable in collection_yes.getActiveVariables():
trace_yes = collection_yes.getVariableCurrentTrace( variable )
trace_no = collection_no.getVariableCurrentTrace( variable )
if trace_yes is not trace_no:
self.addVariableMergeTrace(
variable = variable,
trace_yes = trace_yes,
trace_no = trace_no
)
class ConstraintCollectionHandler(ConstraintCollectionBase):
def __init__(self, parent, handler):
assert handler.isStatementExceptHandler(), handler
ConstraintCollectionBase.__init__(
self,
parent = parent
)
self.variable_actives = dict(parent.variable_actives)
# TODO: The exception type and can be assumed assigned.
branch = handler.getExceptionBranch()
if branch is not None:
result = branch.computeStatementsSequence(
constraint_collection = self
)
if result is not branch:
handler.setExceptionBranch(result)
exception_types = handler.getExceptionTypes()
if exception_types is not None:
for exception_type in exception_types:
self.onExpression(exception_type)
class ConstraintCollectionBranch(ConstraintCollectionBase):
def __init__(self, parent, branch):
ConstraintCollectionBase.__init__(
self,
parent = parent
)
self.variable_actives = dict(parent.variable_actives)
if branch.isStatementsSequence():
result = branch.computeStatementsSequence(
constraint_collection = self
)
if result is not branch:
branch.replaceWith(result)
else:
self.onExpression(
expression = branch
)
def mergeBranches(self, collection_yes, collection_no):
# Branches in branches, should ask parent about merging them.
return self.parent.mergeBranches( collection_yes, collection_no )
# TODO: This make go away once we have keeper variables better covered.
def initVariableUninit(self, variable):
self.parent.initVariableUninit( variable )
self.markCurrentVariableTrace( variable, 0 )
class ConstraintCollectionFunction(CollectionStartpointMixin,
ConstraintCollectionBase,
VariableUsageTrackingMixin):
def __init__(self, parent, function_body):
assert function_body.isExpressionFunctionBody(), function_body
CollectionStartpointMixin.__init__(self)
ConstraintCollectionBase.__init__(
self,
parent = parent
)
VariableUsageTrackingMixin.__init__(self)
self.function_body = function_body
statements_sequence = function_body.getBody()
if statements_sequence is not None and \
not statements_sequence.getStatements():
function_body.setStatements( None )
statements_sequence = None
self.setupVariableTraces( function_body )
if statements_sequence is not None:
result = statements_sequence.computeStatementsSequence(
constraint_collection = self
)
if result is not statements_sequence:
function_body.setBody(result)
# TODO: Should become trace based as well.
self.setIndications()
self.makeVariableTraceOptimizations( function_body )
if not Options.isExperimental() or self.removes_knowledge:
return
# self.dumpTrace()
# Cannot mess with locals yet.
if self.unclear_locals:
return
# Trace based optimization goes here:
for variable_trace in self.variable_traces.values():
variable = variable_trace.getVariable()
# print variable
if variable.isLocalVariable() and not variable.isShared():
if variable_trace.isAssignTrace():
assign_node = variable_trace.getAssignNode()
if not assign_node.getAssignSource().mayHaveSideEffects():
if not variable_trace.getPotentialUsages() and \
not variable_trace.isEscaped():
assign_node.parent.replaceWith(
StatementDelVariable(
variable_ref = assign_node,
tolerant = True,
source_ref = assign_node.getSourceReference()
)
)
for release in variable_trace.releases:
if release.isStatementDelVariable():
release.replaceWith(
None
)
self.signalChange(
"new_statements",
assign_node.parent.getSourceReference(),
"Removed assignment without effect."
)
elif variable_trace.isMergeTrace():
# print variable_trace
if not variable_trace.getDefiniteUsages() and \
not variable_trace.isEscaped() and \
not variable_trace.releases:
pass
# print "HIT", variable_trace
def onLocalVariableAssigned(self, variable, assign_source):
self._getVariableUsage( variable ).markAsWrittenTo( assign_source )
def onLocalVariableRead(self, variable):
self._getVariableUsage( variable ).markAsReadFrom()
def onTempVariableAssigned(self, variable, assign_source):
variable = variable.getReferenced()
# assert variable.getOwner() is self.function_body
self._getVariableUsage( variable ).markAsWrittenTo( assign_source )
def onTempVariableRead(self, variable):
variable = variable.getReferenced()
self._getVariableUsage( variable ).markAsReadFrom()
class ConstraintCollectionModule(CollectionStartpointMixin,
ConstraintCollectionBase,
VariableUsageTrackingMixin):
def __init__(self, signal_change, module):
assert module.isPythonModule()
CollectionStartpointMixin.__init__( self )
ConstraintCollectionBase.__init__(
self,
None,
signal_change = signal_change
)
VariableUsageTrackingMixin.__init__( self )
self.module = module
self.setupVariableTraces( module )
module_body = module.getBody()
if module_body is not None:
result = module_body.computeStatementsSequence(
constraint_collection = self
)
if result is not module_body:
module.setBody(result)
self.setIndications()
self.makeVariableTraceOptimizations( module )
def onModuleVariableAssigned(self, variable, assign_source):
while variable.isModuleVariableReference():
variable = variable.getReferenced()
self._getVariableUsage( variable ).markAsWrittenTo( assign_source )
def onTempVariableAssigned(self, variable, assign_source):
variable = variable.getReferenced()
assert variable.getRealOwner() is self.module, variable.getOwner()
self._getVariableUsage( variable ).markAsWrittenTo( assign_source )
def onTempVariableRead(self, variable):
variable = variable.getReferenced()
assert variable.getRealOwner() is self.module, variable.getOwner()
self._getVariableUsage( variable ).markAsReadFrom()
def getWrittenVariables(self):
return [
variable
for variable, usage in iterItems( self.variable_usages )
if not usage.isReadOnly()
]
Nuitka-0.5.0.1/nuitka/optimizations/Optimization.py 0000644 0001750 0001750 00000010671 12265264105 022614 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Control the flow of optimizations applied to node tree.
Applies constraint collection on all so far known modules until no more
optimization is possible. Every successful optimization to anything might
make others possible.
"""
from logging import debug
from nuitka import ModuleRegistry, Options
from nuitka.Tracing import printLine
from .ConstraintCollections import ConstraintCollectionModule
from .Tags import TagSet
_progress = Options.isShowProgress()
def _attemptRecursion(module):
new_modules = module.attemptRecursion()
for new_module in new_modules:
debug(
"{source_ref} : {tags} : {message}".format(
source_ref = new_module.getSourceReference().getAsString(),
tags = "new_code",
message = "Recursed to module package."
)
)
tag_set = None
def signalChange(tags, source_ref, message):
""" Indicate a change to the optimization framework.
"""
debug(
"{source_ref} : {tags} : {message}".format(
source_ref = source_ref.getAsString(),
tags = tags,
message = message
)
)
tag_set.onSignal(tags)
def _optimizeModulePass(module, tag_set):
module.collection = ConstraintCollectionModule(
signal_change = signalChange,
module = module
)
# Pick up parent package if any.
_attemptRecursion(module)
written_variables = module.collection.getWrittenVariables()
for variable in module.getVariables():
old_value = variable.getReadOnlyIndicator()
new_value = variable not in written_variables
if old_value is not new_value:
# Don't suddenly start to write.
assert not (new_value is False and old_value is True)
module.collection.signalChange(
"read_only_mvar",
module.getSourceReference(),
"Determined variable '{variable_name}' is only read.".format(
variable_name = variable.getName()
)
)
variable.setReadOnlyIndicator(new_value)
def optimizePythonModule(module):
if _progress:
printLine(
"Doing module local optimizations for '{module_name}'.".format(
module_name = module.getFullName()
)
)
global tag_set
tag_set = TagSet()
touched = False
while True:
tag_set.clear()
_optimizeModulePass(
module = module,
tag_set = tag_set
)
if not tag_set:
break
touched = True
return touched
def optimizeShlibModule(module):
# Pick up parent package if any.
_attemptRecursion(module)
global tag_set
tag_set = TagSet()
module.considerImplicitImports(signal_change = signalChange)
def optimize():
while True:
finished = True
ModuleRegistry.startTraversal()
while True:
current_module = ModuleRegistry.nextModule()
if current_module is None:
break
if _progress:
printLine(
"""\
Optimizing module '{module_name}', {remaining:d} more modules to go \
after that.""".format(
module_name = current_module.getFullName(),
remaining = ModuleRegistry.remainingCount()
)
)
if current_module.isPythonShlibModule():
optimizeShlibModule(current_module)
else:
changed = optimizePythonModule(current_module)
if changed:
finished = False
if finished:
break
Nuitka-0.5.0.1/nuitka/optimizations/BuiltinOptimization.py 0000644 0001750 0001750 00000033631 12265264105 024144 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Optimizations of builtins to builtin calls.
"""
from nuitka.nodes.ParameterSpecs import (
ParameterSpec,
TooManyArguments,
matchCall
)
from nuitka.Utils import python_version
import sys, math
class BuiltinParameterSpec(ParameterSpec):
def __init__( self, name, arg_names, default_count, list_star_arg = None,
dict_star_arg = None ):
ParameterSpec.__init__(
self,
name = name,
normal_args = arg_names,
list_star_arg = list_star_arg,
dict_star_arg = dict_star_arg,
default_count = default_count,
kw_only_args = ()
)
self.builtin = __builtins__[ name ]
def __repr__(self):
return "" % self.name
def getName(self):
return self.name
def isCompileTimeComputable(self, values):
for value in values:
if value is not None and not value.isCompileTimeConstant():
return False
else:
return True
def simulateCall(self, given_values):
# Using star dict call for simulation and catch any exception as really
# fatal, pylint: disable=W0142,W0703
try:
given_normal_args = given_values[ : len( self.normal_args ) ]
if self.list_star_arg:
given_list_star_args = given_values[ len( self.normal_args ) ]
else:
given_list_star_args = None
if self.dict_star_arg:
given_dict_star_args = given_values[ -1 ]
else:
given_dict_star_args = None
arg_dict = {}
for arg_name, given_value in zip(self.normal_args, given_normal_args):
assert type( given_value ) not in ( tuple, list ), \
( "do not like a tuple %s" % ( given_value, ))
if given_value is not None:
arg_dict[ arg_name ] = given_value.getCompileTimeConstant()
if given_dict_star_args:
for given_dict_star_arg in reversed(given_dict_star_args):
arg_name = given_dict_star_arg.getKey()
arg_value = given_dict_star_arg.getValue()
arg_dict[ arg_name.getCompileTimeConstant() ] = arg_value.getCompileTimeConstant()
except Exception as e:
sys.exit( "Fatal problem: %r" % e )
if given_list_star_args:
return self.builtin(
*( value.getCompileTimeConstant() for value in given_list_star_args ),
**arg_dict
)
else:
return self.builtin( **arg_dict )
class BuiltinParameterSpecNoKeywords(BuiltinParameterSpec):
def allowsKeywords(self):
return False
def simulateCall(self, given_values):
# Using star dict call for simulation and catch any exception as really fatal,
# pylint: disable=W0142,W0703
try:
if self.list_star_arg:
given_list_star_arg = given_values[ len( self.normal_args ) ]
else:
given_list_star_arg = None
arg_list = []
refuse_more = False
for _arg_name, given_value in zip( self.normal_args, given_values ):
assert type( given_value ) not in ( tuple, list ), ( "do not like tuple %s" % ( given_value, ))
if given_value is not None:
if not refuse_more:
arg_list.append( given_value.getCompileTimeConstant() )
else:
assert False
else:
refuse_more = True
if given_list_star_arg is not None:
arg_list += [ value.getCompileTimeConstant() for value in given_list_star_arg ]
except Exception as e:
print >> sys.stderr, "Fatal error: ",
sys.exit( repr( e ) )
return self.builtin( *arg_list )
class BuiltinParameterSpecExceptions(BuiltinParameterSpec):
def __init__(self, exception_name, default_count):
# TODO: Parameter default_count makes no sense for exceptions probably.
BuiltinParameterSpec.__init__(
self,
name = exception_name,
arg_names = (),
default_count = default_count,
list_star_arg = "args"
)
def allowsKeywords(self):
return False
def getKeywordRefusalText(self):
return "exceptions.%s does not take keyword arguments" % self.name
def getCallableName(self):
return "exceptions." + self.getName()
def makeBuiltinParameterSpec(exception_name):
if exception_name == "ImportError" and python_version >= 330:
# TODO: Create this beast, needs keyword only arguments to be supported,
# currently user of this function must take care to not have them.
pass
return BuiltinParameterSpecExceptions(
exception_name = exception_name,
default_count = 0
)
builtin_int_spec = BuiltinParameterSpec( "int", ( "x", "base" ), 2 )
# These builtins are only available for Python2
if python_version < 300:
builtin_long_spec = BuiltinParameterSpec(
"long",
("x", "base"),
2
)
builtin_execfile_spec = BuiltinParameterSpecNoKeywords(
"execfile",
("filename", "globals", "locals"),
2
)
builtin_unicode_spec = BuiltinParameterSpec(
"unicode",
("string", "encoding", "errors"),
3
)
builtin_xrange_spec = BuiltinParameterSpec(
"xrange",
("start", "stop", "step"),
2
)
builtin_bool_spec = BuiltinParameterSpec( "bool", ( "x", ), 1 )
builtin_float_spec = BuiltinParameterSpec( "float", ( "x", ), 1 )
# This builtin have variable parameters for Python2/3
if python_version < 300:
builtin_str_spec = BuiltinParameterSpec( "str", ( "object", ), 1 )
else:
builtin_str_spec = BuiltinParameterSpec( "str", ( "object", "encoding", "errors" ), 3 )
builtin_len_spec = BuiltinParameterSpecNoKeywords( "len", ( "object", ), 0 )
builtin_dict_spec = BuiltinParameterSpec( "dict", (), 0, "list_args", "dict_args" )
builtin_len_spec = BuiltinParameterSpecNoKeywords( "len", ( "object", ), 0 )
builtin_tuple_spec = BuiltinParameterSpec( "tuple", ( "sequence", ), 1 )
builtin_list_spec = BuiltinParameterSpec( "list", ( "sequence", ), 1 )
builtin_set_spec = BuiltinParameterSpecNoKeywords( "set", ( "iterable", ), 1 )
builtin_import_spec = BuiltinParameterSpec( "__import__", ( "name", "globals", "locals", "fromlist", "level" ), 4 )
builtin_open_spec = BuiltinParameterSpec( "open", ( "name", "mode", "buffering" ), 2 )
builtin_chr_spec = BuiltinParameterSpecNoKeywords( "chr", ( "i", ), 0 )
builtin_ord_spec = BuiltinParameterSpecNoKeywords( "ord", ( "c", ), 0 )
builtin_bin_spec = BuiltinParameterSpecNoKeywords( "bin", ( "number", ), 0 )
builtin_oct_spec = BuiltinParameterSpecNoKeywords( "oct", ( "number", ), 0 )
builtin_hex_spec = BuiltinParameterSpecNoKeywords( "hex", ( "number", ), 0 )
builtin_repr_spec = BuiltinParameterSpecNoKeywords( "repr", ( "object", ), 0 )
builtin_dir_spec = BuiltinParameterSpecNoKeywords( "dir", ( "object", ), 1 )
builtin_vars_spec = BuiltinParameterSpecNoKeywords( "vars", ( "object", ), 1 )
builtin_locals_spec = BuiltinParameterSpecNoKeywords( "locals", (), 0 )
builtin_globals_spec = BuiltinParameterSpecNoKeywords( "globals", (), 0 )
builtin_eval_spec = BuiltinParameterSpecNoKeywords( "eval", ( "source", "globals", "locals" ), 2 )
if python_version >= 300:
builtin_exec_spec = BuiltinParameterSpecNoKeywords( "exec", ( "source", "globals", "locals" ), 2 )
# Note: Iter in fact names its first argument if the default applies "collection", but it
# won't matter much, fixed up in a wrapper.
builtin_iter_spec = BuiltinParameterSpecNoKeywords( "iter", ( "callable", "sentinel" ), 1 )
builtin_next_spec = BuiltinParameterSpecNoKeywords( "next", ( "iterator", "default" ), 1 )
# Note: type with 1 and type with 3 arguments are too different.
builtin_type1_spec = BuiltinParameterSpecNoKeywords( "type", ( "object", ), 0 )
builtin_type3_spec = BuiltinParameterSpecNoKeywords( "type", ( "name", "bases", "dict" ), 0 )
builtin_super_spec = BuiltinParameterSpecNoKeywords( "super", ( "type", "object" ), 1 if python_version < 300 else 2 )
builtin_hasattr_spec = BuiltinParameterSpecNoKeywords( "hasattr", ( "object", "name" ), 0 )
builtin_getattr_spec = BuiltinParameterSpecNoKeywords( "getattr", ( "object", "name", "default" ), 1 )
builtin_setattr_spec = BuiltinParameterSpecNoKeywords( "setattr", ( "object", "name", "value" ), 0 )
builtin_isinstance_spec = BuiltinParameterSpecNoKeywords( "isinstance", ( "instance", "cls" ), 0 )
class BuiltinRangeSpec(BuiltinParameterSpecNoKeywords):
def __init__(self, *args):
BuiltinParameterSpecNoKeywords.__init__( self, *args )
def isCompileTimeComputable(self, values):
result = BuiltinParameterSpecNoKeywords.isCompileTimeComputable(
self,
values = values
)
if result:
arg_count = len(values)
if arg_count == 1:
low = values[0]
# If it's not a number constant, we can compute the exception
# that will be raised.
if not low.isNumberConstant():
return True
return low.getConstant() < 256
elif arg_count == 2:
low, high = values
# If it's not a number constant, we can compute the exception
# that will be raised.
if not low.isNumberConstant() or not high.isNumberConstant():
return True
return high.getConstant() - low.getConstant() < 256
elif arg_count == 3:
low, high, step = values
if not low.isNumberConstant() or \
not high.isNumberConstant() or \
not step.isNumberConstant():
return True
low = low.getConstant()
high = high.getConstant()
step = step.getConstant()
# It's going to give a ZeroDivisionError in this case.
if step == 0:
return True
if low < high:
if step < 0:
return True
else:
return math.ceil( float( high - low ) / step ) < 256
else:
if step > 0:
return True
else:
return math.ceil( float( high - low ) / step ) < 256
else:
assert False
else:
return False
builtin_range_spec = BuiltinRangeSpec("range", ( "start", "stop", "step" ), 2)
def extractBuiltinArgs( node, builtin_spec, builtin_class,
empty_special_class = None ):
try:
kw = node.getCallKw()
# TODO: Could check for too many / too few, even if they are unknown, we
# might raise that error, but that need not be optimized immediately.
if not kw.isMappingWithConstantStringKeys():
return None
pairs = kw.getMappingStringKeyPairs()
if pairs and not builtin_spec.allowsKeywords():
raise TooManyArguments(
TypeError( builtin_spec.getKeywordRefusalText() )
)
args = node.getCallArgs()
if not args.canPredictIterationValues():
return None
positional = args.getIterationValues()
if not positional and not pairs and empty_special_class is not None:
return empty_special_class(source_ref = node.getSourceReference())
args_dict = matchCall(
func_name = builtin_spec.getName(),
args = builtin_spec.getArgumentNames(),
star_list_arg = builtin_spec.getStarListArgumentName(),
star_dict_arg = builtin_spec.getStarDictArgumentName(),
num_defaults = builtin_spec.getDefaultCount(),
positional = positional,
pairs = pairs
)
except TooManyArguments as e:
from nuitka.nodes.NodeMakingHelpers import (
makeRaiseExceptionReplacementExpressionFromInstance,
wrapExpressionWithSideEffects
)
return wrapExpressionWithSideEffects(
new_node = makeRaiseExceptionReplacementExpressionFromInstance(
expression = node,
exception = e.getRealException()
),
old_node = node,
side_effects = node.extractPreCallSideEffects()
)
args_list = []
for argument_name in builtin_spec.getArgumentNames():
args_list.append( args_dict[ argument_name ] )
if builtin_spec.getStarListArgumentName() is not None:
args_list.append( args_dict[ builtin_spec.getStarListArgumentName() ] )
if builtin_spec.getStarDictArgumentName() is not None:
args_list.append( args_dict[ builtin_spec.getStarDictArgumentName() ] )
# Using list reference for passing the arguments without names,
# pylint: disable=W0142
return builtin_class(
*args_list,
source_ref = node.getSourceReference()
)
Nuitka-0.5.0.1/nuitka/optimizations/VariableTraces.py 0000644 0001750 0001750 00000014230 12265264105 023010 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Variable trace objects.
Variable traces indicate the flow of variables and merges their versions for
the SSA (Single State Assignment) form being used in Nuitka.
Variable version can start as:
* Unknown (maybe initialized, maybe not, we cannot know)
* Uninit (definitely not initialized, first version, or after "del" statement)
* Init (definitely initialized, e.g. parameter variables)
* Merge (result of diverged code paths)
"""
from logging import debug
class VariableTraceBase:
def __init__(self, variable, version):
self.variable = variable
self.version = version
# List of references.
self.usages = []
# List of releases of the node.
self.releases = []
# List of merges.
self.merges = []
# If not None, this indicates the last usage, where the value was not
# yet escaped. If it is 0, it escaped immediately. Escaping is a one
# time action.
self.escaped_at = None
def isNode(self):
return False
def getVariable(self):
return self.variable
def getVersion(self):
return self.version
def addUsage(self, ref_node):
self.usages.append(ref_node)
def addMerge(self, trace):
self.merges.append(trace)
def addRelease(self, release_node):
self.releases.append(release_node)
def onValueEscape(self):
self.escaped_at = len(self.usages)
def isEscaped(self):
return self.escaped_at is not None
def getPotentialUsages(self):
return self.usages + \
sum(
[
merge.getPotentialUsages()
for merge in
self.merges
],
[]
)
def getDefiniteUsages(self):
return self.usages
def getReleases(self):
return self.releases
def isAssignTrace(self):
return False
def isUninitTrace(self):
return False
def isUnknownTrace(self):
return False
def isMergeTrace(self):
return False
class VariableUninitTrace(VariableTraceBase):
def __init__(self, variable, version):
VariableTraceBase.__init__(
self,
variable = variable,
version = version
)
def __repr__(self):
return "".format(
variable = self.variable,
version = self.version
)
def isUninitTrace(self):
return True
def dump(self):
debug(
"Trace of %s %d:",
self.variable,
self.version
)
debug(" Starts out uninitialized")
for count, usage in enumerate(self.usages):
if count == self.escaped_at:
debug(" Escaped value")
debug(" Used at %s", usage)
class VariableUnknownTrace(VariableTraceBase):
def __init__(self, variable, version):
VariableTraceBase.__init__(
self,
variable = variable,
version = version
)
def __repr__(self):
return "".format(
variable = self.variable,
version = self.version
)
def dump(self):
debug(
"Trace of %s %d:",
self.variable,
self.version
)
debug(" Starts unknown")
for count, usage in enumerate(self.usages):
if count == self.escaped_at:
debug(" Escaped value")
debug(" Used at %s", usage)
def isUnknownTrace(self):
return True
class VariableAssignTrace(VariableTraceBase):
def __init__(self, assign_node, variable, version):
VariableTraceBase.__init__(
self,
variable = variable,
version = version
)
self.assign_node = assign_node
def __repr__(self):
return """\
""".format(
variable = self.variable,
version = self.version,
source_ref = self.assign_node.getSourceReference()
)
def dump(self):
debug("Trace of %s %d:",
self.variable,
self.version
)
for count, usage in enumerate(self.usages):
if count == self.escaped_at:
debug(" Escaped value")
debug(" Used at %s", usage)
def isAssignTrace(self):
return True
def getAssignNode(self):
return self.assign_node
class VariableMergeTrace(VariableTraceBase):
""" Merge of two traces.
Happens at the end of two conditional blocks. This is "phi" in
SSA theory.
"""
def __init__(self, variable, version, trace_yes, trace_no):
assert trace_no is not trace_yes, (variable, version, trace_no)
VariableTraceBase.__init__(
self,
variable = variable,
version = version
)
self.trace_yes = trace_yes
self.trace_no = trace_no
trace_yes.addMerge(self)
trace_no.addMerge(self)
def isMergeTrace(self):
return True
def dump(self):
debug(
"Trace of %s %d:",
self.variable,
self.version
)
debug(
" Merge of %s <-> %s",
self.trace_yes,
self.trace_no
)
Nuitka-0.5.0.1/nuitka/optimizations/__init__.py 0000644 0001750 0001750 00000001501 12265264105 021655 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Dummy file to make this directory a package. """
Nuitka-0.5.0.1/nuitka/TreeXML.py 0000644 0001750 0001750 00000002350 12265264105 016470 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" XML node tree handling
Means to create XML elements from Nuitka tree nodes and to convert the
XML tree to ASCII or output it.
"""
from nuitka import Tracing, Utils
try:
import lxml.etree
except ImportError:
lxml = None
def toString(xml):
return lxml.etree.tostring( xml, pretty_print = True )
def dump(xml ):
value = toString( xml ).rstrip()
if Utils.python_version >= 300:
value = value.decode( "utf-8" )
Tracing.printLine( value )
Nuitka-0.5.0.1/nuitka/Constants.py 0000644 0001750 0001750 00000015404 12265264105 017170 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Module for constants in Nuitka.
This contains tools to compare, classify and test constants.
"""
import math
# pylint: disable=W0622
from .__past__ import long, unicode, iterItems
# pylint: enable=W0622
from .Builtins import builtin_anon_names
from .Utils import python_version
NoneType = type( None )
def compareConstants(a, b):
# Many many cases to deal with, pylint: disable=R0911,R0912
# Supposed fast path for comparison.
if type( a ) is not type( b ):
return False
# Now it's either not the same, or it is a container that contains NaN or it
# is a complex or float that is NaN, the other cases can use == at the end.
if type( a ) is complex:
return compareConstants( a.imag, b.imag ) and \
compareConstants( a.real, b.real )
if type( a ) is float:
# Check sign first, -0.0 is not 0.0, or -nan is not nan, it has a
# different sign for a start.
if math.copysign( 1.0, a ) != math.copysign( 1.0, b ):
return False
if math.isnan( a ) and math.isnan( b ):
return True
return a == b
if type( a ) in ( tuple, list ):
if len( a ) != len( b ):
return False
for ea, eb in zip( a, b ):
if not compareConstants( ea, eb ):
return False
else:
return True
if type( a ) is dict:
if len( a ) != len( b ):
return False
for ea1, ea2 in iterItems( a ):
for eb1, eb2 in iterItems( b ):
if compareConstants( ea1, eb1 ) and \
compareConstants( ea2, eb2 ):
break
else:
return False
else:
return True
if type( a ) in ( frozenset, set ):
if len( a ) != len( b ):
return False
for ea in a:
if ea not in b:
# Due to NaN values, we need to compare each set element with
# all the other set to be really sure.
for eb in b:
if compareConstants( ea, eb ):
break
else:
return False
else:
return True
if type( a ) is range:
return str( a ) == str( b )
# The NaN values of float and complex may let this fail, even if the
# constants are built in the same way.
return a == b
# These builtin type references are kind of constant too. TODO: The list is
# totally not complete.
constant_builtin_types = int, set, str, float, list, tuple, dict, complex
if python_version >= 300:
constant_builtin_types += (
range,
bytes,
)
else:
constant_builtin_types += (
unicode,
long,
# This has no name in Python, but the natural one in C-API.
builtin_anon_names[ "instance" ]
)
def isConstant(constant):
# Too many cases and all return, that is how we do it here,
# pylint: disable=R0911,R0912
constant_type = type( constant )
if constant_type is dict:
for key, value in iterItems( constant ):
if not isConstant( key ):
return False
if not isConstant( value ):
return False
else:
return True
elif constant_type in ( tuple, list ):
for element_value in constant:
if not isConstant( element_value ):
return False
else:
return True
elif constant_type in ( str, unicode, complex, int, long, bool, float,
NoneType, range, bytes, set ):
return True
elif constant in ( Ellipsis, NoneType ):
return True
elif constant_type is type:
return constant in constant_builtin_types
else:
return False
def isMutable(constant):
""" Is a constant mutable
That means a user of a reference to it, can modify it. Strings are
a prime example of mutable, dictionaries are mutable.
"""
constant_type = type( constant )
if constant_type in ( str, unicode, complex, int, long, bool, float,
NoneType, range, bytes ):
return False
elif constant_type in ( dict, list, set ):
return True
elif constant_type is tuple:
for value in constant:
if isMutable( value ):
return True
else:
return False
elif constant is Ellipsis:
return False
elif constant in constant_builtin_types:
return True
else:
assert False, constant_type
def isIterableConstant(constant):
return type( constant ) in (
str, unicode, list, tuple, set, frozenset, dict, range, bytes
)
def getConstantIterationLength(constant):
assert isIterableConstant( constant )
return len( constant )
def isNumberConstant(constant):
return type( constant ) in ( int, long, float, bool )
def isIndexConstant(constant):
return type( constant ) in ( int, long, bool )
class HashableConstant:
def __init__(self, constant):
self.constant = constant
try:
# For Python3: range objects with same ranges give different hash
# values. It's not even funny, is it.
if type( constant ) is range:
raise TypeError
self.hash = hash( constant )
except TypeError:
self.hash = 55
def getConstant(self):
return self.constant
def __hash__(self):
return self.hash
def __eq__(self, other):
assert isinstance( other, self.__class__ )
return compareConstants( self.constant, other.constant )
def createConstantDict(keys, values, lazy_order):
if lazy_order:
constant_value = {}
keys = list(keys)
keys.reverse()
values = list(values)
values.reverse()
else:
constant_value = dict.fromkeys(
[ key for key in keys ],
None
)
for key, value in zip( keys, values ):
constant_value[ key ] = value
return constant_value
Nuitka-0.5.0.1/nuitka/oset.py 0000644 0001750 0001750 00000006203 12265044767 016175 0 ustar hayen hayen 0000000 0000000 # Copyright 2009 Raymond Hettinger
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#
# Note: Kay Hayen did some changes for Nuitka keeping this license. These changes are
# not improvements, use the original source instead.
""" This module is only an abstraction of OrderedSet as present in 2.7 and 3.1 but not in
2.6.
It was originally downloaded from http://code.activestate.com/recipes/576694/
"""
# pylint: disable=W0622,W0221
import collections
KEY, PREV, NEXT = range(3)
class OrderedSet(collections.MutableSet):
def __init__(self, iterable=None):
self.end = end = []
end += [None, end, end]
self.map = {}
if iterable is not None:
self |= iterable
def __len__(self):
return len(self.map)
def __contains__(self, key):
return key in self.map
def add(self, key):
if key not in self.map:
end = self.end
curr = end[PREV]
curr[NEXT] = end[PREV] = self.map[key] = [key, curr, end]
def update(self, values):
for value in values:
self.add( value )
def discard(self, key):
if key in self.map:
key, prev, next = self.map.pop(key)
prev[NEXT] = next
next[PREV] = prev
def __iter__(self):
end = self.end
curr = end[NEXT]
while curr is not end:
yield curr[KEY]
curr = curr[NEXT]
def __reversed__(self):
end = self.end
curr = end[PREV]
while curr is not end:
yield curr[KEY]
curr = curr[PREV]
def pop(self, last=True):
if not self:
raise KeyError('set is empty')
key = next(reversed(self)) if last else next(iter(self))
self.discard(key)
return key
def __repr__(self):
if not self:
return '%s()' % (self.__class__.__name__,)
return '%s(%r)' % (self.__class__.__name__, list(self))
def __eq__(self, other):
if isinstance(other, OrderedSet):
return len(self) == len(other) and list(self) == list(other)
return set(self) == set(other)
def __del__(self):
self.clear()
Nuitka-0.5.0.1/nuitka/odict.py 0000644 0001750 0001750 00000014124 12265044767 016326 0 ustar hayen hayen 0000000 0000000 # :copyright: (c) 2008 by Armin Ronacher and PEP 273 authors.
# :license: modified BSD license.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of the nor the
# names of its contributors may be used to endorse or promote products
# derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY
# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# Kay Hayen did some changes for Nuitka, and put everything he added under the same
# modified BSD license.
""" This module is only an abstraction of OrderedDict as present in 2.7 and 3.1.
It is not in 2.6, for this version we are using the odict.py as mentioned in the PEP-0372
This can be removed safely after the transition, note that the documentation was removed,
as it's not interesting really, being redundent to Python 2.7 documentation. """
# pylint: disable=E0611,W0141
try:
from collections import OrderedDict
except ImportError:
from itertools import izip, imap
from copy import deepcopy
missing = object()
class OrderedDict(dict):
def __init__(self, *args, **kwargs):
dict.__init__(self)
self._keys = []
self.update(*args, **kwargs)
def __delitem__(self, key):
dict.__delitem__(self, key)
self._keys.remove(key)
def __setitem__(self, key, item):
if key not in self:
self._keys.append(key)
dict.__setitem__(self, key, item)
def __deepcopy__(self, memo=None):
if memo is None:
memo = {}
d = memo.get(id(self), missing)
if d is not missing:
return d
memo[id(self)] = d = self.__class__()
dict.__init__(d, deepcopy(self.items(), memo))
d._keys = self._keys[:]
return d
def __getstate__(self):
return {'items': dict(self), 'keys': self._keys}
def __setstate__(self, d):
self._keys = d['keys']
dict.update(d['items'])
def __reversed__(self):
return reversed(self._keys)
def __eq__(self, other):
if isinstance(other, OrderedDict):
if not dict.__eq__(self, other):
return False
return self.items() == other.items()
return dict.__eq__(self, other)
def __ne__(self, other):
return not self.__eq__(other)
def __cmp__(self, other):
if isinstance(other, OrderedDict):
return cmp(self.items(), other.items())
elif isinstance(other, dict):
return dict.__cmp__(self, other)
return NotImplemented
@classmethod
def fromkeys(cls, iterable, default=None):
return cls((key, default) for key in iterable)
def clear(self):
del self._keys[:]
dict.clear(self)
def copy(self):
return self.__class__(self)
def items(self):
return zip(self._keys, self.values())
def iteritems(self):
return izip(self._keys, self.itervalues())
def keys(self):
return self._keys[:]
def iterkeys(self):
return iter(self._keys)
def pop(self, key, default=missing):
if default is missing:
return dict.pop(self, key)
elif key not in self:
return default
self._keys.remove(key)
return dict.pop(self, key, default)
def popitem(self, key):
self._keys.remove(key)
return dict.popitem(key)
def setdefault(self, key, default=None):
if key not in self:
self._keys.append(key)
dict.setdefault(self, key, default)
def update(self, *args, **kwargs):
sources = []
if len(args) == 1:
if hasattr(args[0], 'iteritems'):
sources.append(args[0].iteritems())
else:
sources.append(iter(args[0]))
elif args:
raise TypeError('expected at most one positional argument')
if kwargs:
sources.append(kwargs.iteritems())
for iterable in sources:
for key, val in iterable:
self[key] = val
def values(self):
return map(self.get, self._keys)
def itervalues(self):
return imap(self.get, self._keys)
def index(self, item):
return self._keys.index(item)
def byindex(self, item):
key = self._keys[item]
return (key, dict.__getitem__(self, key))
def reverse(self):
self._keys.reverse()
def sort(self, *args, **kwargs):
self._keys.sort(*args, **kwargs)
def __repr__(self):
return 'OrderedDict(%r)' % self.items()
__copy__ = copy
__iter__ = iterkeys
Nuitka-0.5.0.1/nuitka/freezer/ 0000755 0001750 0001750 00000000000 12265271051 016276 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/nuitka/freezer/BytecodeModuleFreezer.py 0000644 0001750 0001750 00000005373 12265264105 023111 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Freezer for bytecode compiled modules. Not C++ compiled modules.
This is including modules as bytecode and mostly intended for modules, where
we know compiling it useless or does not make much sense, or for standalone
mode to access modules during CPython library init that cannot be avoided.
The level of compatibility for C++ compiled stuff is so high that this is not
needed except for technical reasons.
"""
from nuitka.codegen import ConstantCodes, CodeTemplates
from nuitka.codegen.Indentation import indented
from nuitka import Options
from logging import info
frozen_modules = []
def addFrozenModule(frozen_module):
assert not isFrozenModule(frozen_module[0]), frozen_module[0]
frozen_modules.append(frozen_module)
def getFrozenModuleCount():
return len(frozen_modules)
def isFrozenModule(module_name):
for frozen_module in frozen_modules:
frozen_module_name, _code_data, _is_package, _filename, _is_late = \
frozen_module
if module_name == frozen_module_name:
return True
else:
return False
stream_data = ConstantCodes.stream_data
def generateBytecodeFrozenCode():
frozen_defs = []
for frozen_module in frozen_modules:
module_name, code_data, is_package, _filename, _is_late = frozen_module
size = len(code_data)
# Packages are indicated with negative size.
if is_package:
size = -size
frozen_defs.append(
"""\
{{ (char *)"{module_name}", (unsigned char *){data}, {size} }},""".format(
module_name = module_name,
data = stream_data.getStreamDataCode(
value = code_data,
fixed_size = True
),
size = size
)
)
if Options.isShowInclusion():
info("Embedded as frozen module '%s'.", module_name)
return CodeTemplates.template_frozen_modules % {
"frozen_modules" : indented(frozen_defs)
}
Nuitka-0.5.0.1/nuitka/freezer/Standalone.py 0000644 0001750 0001750 00000036672 12265270763 020767 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Pack and copy files for standalone mode.
This is in heavy flux now, cannot be expected to work or make sense on all
the platforms.
"""
import os
import subprocess
import sys
from logging import debug, info
import marshal
from nuitka import Utils
from nuitka.codegen.ConstantCodes import needsPickleInit
python_dll_dir_name = "_python"
def getDependsExePath():
if Utils.getArchitecture() == "x86":
depends_url = "http://dependencywalker.com/depends22_x86.zip"
else:
depends_url = "http://dependencywalker.com/depends22_x64.zip"
import urllib
if "APPDATA" not in os.environ:
sys.exit("Error, standalone mode cannot find 'APPDATA' environment.")
nuitka_app_dir = os.path.join(os.environ["APPDATA"],"nuitka")
if not Utils.isDir(nuitka_app_dir):
os.makedirs(nuitka_app_dir)
nuitka_depends_zip = os.path.join(
nuitka_app_dir,
os.path.basename(depends_url)
)
if not Utils.isFile(nuitka_depends_zip):
print("""\
Nuitka will make use of Dependency Walker (http://dependencywalker.com) tool
to analyse the dependencies of Python extension modules. Is it OK to download
and put it in APPDATA (no installer needed, cached, one time question)."""
)
reply = raw_input("Proceed and download? [Yes]/No ")
if reply.lower() in ("no", "n"):
sys.exit("Nuitka does not work in --standalone on Windows without.")
info("Downloading", depends_url)
urllib.urlretrieve(
depends_url,
nuitka_depends_zip
)
nuitka_depends_dir = os.path.join(
nuitka_app_dir,
Utils.getArchitecture()
)
if not Utils.isDir(nuitka_depends_dir):
os.makedirs(nuitka_depends_dir)
depends_exe = os.path.join(
nuitka_depends_dir,
"depends.exe"
)
if not Utils.isFile(depends_exe):
info("Extracting", depends_exe)
import zipfile
depends_zip = zipfile.ZipFile(nuitka_depends_zip)
depends_zip.extractall(nuitka_depends_dir)
assert Utils.isFile(depends_exe)
return depends_exe
def loadCodeObjectData(precompiled_filename):
# Ignoring magic numbers, etc. which we don't have to care for much as
# CPython already checked them (would have rejected it otherwise).
return open(precompiled_filename, "rb").read()[8:]
module_names = set()
def _detectedSourceFile(filename, module_name, module_names, result, is_late):
source_code = open(filename,"rb").read()
if Utils.python_version >= 300:
source_code = source_code.decode("utf-8")
filename = filename.decode("utf-8")
if module_name in module_names:
return
debug(
"Freezing module '%s' (from '%s').",
module_name,
filename
)
result.append(
(
module_name,
marshal.dumps(
compile(source_code, filename, "exec")
),
Utils.basename(filename) == "__init__.py",
filename,
is_late
)
)
module_names.add(module_name)
def _detectedShlibFile(filename, module_name, module_names, result):
if Utils.python_version >= 300:
filename = filename.decode("utf-8")
parts = module_name.split(".")
if len(parts) == 1:
package_name = None
name = module_name
else:
package_name = ".".join(parts[:-1])
name = parts[-1]
from nuitka.nodes.FutureSpecs import FutureSpec
from nuitka.nodes.ModuleNodes import PythonShlibModule
from nuitka import SourceCodeReferences
source_ref = SourceCodeReferences.fromFilename(
filename = filename,
future_spec = FutureSpec()
)
shlib_module = PythonShlibModule(
package_name = package_name,
name = name,
source_ref = source_ref
)
from nuitka import ModuleRegistry
ModuleRegistry.addRootModule(shlib_module)
module_names.add(module_name)
def _detectImports(command, is_late):
# print(command)
# Print statements for stuff to show, the modules loaded.
if Utils.python_version >= 300:
command += r'import sys; print("\n".join(sorted("import " + module.__name__ + " # sourcefile " + module.__file__ for module in sys.modules.values() if hasattr(module, "__file__") and module.__file__ != "")), file = sys.stderr)' # do not read it, pylint: disable=C0301
process = subprocess.Popen(
args = [sys.executable, "-s", "-S", "-v", "-c", command],
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
_stdout, stderr = process.communicate()
# Don't let errors here go unnoticed.
assert process.returncode == 0
result = []
debug("Detecting imports:")
# bug of PyLint, pylint: disable=E1103
for line in stderr.replace(b"\r", b"").split(b"\n"):
if line.startswith(b"import "):
# print(line)
parts = line.split(b" # ", 2)
module_name = parts[0].split(b" ", 2)[1]
origin = parts[1].split()[0]
if Utils.python_version >= 300:
module_name = module_name.decode("utf-8")
if origin == b"precompiled":
# This is a ".pyc" file that was imported, even before we have a
# chance to do anything, we need to preserve it.
filename = parts[1][len(b"precompiled from "):]
if Utils.python_version >= 300:
filename = filename.decode("utf-8")
if module_name in module_names:
continue
debug(
"Freezing module '%s' (from '%s').",
module_name,
filename
)
result.append(
(
module_name,
loadCodeObjectData(
precompiled_filename = filename
),
"__init__" in filename,
filename,
is_late
),
)
module_names.add(module_name)
elif origin == b"sourcefile":
filename = parts[1][len(b"sourcefile "):]
if filename.endswith(b".py"):
_detectedSourceFile(
filename = filename,
module_name = module_name,
module_names = module_names,
result = result,
is_late = is_late
)
else:
_detectedShlibFile(
filename = filename,
module_name = module_name,
module_names = module_names,
result = result
)
elif origin == b"dynamically":
# Shared library in early load, happens on RPM based systems and
# or self compiled Python installations.
filename = parts[1][len(b"dynamically loaded from "):]
_detectedShlibFile(
filename = filename,
module_name = module_name,
module_names = module_names,
result = result
)
return result
def detectLateImports():
command = ""
# When we are using pickle internally (for some hard constant cases we do),
# we need to make sure it will be available as well.
if needsPickleInit():
command += "import {pickle};".format(
pickle = "pickle" if Utils.python_version >= 300 else "cPickle"
)
# For Python3 we patch inspect without knowing if it is used.
if Utils.python_version >= 300:
command += "import inspect;"
if command:
result = _detectImports(command, True)
debug("Finished detecting late imports.")
return result
else:
return ""
def detectEarlyImports():
# TODO: Should recursively include all of encodings module.
command = "import encodings.utf_8;import encodings.ascii;"
if Utils.getOS() == "Windows":
command += "import encodings.mbcs;"
# String method hex depends on it.
if Utils.python_version < 300:
command += "import encodings.hex_codec;"
command += "import locale;"
result = _detectImports(command, False)
debug("Finished detecting early imports.")
return result
def detectBinaryDLLs(binary_filename, package_name):
result = set()
if Utils.getOS() in ("Linux", "NetBSD"):
# Ask "ldd" about the libraries being used by the created binary, these
# are the ones that interest us.
process = subprocess.Popen(
args = [
"ldd",
binary_filename
],
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
stdout, _stderr = process.communicate()
for line in stdout.split(b"\n"):
if not line:
continue
if b"=>" not in line:
continue
part = line.split(b" => ", 2)[1]
if b"(" in part:
filename = part[:part.rfind(b"(")-1]
else:
filename = part
if not filename:
continue
if Utils.python_version >= 300:
filename = filename.decode("utf-8")
result.add(filename)
elif Utils.getOS() == "Windows":
depends_exe = getDependsExePath()
env = os.environ.copy()
path = env.get("PATH","").split(";")
path += sys.path
if package_name is not None:
for element in sys.path:
candidate = Utils.joinpath(element, package_name)
if Utils.isDir(candidate):
path.append(candidate)
env["PATH"] = ";".join(path)
subprocess.call(
(
depends_exe,
"-c",
"-ot%s" % binary_filename + ".depends",
"-f1",
"-pa1",
"-ps1",
binary_filename
),
env = env,
)
inside = False
for line in open(binary_filename + ".depends"):
if "| Module Dependency Tree |" in line:
inside = True
continue
if not inside:
continue
if "| Module List |" in line:
break
if "]" not in line:
continue
# Skip missing DLLs, apparently not needed anyway.
if "?" in line[:line.find("]")]:
continue
dll_filename = line[line.find("]")+2:-1]
assert Utils.isFile(dll_filename), dll_filename
# The executable itself is of course excempted.
if Utils.normcase(dll_filename) == \
Utils.normcase(Utils.abspath(binary_filename)):
continue
dll_name = Utils.basename(dll_filename).upper()
# Win API can be assumed.
if dll_name.startswith("API-MS-WIN"):
continue
if dll_name in ("SHELL32.DLL", "USER32.DLL", "KERNEL32.DLL",
"NTDLL.DLL", "NETUTILS.DLL", "LOGONCLI.DLL", "GDI32.DLL",
"RPCRT4.DLL", "ADVAPI32.DLL", "SSPICLI.DLL", "SECUR32.DLL",
"KERNELBASE.DLL", "WINBRAND.DLL", "DSROLE.DLL", "DNSAPI.DLL",
"SAMCLI.DLL", "WKSCLI.DLL", "SAMLIB.DLL", "WLDAP32.DLL",
"NTDSAPI.DLL", "CRYPTBASE.DLL", "W32TOPL", "WS2_32.DLL",
"SPPC.DLL", "MSSIGN32.DLL", "CERTCLI.DLL", "WEBSERVICES.DLL",
"AUTHZ.DLL", "CERTENROLL.DLL", "VAULTCLI.DLL", "REGAPI.DLL",
"BROWCLI.DLL", "WINNSI.DLL", "DHCPCSVC6.DLL", "PCWUM.DLL",
"CLBCATQ.DLL", "IMAGEHLP.DLL", "MSASN1.DLL", "DBGHELP.DLL",
"DEVOBJ.DLL", "DRVSTORE.DLL", "CABINET.DLL", "SCECLI.DLL",
"SPINF.DLL", "SPFILEQ.DLL", "GPAPI.DLL", "NETJOIN.DLL",
"W32TOPL.DLL", "NETBIOS.DLL", "DXGI.DLL", "DWRITE.DLL",
"D3D11.DLL", "WLANAPI.DLL", "WLANUTIL.DLL", "ONEX.DLL",
"EAPPPRXY.DLL", "MFPLAT.DLL", "AVRT.DLL", "ELSCORE.DLL",
"INETCOMM.DLL", "MSOERT2.DLL", "IEUI.DLL", "MSCTF.DLL",
"MSFEEDS.DLL", "UIAUTOMATIONCORE.DLL", "PSAPI.DLL",
"EFSADU.DLL", "MFC42U.DLL", "ODBC32.DLL", "OLEDLG.DLL",
"NETAPI32.DLL", "LINKINFO.DLL", "DUI70.DLL", "ADVPACK.DLL",
"NTSHRUI.DLL", "WINSPOOL.DRV", "EFSUTIL.DLL", "WINSCARD.DLL",
"SHDOCVW.DLL", "IEFRAME.DLL", "D2D1.DLL", "GDIPLUS.DLL",
"OCCACHE.DLL", "IEADVPACK.DLL", "MLANG.DLL", "MSI.DLL",
"MSHTML.DLL", "COMDLG32.DLL", "PRINTUI.DLL", "PUIAPI.DLL",
"ACLUI.DLL", "WTSAPI32.DLL", "FMS.DLL", "DFSCLI.DLL",
"HLINK.DLL", "MSRATING.DLL", "PRNTVPT.DLL", "IMGUTIL.DLL",
"MSLS31.DLL", "VERSION.DLL", "NORMALIZ.DLL", "IERTUTIL.DLL",
"WININET.DLL", "WINTRUST.DLL", "XMLLITE.DLL", "APPHELP.DLL",
"PROPSYS.DLL", "RSTRTMGR.DLL", "NCRYPT.DLL", "BCRYPT.DLL",
"MMDEVAPI.DLL", "MSILTCFG.DLL", "DEVMGR.DLL", "DEVRTL.DLL",
"NEWDEV.DLL", "VPNIKEAPI.DLL", "WINHTTP.DLL", "WEBIO.DLL",
"NSI.DLL", "DHCPCSVC.DLL", "CRYPTUI.DLL", "ESENT.DLL",
"DAVHLPR.DLL", "CSCAPI.DLL", "ATL.DLL", "OLEAUT32.DLL",
"SRVCLI.DLL", "RASDLG.DLL", "MPRAPI.DLL", "RTUTILS.DLL",
"RASMAN.DLL", "MPRMSG.DLL", "SLC.DLL", "CRYPTSP.DLL",
"RASAPI32.DLL", "TAPI32.DLL", "EAPPCFG.DLL", "NDFAPI.DLL",
"WDI.DLL", "COMCTL32.DLL", "UXTHEME.DLL", "IMM32.DLL",
"OLEACC.DLL", "WINMM.DLL", "WINDOWSCODECS.DLL", "DWMAPI.DLL",
"DUSER.DLL", "PROFAPI.DLL", "URLMON.DLL", "SHLWAPI.DLL",
"LPK.DLL", "USP10.DLL", "CFGMGR32.DLL", "MSIMG32.DLL",
"POWRPROF.DLL", "SETUPAPI.DLL", "WINSTA.DLL", "CRYPT32.DLL",
"IPHLPAPI.DLL", "MPR.DLL", "CREDUI.DLL", "NETPLWIZ.DLL",
"OLE32.DLL", "ACTIVEDS.DLL", "ADSLDPC.DLL", "USERENV.DLL"):
continue
result.add(dll_filename)
os.unlink(binary_filename + ".depends")
else:
# Support your platform above.
assert False, Utils.getOS()
return result
def detectUsedDLLs(standalone_entry_points):
result = set()
for binary_filename, package_name in standalone_entry_points:
result.update(
detectBinaryDLLs(
binary_filename = binary_filename,
package_name = package_name
)
)
return result
Nuitka-0.5.0.1/nuitka/freezer/__init__.py 0000644 0001750 0001750 00000001501 12265264105 020406 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
""" Dummy file to make this directory a package. """
Nuitka-0.5.0.1/images/ 0000755 0001750 0001750 00000000000 12265271051 014606 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/images/Nuitka-Logo-Symbol.png 0000644 0001750 0001750 00000010544 12265044767 020730 0 ustar hayen hayen 0000000 0000000 ‰PNG
IHDR Æ Ç BÇ,ø gAMA ± üa sRGB ®Î é cHRM z& €„ ú €è u0 ê` :˜ pœºQ< pHYs H H FÉk> bKGD ùC» YIDATxÚíÝÛo õðïovv×ÆŽq(—%ÁÄ€Jqm!y€´i@ñ¥‘ÚªÞ<ö¡T¨U[©·'úTU« ÁVˆ–[…Z¨š‡¶`lǪJLl !NìÓo{wî;ç·;“÷2?ïþ>{Ι™ýA˜™™Ùè8Îv [ Ø ` Œ5ñö‚‡ý ÔR?^dÍOkÿ]ÿÑ1À×®6=¶Ç=Œ®Ò‘°g÷yÞÓ?JÈýÏÆÉÚ…ø_m S^nÀG – ¼cŒ™ðr©TznÓ¦M $y"7Þ›#æàÁƒÛEäa Û”9åãÅü¢`rÎK‰Bš„H
ìßǟ羋õåÉq~Ø$OÃíŸ`°“`8 .¨ÿû3"r+€ï.//×fgg_ ðØÀÀÀ3ƯeÓÓÓ7¿õÖ[¿‘›9ÍíE\¾÷†ZCqfDI¶Ýž(¸À½³³³oÌÌÌ<488øJ”° 7Æ™™™ùã8¯E£ÄOž)Šäh{ cþ2;;û³ýû÷—Á˜œœì}ÆóH]££QQ$ #"ßëëëûóáÇ/ˆUJMNNöº®û‚ˆÜÂišñDWB!b|·ã÷|™¢•mä ”
Š»_™œœ¼khhèX`Æ‘’ëº{ EN2E¬9©„¢ 1äºîÄääd%Æ ðNÓ¼¡6¡(ŒŽ/¹®û _SSS7ø>§i‘PÀŠÂàxhjjjë"b ÇyŒv>Q„—SD·!w ç1)i¾§§§wcØWXl¾³EÔ ÛA‘ãæ»1®=pàÀ7 üÆ cÌCœÂyͱºo½L!…Êð<ïa pÞ|óÍK ÜÃéšcB™ÕSÆÜ:55µÅq çËì-òŽBˆ"[ ;\ wHÿÈs… à Ÿ'Ê(rs®TÒ¸Ó‘ë9móŒ"ö(TP4®w \É©«ˆb ÒN…r¥`=§¯•´¡ˆBˆ"Û¨: º8k‰"5Š‚ö§.oež§ŽB$¬ÖE‘ó³kÃáÔµÕg衈,i˜)£ciK)FP5£‡"²”RAQØãÌölEQ÷LFaQ€(ƒ(ˆ‚0rÛcè¡èh âaómÂuâ!d•}l¾é *Lº`3at2‹¶ €l¾5:(œ/Eì1:#[ˆ>ŠÀs¥ôQˆxì1–SJ(bd*L!…M „Qw
(xä›Ñb¡†Bˆ‚0ˆ‚(Ø|Ɇ&
8ÀçYAÁæ›Ñ™(â6ß™£`)Åh½˜ÒC! F‘9
6ß ¢h~lA“{ m"PHŒÅô2{ FKyC Eð§¶2Š¦FP„ŸH¨T>ñ8£õrJ…ØEq4ßì1¬$ }á—3ÎzémŠ8‡\Î\˸F¦ˆõ¥ f
–Rç ŠßùVCÁãŒE!š(pî¡ [-†6
QDq~ƒÍ·Õl¡ˆÂ„4ÀZåW ad’7Ô2…D¤+…ò‰«„0ò€BâŒ!Óò©ø}K) ¥”:
I°°sFåÓêÛy ƒ‘QæÈL(Åy¥÷pE÷³!Û–ˆ ÓøZž=â1
¯-aä $öö5ʧª9†jåZ_Í0£‹dJ6{ßÞ‹ÎD‘²|Šœ¤9FÁæ» (Bk|…ò©è(Ø|ÛסƒÂD¬vžõÞ§‚£`)UQÛfù”êC¥”E z(âaù7SFPˆúiçç
ö6Hˆ&Æjç,Ÿ}(0cèw2…Äîqˆ"Þc Ê eIÖ–"ŠX% ‹{¦¬£öiÏ0& {.ÔPÄ[»–(b?VØ|[¬¦4Q$\íœ("ÇÊŒa³ÑPBóÒD‘à¼1Â(4
E
ÌVmh¢¢È ì1ÔMÔOäÓD!Qßà#ŠÄceÆ°—0ÔP0Sd‹‚0
ƒ"ê`Q$+aX•A9Að8†… ÃSGÞcEšÅ©™1ÚPNe"vÖ ŠX(£
å”] Š(î =BY_Ûƒ0¬öD‘l¾ „Âï}<ôÉV úø®s*#g\%$ç(VŽ©›€í %¾)‚¥”Õ3
* ˜„Á—Àž=”‘y)Å ÃF¥‹B`À÷‘=F®bC·iZ![@MOO¦³é ѽPÃ8ý²SS%'Á%dÃxü“ø–¢Í—Çÿ͉„T±þ{ÉšVq©ÿϼúê«ü¨QŽékØ;}5/{ð›"‘(¤Hð•eƒ¶¹êç$ ÂA!ké"ÇàWBJ’× p›k×ÿba!®é/ad°Š’!Š< à•¬âp°k°‚½3'áyDÑÉ( Àí©ÌÃ--¥ÚË’î.¢³¤÷•VÇäÿ¸ãK±ìuùÞûÚ%Œ¢Š½Ó'á Qt*
À|tè'Ò×u4鮖ɶQZ¼_Àm‘ãkã_¬É¹ûpªÖøò½yt¹Žƒ(:EºUBˆ"ô~]¥y m|•ÒñÀ—ðº ]Œ\S…cˆ¢Q €“¨Ç ŠX÷ë*-àºþ}(;ÇWV;÷ùïº+
9ˆ¢ãP$»p Q$º_·»€¡þ}¨”>
Í ÷VaŒ!ŠB!±K)¢Hu¿Ó8ÊN0ŽÏ_ä⾫+0ÆE‡ ˆw ƒ(Z·;á+eUPÜpq;ë8ˆ¢ý(¢aE&ãïvahã Pv_ê/\RÆ}[ª0kÎ !Šv €„•RD‘éøÏsçp]ÿ\s"°!¿á"_½ººòí
¢hŠàŒA*ãïqçê=Gpæ¸ñ’òY DÑâ ƒ(TÇßSžÃPÿÜ 7]ZÁÎÁn¢h ŠæRŠ(¬Œ¿§<‡áþ ¸Î‰Ð̱} ‹(l£hj¾‰Âêø{ÊG1Ü¿¥žã–Ï–±} J6Q¬é1ˆ¢-ã_Á^VÝvYÛ¯î&
åÓêçvˆ¢½ãï- ÅpÿS8*+eQ¨–Oð-¥ˆ¢mãï-ÏÕqœ ÄqûåUÜ;ÐMŠå“ ¢h÷øOgŽ’Y ì9n¿¼‚{¯ê&
¥ò) cE»Çß[>‚¡þ}(™S™ãŽ+ª¸çª.¢P(ŸB`E»ÇßWù Ãý¡8îü\¶mî&ŠŒË§ð ƒ(Ú>þ¾Êû‘8¶nêÂÖM]D‘aùÜcEÇŒ?Žm›»q÷¦n¢È¨|Zãô¥°ˆ¢óÆßWyÃýpp2°!߶¹kQ´\>3ð‡r~÷ Qtðø?>u ^ŸAMʾo¢°°è5®÷‡T •i (I ‘Ñ’M}cÚ-¹Ì?þ¾Ê{ÞðÞ8æÃà‚n®—e8D‘ñ¯¯þ·~œc‰³ÖB¸D‘Ÿñ¯¯¬àx}nW`YÅÈ&Ìëÿø½TÜ) ¹$.‘x¿´6–4ã ž£ —ö¼ÞÊœ*ê÷Žß€Ù…mœ½šãÃãóUÈ06t F/æT3]ØÂ
ŒÌz Föa©ücèe ®vžO|ߘ1òDmGC-c|ZsP“ºÙLþ&Ik½zÂ
Ç[í?ùLs °®bˆ¢¨0^<Ú‡÷ƒwýIÙgÛËÙCïMWj8MÀç¿r ùâgŸ¿á´††ç\}¿ú0Cžßï«¡k· z¿³À†nÞX%Š¢Âý´%
_§Ÿ?´ÎW>Π£M=Q„£ˆ®ß´>2¬Ã Š(‚p´ãˆ{ ñ<¢H…"¬Î×GÁ C»”"ŠT(âíÒþæ C±”"Šä™"b‚E`EQˆzÑø&EÜ¿9¨Î·ƒ‚=†½RD‘E¼`¦È9 ¢H†Bâï®%Šü6ßD‘ Eºƒ|¡`eaw-Qïù&ŠT($æI„
(x¡µÝµD‘E²™)òWJE*‘óÔÚZ¸ µæ›(Ò ¢(tA)Q˜ˆ:_{ +¥Q$ÍíDÁÌa!clé]ÄÅU§éE— %[üvW6^o- A•€óˆšç˜ø¬1&>CYý;sæ“üè§À켧ŒÂE¡a¬;™ãáߟߘó0;¯)bwßÙ£
}E¬U¥~-gM"&ú‹Jj(x ÃÂ^©F¢c3iP´3Sad CEœoñat¤ ¢`°Çh˜RŠ(ê¥Zà¹RÊ(x®3Fút¡Š"äûV/yÌ ŒT6,£œÀDATL©¢6¢ ÂHËBEº¬Al¾Û ž'ê(D¼çKe‚Í73FëY#óL•1Qp×-at2
!
ÂÈk󋢕ƒ|DÁ £¨($â‹JÊ(Øc0c¤¤¡‰"b f
ÂèÐ|¡BÚ…‚H£e š(~Q‰(Øc´ÝDý8†.
²àš.Šèc( fŒ C9S$^ñ<³LA„‘A§¡‡BÚ‡‚6£%DÁ »(¤(¨ƒÍwêæ[…„-ºf›ofŒp Ù] ’™‚0
„"Ñ:DAÁÂ
i
aÑ*M§Î×CÁ ƒ£z(’ä-f
Âè¤.CEº Tf„‚Ù‚0Zm4ÔPH¢A{Œ0á‰o1\‰#ƒRJE¬ÅÔ2QFúÎ[1SŹ £Vd z(²:òMÚc ÐW(¢BŽcˆ'VP°ÇÐB€#ͪ("K)5a!>t ÌEúË)Ç Qä.f ù{qû ¢`$cÌkŽˆ¼TLº(Ò ù&Šíwcà÷1ÆÆÆž/•J7øc®÷JYË`¦ÈO¼X«Õn {:ènØ£wíÚõ ;Ÿxâ‰íƘGÜ’«¼-(@ÿ2Æ<2::º/êŽnœg Às{÷îý¢çyß4ÆìpQEâ}c̳ ~;22òRýLÈ0i·¶gÏž×u‡ lPé¤Wâíù¥ §,
œýñ‚ñ4Ýäó›ÆèrK'îÙÒûÏÆß_³áÕáuÕ…þ8Ï‘6>Yꟛž»m’sÞ?<Ï[2Ƽ#"SãããÓižãÿ†ö`Âþ z %tEXtdate:create 2013-08-08T10:34:23+02:00íq‚# %tEXtdate:modify 2013-08-08T10:34:23+02:00œ,:Ÿ IEND®B`‚ Nuitka-0.5.0.1/images/Nuitka-Logo-Horizontal.png 0000644 0001750 0001750 00000016641 12265044767 021620 0 ustar hayen hayen 0000000 0000000 ‰PNG
IHDR Þ a xuÀ× gAMA ± üa sRGB ®Î é cHRM z& €„ ú €è u0 ê` :˜ pœºQ< pHYs H H FÉk> bKGD ùC» –IDATxÚíyœ UµÇ¿ÝÓ3™%ë$˜@A’ ‚ ¸E@QÙƒKØ…ˆd‘M6‚ ø@(ûN@>AÀ‡²ˆ„$1d'û$™™¾ïS=éîé®{k¦«»ªû|?¶ºnuWߪ[¿:÷žEQzÏ8`*° hf —ýµk 8 X˜¼W'ðТ]¤(Š¢¸rIAɼæÛiñªO `?í"EQÅ…,‚’ßA5ÞO3-}tˆ^JµCR»@Q” Ò üÜ¡ÝÇó´»EHzÓ“œ3gÎîÀÀ( ±’9}‰¹b}Úg
$
iylØkDÃì†:ÓÙ§nEKcránmsm×?ëÍæK7t¶®íéqc:‰Ä`Nccãs#GŽlÓKL©bö†8¶ œ£]¦(yÂûî»ï6¶··Ÿ6{öì³€Eå §/êàébbƈ|ƒÁ°Û°:ê“ Ù¬ñ™LÃ,‘Í›å1òþ¢
û²ÞônFÌxŸÕÖÖÖ6kÖ¬)©Têâ·Þz‘^jJ²C€¶›Àrí6¥Öéšjž;wî– /WGIt"—'ºÞÿòZE·À^½¥ ˜ØÑÑñÖÌ™3÷ÖKM©æû‡#uÚeŠâ
œ3f´¶··OK§ÓcŒ1DíHtó÷uÝðŽ0õí·ßÞ=Äóx=°ØHqçõÀëÀ®1½V¿,-òÛÖçêp.;s´]î?EQáH¥R·#—?Ñín¹º‰î¦m¥'‘H4MŸ>=ŒX½p
0¨÷i×<ôáµz"0¸È¶fà:œËÎ4$6Õ…{C`Š7á}ë·ö4Æ EK·Õk]“eõfíä&º,澆766žÂ9ì+[ÄðZµ9ø5ép.;+‘^ Ë´»ÅÞD"qlµXº¢§&'GÑ
ÏâÍ:þãBøØTö™|B/¥\Üà³}p '¾Š¢ IcÌ—â`íºŠnî¾é@¢[†ß²ÝŒ3†•ø &z°O à:½ü•1 8 x
øØ ¼
\‰,o¼¦]¤(¹Ö҈غ΢[À¥¹â–nN‡§R£"òôÿ
$MÝÓ: ”0Í{)Šâ`ñö¼ÅHt³×t Ñ-Ïoéèèè¡óþYŠ'†S<5ãGÀAÚEJ$„7Dt»… 9Šn¹-߈ð)ÔX© ¾JñèŒH^iE©8©bq²‘Þt0Ñë2[GÝE7ýQb.þ,Ñá Äœ–^nWµx7Éf0Ñíf/;‰n͆ ~¦CAQE-Þ<ítÝ®õTd×Ut-™²JE"‘ˆbŸ ܼ¢CBQE-^±x]E·j;ˆî¦¶UÉ;t/Ë” üR‡ƒ¢(ŠZ¼›¤×Ut³,Þî"k]þoggg%:ðMà$íb1ö¾Ü©ÃBQ¥Æ-^‚ˆnŽÕDt«~÷|ìyuŽ: (Š¢„+¼QáÍXÑÍηœ&º•ª´T&cÏ—;ÜhEQ¥–-Þ ¢›g*;Zº5áÕ|#0ËÒæGÀÖ:4EQÂ! q¼Y©]tÅÒ5ÙeDW2]Uy o;p&ð¸O›L çCtx(Š¢Ô¨ÅDt»—ãu]j'qÆH2{?¾¢ÃCQ¥f-^œE×PÀ«9€èÖH ï$` üó4ß ìtè0QE©9‹×]té¶Æë*º5•¹j&ð+K› €Suˆ(Š¢Ô¤Åë.ºÆ•¹*€èšò¬ñ¦Óé¨të¥À`3Ÿ6— q½Ku¨(Š¢Ô’Å@t [®vÑÁ\Í+ -m¢yœEQjÏâ%€èšœ\Í&€èZS¯Ñ’|“ç'ßÈ:–‚ïülrÚM ¡^½8Ø٧͉H çWu¸(E¸8h.°m’í/ÚMŠâ oLT×Yt3!Ht1†¾©ùôå‚„ ù×û…ü\Ñ éÆÖul8ø»eVäFà :\”"œ^Dt†zo*¼Š’¹©Æ!sUÚ¸ˆnÆÍÊeè•H–Zt#:ý,p¿¥ÍçoÕ虈¤ÚÌ;ùtÿô‹Ð±Ž>,p¬˜‹8Ì…As/·ìéÞáq|]oùÌý|Vök)ðµ*»®[‘±³E~÷GÀ6e<¦`àà à·ÀŸ—‡Ð÷eÀzoü–ïÿ ®ŽC"2"»”‹×ÕÒí¶V1Ñî¬þYޥɧÍÕÀ£ÞÅ^KœRD\“ÀîÀç°ÇE—‹ À°"Û> ü'&ý~ 06"Ç2Ø³Ú ¯’kz4ð$0ÂÒn#â
ÞÃâAÞÃýØ€šÔä½[ؾ øp·wî6¨Å8DZ›èCý¢"º‘UÞ÷½'E?FP›yœ{¹]µg4éñ„Â探»Î{_Â1ìÜì}ö4d©âÓ!‚ÀáÀÀàbÄa´òÂkÅ>½ÜõßùÎb‘ÝH;²ý øÀÒæGžå¤(Jüh¦:ˆn'pð¯ÿþÀsÞçNú—ñ··"á‘ï ǨÅëbíµtM7Ñ5kíFIt!nê \œë`1ýBï_Š;À €] ÚžJi§Õ[¿"K2Ÿ¯p?´w ÷Vr(F¯ÿôrv¬o¡}£ º1ˆ¾xÞÒæP$ݤ¢„I› OI¹Yã·qâÔTJ~
|9bý1Þ{諯ï ü¦—sld[½‘ÝXÄLŸŽx’ú1™Ø„¢)1å)àˆ šeÀïbÜ—ÇcŸÍ˜‚=©NO Ñ~Ùx°÷²XÜ<]-]S°ÊŠn@^nG¼8‹±#ð}$¾WQÂ`Á½šˆHÑ4Ä[º–çhÁþønXžë ˜Ìñ^³½×"`-°Æûw-â¡ÜˆD lŽ„=íäYÖ{жý§«‹Ê*¼ñÈ\å.º¹¹šÓ*º=ã|àHü .îBó8+JTÙÖ³èê-í¦#Þ¿•p@yYO~xX`ßuÞkð⽌'Ä'"µÇ9|ÎyH ðËåúѱXã
"ºÝDNE·',.·´ˆ¬Ý(Š=Z‘ÚÛ¶Lyï_V—ñØæ ?A"$ÆxÂ÷d@ÑõãCïÞ´÷àa£‰êP‹·°ø:ˆn–Å»jãf¯úz–PfÍh˜üŽBS0ßrÁÿ6Ÿt$úÍ‹ÉÀ œì=5ã$$ ï5½Ï)Jd¨÷g[K»€‘×0Ɉú_k=‘-Ë‘,X×!5ÈýØØ x¡, ÑõþßIt³DuCg6töÔoI&“«b2xÛ½‹õqËŒÉÀõ^§(‘áfdm×
À!”'“Ùw<Ë{N…úã,dýwoK»‰eÞ8X¼É@¢køpu;M©„ÝY<}£±lÏÞP¸ëŒÏŸ†úT"Npž@¼Ký R¾ |¸GïwŠRqÎNp°eŽEÖUËey.¯`Ÿ¤‘¤¯ã?õ~„'¾ëÕâöÛºž…«;YµÑ.ºÆîxuMîºpVIÁ\ËÙ䊹Å{zS8SÄ ]Ÿ±©¤`~E%c`Üöƒ¶Ù@žäMÃø9h\
pô˜¦¼¾UýEw“`FOtÁàÔÕÑb¦wÁþЧÍHàLjÄ¢(ågWàH†*?&#kžµÈ͈#—ŸÁ¹OY„wûaÏÓ'U )KÁ”‹Ì.Ÿõ^ŽÀÉSl,Û³ÞkëÀŒ¥Óžn¦µ)ÉÑcû0ez«6ÄStãâÈV€K‘Ê7C|Úœ
܆”èR¥|Œ@fœlå DBlj•H,÷A>mv)Ç'Š€è‚¡)µ‚ O%•hÃCkc‚cÆ6ѯxŠ®8š¼ˆ›¿-£æqV”òÓ‚>ØÂÒîyïá9]ãýesž*K)ÊdTE7³Osj; ~”TRÖ»[›D|[ˆèLœ/Ø[°‡
†ÝsPQ”ÒÝ¿ïv¶´› |ƒ28
Å€Z¶ú”Wx#&º™m-õKÙqð£Ôe,ߌøÖ/Ñ5±Þ4p†C»ÉH@º¢(ár'¨~,D¢–kwâÙlc‹°"uÑͼגZÂèÁtY¾Cš“ »sK®øF\tMü/Úgû-mF§èøV”P9ûzí$+Õ<í®.>rh3¸<ÂqÑÍ _KýRvl}¸Ëò Ü”`ÂN-4¥ˆ‡èS
îÙØK¤]VŽ‹WQj”}H?:¸ÔWµ»rèÄž 3ô\ ÉnQÑͼ߷a©gùn `hK’cvn¡©>¡¢[ æ!ißü„æqV”0Ø )`ËÁp2âÁ«tg¥e{c,Þøˆnæ½¾õ‹Ù±õQ’lÀÃЖ$G{–¯ŠnY¸
˜ois’¦MQ”Ò0Iá:ÐÒîb¤´§RÜêõ#t •dÜD7ów¿†=øaêؼ_ ÇìÒ—¦TBE7|Öç8\¼“uŒ+JIh >aiw+²Ô£D˜T E7sÌý2zðü±ô:MÃúÖñ±Í<5k«™M9—3®MÝE—¬r‚…„“¬ÄYëÈyûv‰zÞ¾ù4¥Õ’VñnàTàs>m¾ŒîÓ¡¦(½âVàó–6O¢Ž1Þ˜Šnæïþ
3øÞXv(¦žQë9y÷‘íðD"1§Š?Ï ^¿®ó5H€›7EéG[Úü8qªªF† õ{G ±¶Ã€!Óî@d}¶É{õAfÞ’Þ¿‘ sLYß©úbHqÑÍÞ^Tt±ˆ.>3÷ó›ëWRÇ|ס
ôoø/Ÿ4•7— ŒñÕ€ÊÓÙÙYMƒáeà÷øWC…äq¾X˜ñاŽß¾¬‚ßÛØø40 OÜèWUï› öŠìÁÝl*ƒß÷ÝŒ(·6¾G*¹žöt³Õòr>¶àWø8“ÇYã Å=€;ð/|° I±(¦¿1ì‰äO> qȬú<‘®Ç›½¶ê4ý³R&—#SÊÅhBBŽÔîR'Fâ Ú² É\5+†¿¯8 øžgÑÖ©—÷e}:™#v›Jîeyìv9e •ò ”rXdö5tÿ¬MYœ²‹È{›5Á^#²K¿š kÎJ…˜ŒÄ
úÕ><£Ý¥(¾ôC††ZÚý{Òÿ¨1©vö]ʨ"²ÂûÞº>¬éHf‰î¦Š?›<€³Beº½çýÝ¥‘¦K+³=~sÛæ}GVèÍV’ì9< ¥›ÿ=J¹iGÒ×Mµ´»)»Õ©]¦(©îAÖ7mŒCŠÚ_ƒß•D|A®Ä¿¼hM̵t{*ºYåùºö.ºÝ…³ÞÕJ¥x {¦œ1ÞÔ’¢(…¹ ɯìÊ•ø‡ôEÅ‚ ©pVÑíD¦á× Ñ Qˆ”1&R¢kLº€øº‰n÷}£G"‘¨æ›Æ$¤ú‡_:»Ë`–¢(¹œ
œô Ü‹Ì$-‰àoú82¶c >Ë Ü3·€÷ zó½ûÉ
ïe«9ü °e$„7–®1“è‘è¢SÌQà-àWÀ}Ú´"ÎX§jw)Jû#iV{ÂpàNÄ+8J…î‡"> £z¸ð ðwd-û`U5œìTeD—"¢krÓ3vé‹èÆc·Êâx q)0Á2¥4¸xCï·%%©][z»óà"oüEFà‘ ˆn;²lõdéj]uÔŠˆ®)(ºÆäÕ]%2¬@2íøQ‡8Z)%~Ö.¨JÞsæŸ ûF䘯> ý:¤¢Ùà0$/õºj=¡ÉòO/û‹®É[ïuÝü}£ùªn¦[ÚŒCãzð2”êb2
ý[ìÞËIdÊy‹
óÎȬ–+ ±¼‹kaœ$+¿¦›+ºÝq]%R¤‘<Î6®¥†cùBàcÚUÅ:$d&AÆyÀ? ®{*<û1·eNàtïüÃ2 _Åï9IÑ°ˆˆn&éFδ³£èµx#Æßû-mFa//u¢ä¦>¥Z辉$ÉÈÐ å`~øY…Ž{7à‹í ’Dã—м–Ê[¼DOt»ž@Õ‹”q6öªDçÒs¯Ç(БãhŽy?*¹|ŸÂ i ßÆî½|’N²Ü ïØî§H•r3Œä‚NVÊ{¹¸¥›ÎÙ?ˆèªÅ9æa_—j"Ú™wl'ODŽóS¨Wsµpð?>Ûÿ\â0sG[.êo9´{ )¬#¢p‚Sㆬ¡£KË6Ýcº[ž&ïdòBg»—Ý<4+íhéšîe ƒÔéU¢ÈUÞS°ß$ð%dz:j´[¶ŠÈqî©—ZUð;ÜJh^d¬ÚߧÍ@à>¯ÝÆ2 ûXÇñp‘ø
‹1Q8É©ÍÃ= ]ù›í¢›ÉùœS•ÈUtcbU¦ÓéZ»‘¬C¦“ï´´»©Áµ@çõ–íQqhú²jVìywoà4ð àU`¤O»Ý4”åHX³»C›÷±çt“]£p¢Ë35å*ºÆtÏ@å(ºJ¤¹ xÞáiùä »m:
ëª
ÀÞz™Åš—¢÷A <—!ÎV6ëñûˆ£VØìæÐæÉ
?\!Â[ÞõM»èæì“N]õjŽ8g`w¹œèLÝf°å¿Ý6Çx 2XëÄy€]IÏF¼ˆ[dÀ-Àö!ÿ—øáT°G£kÄâ
&º…ÇŽ]t•Èó2v/ÆÁTÎ颶ø ©¼—äÑzyµ[nòàAK›¾Hx_sˆÇáòÐünûé𨜰ð-Þt K7Ï;9˜èÊw©ÅiÎÇžäü{DÄÂã‡ÚØ
ßVÀ!ª¹€Ä¹Ö*' ³-mF¿ ñ\f]*Y•ì¤¨œ¬²¬ñ:‹n±z¼¢«ë¼±`¿çG ’ù&*üÇ¡Í~<¾³#`qG…¶þí«€# úàXàÄŽÁÅs´R!oû!!w5bñ]“NçZ‡DW×xcÁd‡'ó/GhZèM‡6•²8·#ši•be{s•ÿþ× 8´û°Sß¿Ô¡Í°
õM¤–°ÊdñIÐAžÅëj骨ńÀ™í®%IÿßZÚ|–ÒúBø5Z‘(ßêó£
ôÁmÀí–6Èzo©ûc‰C›‘è“cq uª.‹7PV¬<Ë5ˆèÆÁâíèèÐÛ£ÔÛœfi³ÑÉãüW‡6g–ù˜NöéXW3¶õÃá52¾Nõ¬_?¶E’u”—e™¯”¹/†#qÌ‘¢Lóíî¢ÛÝru´t5_sܘ„ÝæÜ
=!çó¤ãSu¹¬Þ/×Ù–®aáµÍL ‰Èõ6mÈz¯màܪˆ¹òœC›Ë8KÓ² c5ó¡ÃÃܸ_³qs¢ºØ£Dßù"ö®AHÆ°éƒL§ïÖƒÔð…·,ö®£èv'µt«K°;d Eå³Í¬AŠuÛø â¸{ ù¬[‹l¿À²µ¯w"…9ü8®†Æ×Ø#ê‘|Î%ø¾ÕÀSí®"ÜT«ƒ€?Ùþ'Ëjèã$Y /€èfÇâ]Í\CV :´»‘ÊWÞ¹Úñ o"p]Ç{Š'ºÅ¦Í B’”جòjǶθ7Á×ÆãÌÙäÖô-Ä(à”f)Â¥¾î0OüZCø½»{ã`o K÷ǖϽÔgÙ,^'K7+‰FPÑU«7¶ÜL·´Ù™ÊOÎð¦®\˜„8mQ‚ïÝYcþ
ÅK®Fê¯Ú¨¯ëéßm~GÏËÃÅÍ3º ɽÌÒî«Ày%ø¾§· Úí
¼æ}o)ì= ¿ˆ)ÄÛ7,Ÿz4EyÖx]E×kkõh.$ºFãxcJ7(ˆÆ$ì+f"åÛz»8Ö»I¼ `i{n©økàzr)-¹%’3ص”b ð]ïq¥7û'> &8X'—•à× ^Õ.7»‘HE¦§Øýž‚;z3Ló¼qà÷o §;|fèñÞ)d]$Ü)¨ ¢[¸:‘]tãbñc4ž¨ðÍòÄË2Ê,ðîï Û÷CÒdž <ü
ø0XŒ8lÕyVÔ0ï&òdmj;Çï¸)xîB-ïóžu7ØA|_ðnü{û-òfzVÓnHøˈ‡l†=€›bÖ/O!Yã.òiSÜ
ì‚ÝCÜ¿x34®¥÷÷^ ¼™¢W¼‡œÞƒîjdú·/2-þI$v~oÜ‹ ¬ ÅEkØ'#eŒY l ¢æ]³© ¯éª]ë&ºq°*“Éä"ÕÙ‚œ|-âp‡wc
†Q‡¬+–zmñosªáíð Œ~äØþ Š;ácpLûæ`/Ëu8Ìß}é]щs¼‡È ‰+¶ œLl‹‘)íw¢r~“ÈtXØfž“èc
®®–n,,ÞummmóP
1h„¹p&pk,»¯,?qc\KWn2þ¸
oø¶gIú1™vîսΛ)x¿ù ïaãå(ߤ1æ±ò¬óº‰nîo0ÑÁïÓãǯåDî.7Ìù1¹ŒLÝUâ‰oŠwS[p¿ZÞ¥H•«°ÎMœk /¾‰=ÞùÕœloo¿{ÍÑÞ¼®¢ÛÍ«¹j,]€t2™¼ Åö”|no.³2bõ²ý‘|Îå`%’|àX$¶¸ë- µÂýÞÌD×Ř÷Ísˆï $ÄhT/¿k5RðäTdµ ¼¬ÿžHñxÝ
>û7….¼ãÇ_cŒ9Ãáf¬rÝÜêDÁD7âÖîM |ð«ªVîBœ^‚²±BÇûg¤ÜØyž5+ê*[{ýãÇMž~^AésóFÊQ¥ô«˜ûúq”¹xÔÁò»ÞGÄÙj;é7½ ì€ø?øq³Ï8¹=ìÎO vØa÷'‰ ²œ-Ý¢ŽQ±¶tžX²dɤ>{½C›¸MoŸFð¢æK+x¼ë‘iò‘HØÆã%èó
Hüî÷‚?ÁmÝò
$ "‘÷ÜâLEo¶‡Écˆì¥Ø×6‹±Ö³ ¶zò ÝÁ1z v Ÿ=z1 •Ïb$$o’õ¿=çùÌE‰öòŽõÇ™Ë}ÆÉ]aw|Wj¬C=ôʇ zè}cÌu”0Ww¯f»èš¬úºAD7‚^Í€:;;/˜8qbgHß±Y˘@áŒ+Kp‰
¯xâ{öõ–vÏ¢¹-ǽ¸Ó{5{7‚]‘˜Ü-½Î dµw³]í½–y7ÁHìîó!Z¥æFd격ÈͶÒçfâÑ{9ð9ÄyèÓÀ'ˆŽoì¬õú|>0Ë;Ï"!`½ˆß ¿@B̲I{Ÿÿ|úe…×—"qç[P8]âÀÎÇÍÞ«?¶õ$œn”÷ ;À»ž½q²ÒÛo>; I–2#ŽÓ
ÝR„M™2¥¥¥¥e’íä“ÞEÙc:Ó0ù…Õ½‰d«èn5¨¾ã¨¬ØsÄÔ¾Cšç§\-Ýis_¹±³¹ÒÊ»ø¯1æÑD"qÛᇠ> EQEñø‹$!ÄÊk %tEXtdate:create 2013-08-13T07:50:42+02:00Q6íù %tEXtdate:modify 2013-08-13T07:50:42+02:00 kUE IEND®B`‚ Nuitka-0.5.0.1/images/Nuitka-Logo-Vertical.png 0000644 0001750 0001750 00000023263 12265044767 021236 0 ustar hayen hayen 0000000 0000000 ‰PNG
IHDR U ? öcP gAMA ± üa sRGB ®Î é cHRM z& €„ ú €è u0 ê` :˜ pœºQ< pHYs H H FÉk> bKGD ùC» %¨IDATxÚíyœU}ÿßϽ“™Ìd²M@ö … BD«ÕŠ‚((¶
S¢?
H+¨XŠ²i«´?ÜpiíOÑŠ¿ÚbÕ €èÏV\jEIÌ"ûÈB $™ÌÌ=ýã<™äÞçœ{ç9Ïv?ï×k^YæÜ{Ï=÷yÞ÷{¶ï‰©qÇ wôööö EÑkŒ1GDQ4Ï3˜
ô«…&²}Ôð½ßvüxc&ükâßwúç«æNaïÁÚ„ÇN©mfÆ”5IÏÞäy›¾pËrÆÔÙ8ò’"5ù“ÀðPEk€_FQôÃ:èv]é© &Ï]wÝuH£Ñ8x 0¤ñcËÿÚÑ¡PÍn’3M„j0¼}áTž]ŸðøžØ·ÿ‡ÍîÒ4¦µxÍÄÇ4Lwo;§ ÅÀUccc_:ôÐCŸÒ•99zÔ³zõê=¢(ºblll)PS‹LBŒ„Š±åw}–¯’PŸ©Q‡ï1c>Y«Õ.^½zõŇ rÈ?DQ4¦+´3$‚Y»víqQÝ ¼]혓ˆ=„ÚÙxB:Bm¿¹3+Š¢Ï]»ö¦Õ«W匿MRÍŒ5kÖœoŒ¹ØSQt¡zÊ5„PMi›ú¨(Šn]½zõKtÕ©ûŸE—ÿãƘ ÕÙvÿ;ªiÑýÏJ¨%éþ7c/cÌMk×®=aÞ¼y?ÕÕªH5”PßH¨%‰PMîª)u»GQ4`ŒY¾jÕªçë*”TCõÀ•j‰² 5ŸµìBÝ©ýg×jµëVZ5]W£¤šk×®í®ÒpI•„JBŒXçEQt©®HI5Í‹ê|àµD …jâ2Éf•PÝœ{çwÎוéF‘—ƒ{î¹gÖððð_ª%Ê-ÔÝש62j‰'ªvsEV»x³®PEª“bÇŽ of¨%ª¡’M„j*ž´jÕª}u•Jª“¢Ñh,U+H¨* ucÌ[u¥Jª oCÕè2 Õ%· BžX£(zƒ®VÇ8‰š ñF¹1FIg‚µox¡öþg TS½ˆõ¥7ÝtSÏ1Ç 3ª+X‘j'RÕlg‰#Ô6ök)Bõg`ï½÷þ]¹’j§]穲
[Óê.Z3y µ²rÕ}!©vF£ÑЬé„jÚS•P;ì]ÌÒE+©v©NU+”9Bmg€„ÚÆ}¡S,ÐD•û[YÖ§)
µùø¬1
Œ©5ù\ Õ• «¼÷„&o%UQØ4µ.‹èט|"T}«û/DnZ
Ðå7>b .T‰U‘ª˜@£ÑP#„îþêòãu&Ux¡jøHR"§x5L—ß´3Y¥U¨û/$Ô" Ur•T…(¼P)PåTuÿ…ÈÁª#Tã®@0¡Êª’ª˜x»i’!pûf TV U2j¯¡(Ò2UuÿE± ‚tù»®U(RUŽVÓŽPóªäªHUˆœ¢ÕpBM²«„*©fJ£ÑÐøQð(5d—?þwÓ1ÕF&B☪6ÅHª¢jØÕãˆêÔ…ªHUÝ!*(T¯üÿ¡„*¯JªBdïÔ0]þgžÛ¹L5¤PeUuÿ…È:L
¡J¨BR-àm¯
áÛ6¨PMÓ‰ª¦M]¨F׺ÿBd ¬ªo¤ @¨BR"§ºó¡NJ¨Î5ªªTE5Ç2* UríF4¦šxŸë¦È¤}ƒ
µyb³Û±a„ª„*ŠT…¨P„jüâÅBÕ
IUˆÜ¼4B•P…¤*$Ô…j$T‘)SM@ U·¯1™Õ´LR Z¨Õ\§ª¹Eª¢t7jʪïÂÿ”…*$U!ºH¨d T‰UR¢bBuk- PåTIUˆlŠ„**‡&ªÚŠ¤D˜¶
(TcZgþ.T£ÅÿŠT…È!\
(TÿP9}¡
IUˆ¼œšƒPÉ@¨«ºÿB]ÿŒÛ7´PM‹ƒÿ²ŠPuðŸ¤*D®ájÚBMŽC
Õ1ÊÜþ=‡ Z†ï ѯiòÏV©
ñ|žÖÏùtcîºf%U!¡&‹5@—?2cÌé]‘áƒcBÒñ1Æÿx˜FÔ·‡®ÛÖhLUäîÔàBuuýCLJ™,&Á²ªÆ‰%UQ
¯†ªÉT¨íJJBU÷¿ÛnvMTUF¨Mª„\å…j@óTŠTEñ»ÿÙE¨„\7Uq†¤*J¯j› 4†*¡JªBBõÍÉš¶P»e U;ÅÚAcª®[^ãªÛ–L„šœ¤Z]þö…ªAUEª¢ÈjàÕ£û¯.?Úz+©Š
š5ˆPMÚBíæ.¿„ªî¿P/.¿„ªHUTM«…jüÆ Ôåïè E(Rmeã /ÓðBMHR.GB5º/$UQüî(¡º{êò·÷þ©ªû/J¶†ªG.Uuù%TIUH¨“*êòw,TuÿÕýïø^×7s`—êøÓ4ÿ,Õåo_¨º'©Š"k5¡&©êòK¨’ª¨hÈJ¨ œ¨*¡J¨êþ u2BU—_B•T3¼çuAeѶ!…Úp&T‘PÛjC÷„ºÿ¢ Áj¡º¿ÕåW„*©
5¡ UB•TE%µšS„ª.¿„*©ŠJG¬„j$T 5[4Q•€ªd#ÒB5-ª<1<—_=~n×õÒy¶Ú¯ár]À’ªè¶Õ$Ê$ÂP×PK«û/D û4„P $U!¡J¨BÝÿêwME˜ö
/Ôfcª¢s¢(R#Hª¢ˆL©ÁŒÞˆÍÃ&œP› æÎÇFZ™½Åkî Mïêå §µîv‚Á„?šþ®É³ìò%ƒ#éK³cMó—3-kC«ô‡Më`à€9SŸ««WR¤· ñö…}|íöažØf‚cX³~¤Åãš¿¦ÙML; µÉA…f£&þÛ{sCëaÝ3n59‚»ÅkúÔÁ˜ÝåÚ´ñß_U ÚSWok4¦*reF_Ä ú˜5µD¨-‡s$ÔŽ„ª1jEª“¢Ñhhü(¦÷Âé z¹úöí<±MB-¼Pµ|[Rí”Þ-ý½=Û:x¤é°ˆ óZí–5“SóÇídÛ謦¥göE,]0•«Wnç‰í
µ Bµÿ–U%ÕyÎŒ{÷Ýwæš6]’´å/i©OÒdƒ™d¹Ýo––¯ °þ†:«7 Ǧí¶ë©óûøÚÊí<5l$ÔB
UÝSM 5±^£ 2ûÌžz_Ë& ꯱táT¦÷!¡J¨’ª„*¡ºêU‹ 2û‡Ìè}à™õ£»þ M8}ÁT¦÷FjÑ„*¯Jªjq„:þ÷Z4Ê †ndf߃-›tN¥ û™ÞI¨ª¤*¡J¨®rV¬ßOëPÄÒEý N‰$Ô‚ÕȬ’ª„Z<¡NŒX¿ïŽXõ3Ø[“P%TIUB•P]åjÑ/º™}µlê=jœ±¨ŸÁx(@BÍQ¨òª¤:
#¡fP±^ÏŒÞ[N^Íé8ma?ýS" 5W¡R•T¡Y¨;G¬‡9"ÖçL«±tá S" 5'¡*P•T%Ôu¼þµh„̾.Q¬{
ÖXºhS" 5/¡*¢¤*¡–§þõÚh,Ö‡Åzú¢¦Ô$T UR•P%TWýX¿ËŒÞ‡ZŽ±î5ÍŠµ¿ 5c¡ ˆ)©J¨å«½6Êas®gzï£-?š½k,=b©S" Uª¤Z¡"¡æXÿ:ÃÌŸó]¦÷®Kë¢iô×# UB•TKjW 5Ãú×£ >´<1bÝgzSM£· UB•T%T ÕUÿžÚ º–Á)¶ cÝoFÓ
ÒW‹$T UR-`%ÔÌëßSÛÁü9ˇöŸQç´#¦ÑWGB•P%U UBuÕ¿§¶ƒùC×&Šõ€™=œºh¾žHB•P%U UBuÕ¿§¶ùCËœÒZ¬sgõpÚ¢Azë‘„*¡Jªª„êªOm˜s<ÄzÄ ½u$T UR-ŒN͘„ZÐú÷Ô¶³`Îr¦õ´ž¼š;³‡ÓŽ˜No-’PSjC‹ÿÑÁ^ª„ZÄú÷Ô¶³på¬\[Föjúñ 8«‡sÿx&;ÆLËNHǽ—N:;i?wÇOÓyr”žzmåçdI5½«SB-RýmÄz-+7,æé‘=š~‚ƒ½ê¥Ûƒ3ÃjuÿVB-Jý§Ô¶±pη˜6e½.Q¡HUB•PÓ¨ÿ¸XW¬?…§Gæèr’jÙðô>oÛ1Ðæð@«¿“çØUH)œ¨jd#Ôä„*…j”P%4J¨¢HU´ìþ‡Ë› 4†ª’ªÈ©û :ùv^B•Y…¤*rˆTÃ
ÕSlª¨ SuÞ÷L¨Z¨¦yÛíú¼¡„ª1U¡HUdnÖÐÉ·¡
IUtWç?ôi¹ U‚’ªÈɪ¡‡ñ¨„„**‡ÆT¨âz<“Ùy[I‰MÂ
Ucª¡¯!µ¯"UÑ$PÍæ Ãì#TÝðBRù„«Õªœ*$U‘Cœþˆm:H¨BR T Õxf©’PEÑD•¯|$Ô¶„êL¨D¨¦²Ÿ›P¤*Šo×€j;‹ÿÓª’ª¨žPMžB•\…¤*²vj`¡úçT‘PEõИª3¨3|?¡…j ‹ÿÃ
UcªaÑiªŠTEŸ…ª"T!©ŠîëþªÿÁ!„*Á
IUd?V¨^ÿªœ*rFcª T3¡J6BM^§V¨SÕ}¡HUdÜý,TŸî°UB’ªÈÞªa…êZS%¡
IµKÝcL…û9ªèPµÚ¨Z¡5SM橪®S
-Ô¦cª
“‰P5¦–F£ñ”ZA‘j§z¼šï+¼P=BÕ@ª„š(Š S+Hª²ºª] UL"R]VT;To©ªSƒ
µ“}ÿjY'Ÿ|òµ„¤Ú+W®¼ X_E«†ªq
NB-0jµ›Ô
Éh¢*Ë.»¬±|ùòήP¤‘™P›„ª&ª‚^?ßT+(RFãê
Þa…ê›TEjÙ¸ddä'jIuR,^¼øfàß«éÕ0BuçS•PK¥^¾dÉ’1µ„¤:iêõúyÀˆ„ê+T#¡V£££_R3¸Ñ˜ªo|ãïøÎw¾óÀ{+md"Ô– U2ªÆTS§EÑyŠR©¦ÊæÍ›/ˆ¢è7êÊe ¡¶£"Ô¢EÑÇN:餟¨%$ÕT9óÌ3·ŒŒ O6H¨¢kåªO<ñRµ„¤„%K–¹bÅŠ·-[¶lDw½¤œ7½éM¯_¿þ•Q} .iša„Š"Ôòp?pââÅ‹Ï¿ì²Ë”â_RÍŽeË–œ|òɇ_Ê13š«P‘P‹ËzcÌ…ccc‡.^¼ø{jŽÎÑ’ª†€¥Ë—/¿¤Ñhœœ ,*ô–„*,[ŸDQtÍèèèw–,Y²EM2y"5Aú|ó›ßÜ3Š¢…õz} 0³HuÛ>Úèùñ]Ûÿp¢ÂÐh"5³ËÍþ§¯xÞàm³ûëÛvþ¿½ÖìsÀÌ»ñ}ŽNh4"sëº~¦«0©Õëõ5}}}+N8á„aµˆB!„B!„B!„B!„B!„B!„B!„B!„B!„B!„B!„B!„B!„B!„B!:ãhà:`°
¸ø0CMÀ»€'±g‡ïü3ü0MM$„ç²&²ÿ¹x¾šˆß$´‘ŽS ! ÞáŸXgwy;r´Ñ‰º”ŠCMM rb ø;rÏ.Ts IUˆd^ìáYö45—T…Hæ°6Êî ©É„¤*Dz×^]M&ÊraxØAëðíÀJàE%}Ÿ
¬oñÞž.Ð¥9¿o£ìÆøó¢lÇ=;þs/0XÂ÷xã}= Ë sf›=¯»¿ïò¶ÒìÉ"Õ¾6Ê |¸„ïsªã÷ýº2g3vª‹Gª¹D™0mþl*Ù{üã=©k™ ŸNø\ ŽP)R-[¤Ú.}À§Ôt"%Þ¼.þâÛ «ËÀmj"QõHµŒ[ã©
Eª¢°‘ê8Ÿ¦¨ EEØ/A^›€7¨‰Dh©¾ øs5¡¨'Ð:yË,lž!‚JàR`O5£¨ Ó&ù{!R‘êLàãjF!„HGªÄÝ¢#Õ”Bá'Õ»€†ã9>§¦B?©þøª£ÌËSÕœBIÕaÏÈIâïÐ`¾BRõâ1Üû¯÷‹å+„’ªŸÖ8Ê| ø5«BRu3¼ßQFy„’jÜ€ÝGŸÄ›€×ªi…’ª ï‹£Ö$>ô¨y…’ª›UÀç eÎQó
!$U?><î(sþG !DWKu3p‘£Ì,”@!©zóeÜYÙß ¼PÍ,¸{¢m³<¦Çª‰D·HµœëñüŸU3‹Þ ´øÝ^ñ³]!U€Ÿÿæ(ó
à]ھ˰Û{wÀÆ€[€éªëÑÀ#-"Æ»±“!˜äï[ÉxEüÅï{<Ч Ïy< ;´ ø“Š]×CØmék ´>-áà ë4-îŸü%ðÀ€[±“ë÷°—Ž[À½ÀÍØcì¯ Þ,š¬]ŵŽÇÏ+˜ô txs¤E^gTÝæxÝ×èFù²£®zÝ-“¼þšqŸ½âgù$Û¨HgTÍïg×{^GØö½Øsò®ŒÅ9’òg¶ø°»©)³H•øà
G™ýéμ S'ù{Õµ3úUŸ ìÜßÏIl£óÇÔáÅÀãçþa<|t$鯋 Š…ú-àaì)'³²’*qWàG™ ÏÓˆ‹¥d ¸ÎC¨cÀŸÿ?å×? øyü¼Ë€w\†Í-½4+©n.ðˆt>©kSˆÒ_^äQö àú”…öcìÞ+rn‡!àkÀ5I=·ZŠ/ø/À/ eNBKdDx¶©>©r9p²G¹a'ˆÒäo€c
Ö Kbц–*Øñ†£Ì•(/€Ë€Û±y³¸ªÄmy¦G/àjÂLfÎ/h»¼ øv3—¥-·_c^IZ[x8ð ´~U„c °°ÍÇœGò²ª R¬ÕYp´gäù#ઃoàg€û€ßÇ?kãŸuØÍ%[â?Ÿ†ãîûtìäÛÁØeTÇ /mËÇa'°.)U°³üo&y ù#À7·”I19æÅ‘ØG¹ØYò‘ êø[ìøíÏ€_O´ñØñÏ:ìÒÇoÅÿ¿O ¾ ˜íñ<b—ýݪûöè•¿v”™… +B!lîä!G¹û€§2¬ÛïK°+‰ÄR»±M¡&ñH즃ã/uì꧶Cëv¹2½“xp„®_!
Å”X&ó å6¯Ç®á ɸ° |^ ´Ýøu7bwg}Ú£ì±Ø¥ƒJu›ÌÚ5V¢qU!ŠÅ±c©I cwqÝ™A}NEzl ‘fÍù±Ð], -Uð;zå(à-ºŽ…( ïp”1ØmÀ?˨Nã.^4° þ7:ÊB¼vµ¸B>G¯üoòÍ „°kÈ/÷ŒÜ®é²¶yø‚£Ì ðÊ,¤ºÊ£2 ¥kZˆÜxð±;§’¸’î=-ù‹À¨£Ì±YHìò)×Ò©ÏÕµ-Dæì|Ï£·ømÜGÔW™‡±k•“xaVR}÷N å"{¦a“¤ìë(÷ à4Ü»%«Î9~¿0+©| ÷Ñ+'¯Öu.D&Ô°p\ËWoÄ&wîvnvü~/ /+©ú ½v̦®ÏNˆà\Ë2‰G±[s7ª¹ XéQfßZ†ò9ze>p¶>;!‚ònÜã£[°‹íïSs=Ã&2sjWêƒ¸Ó }˜£ÏOˆ ‹{EÎ(vÝåoÔ\ý%·?k©Þ|ÂQf6Ê DÅ&q%Rz7î™îne³ã÷Sk9Têo eÞ…MÅ%„H‡=°\ç,]ŠMß)ZG«IÔóêV첉ÃNZ !&O/öD׃ 後 ~“ –Óëú ½ò*ì±BˆÉñeÜç;݈&‰S!ÏcMÎ~åûØÅÉÛôQ ѧ;Êü76±ühEÛ`lþÕý±kI÷žƒ
™ÌÄ&ÕŸŠ=N¼ è‹{̵øO丹yJõVàŸHΊ3›àRÝB´Íîü=ÀŸ`);ƒØãPŽÄ&°žM6=½["U°G¯œBòÑ+ ¾‚ÖË Ñ/Å §œ”$evqÿº’¾Ç{ ßâ÷±ˆl ªåüúëp ½Ò{–bbï»$œMÝvúF`M ßßð¸î¿ˆƒ³#)ÈnÌZêàsôÊ)¸³‘ !lW÷zìØa7ãNR4fcO y0´.b%‹ ÕüRŠ}å"‰:ð¯ØñDGcN—öì5À_Ľ×BW¶\{Çà,Ý7B´äÓØýú¾\üq "ïïa3Ýí‘Áëa‡F¶`W bñ¦§@÷>l˜¤:}»ÆUYs„˜È9q×îý
6¹òã|OÏÃ.©<<…ç2Ø•«€ßaOc}0þÙˆÍûü÷–Eª¿>œ—Pf;±uŽî!!žáxìÖîNØøgììy‘’Pïü;éÖ Û€Ÿ?ÅŽ ÿx²›ºÿãø ½² ¿1#QîkAøs“›ox-pqÞÏTàÚ„:‚ÝŽ{rîCCp¦g¹¿Á&cÊš½ñ˜,²T}Ž^)z^ עᾂÔóhö¿*|ø? ¿ÿà2 Ô×°ëD³b
ðVr¿Å/$ûû†³EÆçè•7cZ‘Çïg¤ž/“‹*ÁUø¥Éüî Œ³€ofØ›Zèy?\ìq_…Âk)gÑ¥º5îâ»(j^€íŽßerèù¨ôÜ€ÿ¬y8xÀQîÅدYð2÷Ó|X#+^T©|÷Ñ+ ±'@
טpÆ1{WËI¥æWØ„Ôcm¼w¢øùÀ߬ƒOo)ÏìtÞ kÊ$ÕuØõiIÔ±;2ŠÂ eŽË±~ D‰¿Çéæ{ŸÄž®ájƒ3€wªƒO†¬¼|u vÙaå¤
~G¯ ÓN¨ ˜ßz”É+R|>ÅœÜË ×i¢ÿ+?÷(÷yì{i³Þ£ÌÞ9µM[Êe“êüŽ^ùÅHr/ð¨£Ì ‘NÞvˆ€/ ÌT»FkIÌè‚6ø
ðUG™©ØñÕ´ÛÃ'Iö9´Éø-÷*TÁïè•çRœ¼ ?ö(óþŒëôÀ± ˆ¸Ê¸Æëöë’/—sâ¨5‰yØiâ3TöÚŒÛb?:X§[Ö‰ïÃ=±pANßl»r£ç·aVÑê++Zü®ÑÅRuõ(ö(ÈõšmØñUWä~
~Ùä|ù¹G™×gØ»êÇ&Ênw™f¬Rý]Ü}Mb A Yr-î5„õ¸Ûú‚Y€MàÛlëáMŽ:V™G<¾¨î’hu-~RW /Mé5‰{YÛlìN°ÐôÅC /î ø¨]ôœôsmA?øYØqWý~à(³>p]¿æQO|1` ^wq[½ö‹âvhõûOª×–]¿wÔå?rºÖW9êj²ó3 ×ì}سãÒàz×{„°Û»gÇŸs«×¿ÁqŸœXf©‚Ýëìªÿo€ÿ—³T¿Ý|Äú© Ã2gcó´zÍñÅßIËgº@ªßóø|Ží"©N‰#HW›ÜÒðÐñž÷ȧ(òy pwÂëŽÅ½½¤ûdIÙ¥Znóx;r–*Ø£€çÏ÷IçX‹yØ1ݤ×z’gS¼%],_è©^âñÙÜ‹g
¸&Ì(™TÁŽ#¯÷h—¥ðZ‘Ç{ ÿ¹ 8!¥÷8›”iÌñš_ö¸O––]ª`Óþ™Iþd!Õ}ÍmÔéIlŠ¶NÖæ-ÄŽÑ{¼Î;=.éb¹ª ¤ê{-Ý‹ºÄiØ,õ+âÇž]2©Â³ÇW'Õa”tÆœm£Wg⡽c:ìÝ ÷ ·x¼ÎJž]«œtŸœU©‚ T.ºT‰ÖnÝF>ŒÝÙqp ñô`Ô÷Œ»$o‰Ç=WµñÜ×ìR¿¤‹åŸ»@ª=žQÙøÏõØ C㱸 ì*?Œåù ì$åÎù§JlòkŸñÎ4较ûä!ì‡Ë°k¿çbç]êØ™ü=±óoÅn"º½çÞ äyŸ|¨*R=»¤èR¿Áÿ,~~Ì÷w²TÁn ÙîוTªµøËÝõþnbò+E€[
rŸ¬c÷<ª‰ºU9Bã>Š±|ʇ÷ï46“¿ þ”öö»wËÖKØÄ sJÚ.
ìùV;Ê Gµ“ak<äp[Îïù.àåØ,yÞŸob7Ã%¹8ßM“GÏ«ã öé6 ×-R] ‹…úlÊœ»ö±x˜Éµž÷B&?‰´ »QeyNïõ+ñ0Î]m>n¨*ÝÿqÞÖaˆÿxNõ}-6Gdݘ'âöIâÁ„Çÿ¨KºÿãœG{&í »”±û¿3 ôxŸHçd‹{úÀ¦Œî“U¸zÈuŸTIªã]ÛN¹ób*ðWñØMˆ‹dSÜ óY×÷avŸXŽ·t™TÁ ÓühŠŸÅÀ + UâÏÅõ~oÆ®uMƒç`÷á?è>¹»õÖ§÷~qÂ}ò6 ©þKɤz$öÜvtEêÝ‹Ý‚w]‹¬Ÿíص®Ë€™%èR&½—oä\¿Ø# êð³Ø‚=•ô$:?’ü7Ž×8>‡v™…Ý.îzÿø<Ί#‘IÞ'waW̤~šðU´^ÏøXN Ød9+î~øl
x€tC¤Á@Ü9? ýi|lŒ…;߬ k°»^¾— 0½DŸÕE _"ë€×¤žõxŒï’8J»= Ý OÆâ½ø:6¡ÏËH'ŸÃ»ZDhcq48˜S›ì…ÝZ}_‚à>ø ïÕq[_
ü$¾OÖcç Æâ/€ a×¥f™ÀH!„Bˆ‚ð?WÛ²6i@ %tEXtdate:create 2013-08-13T07:50:42+02:00Q6íù %tEXtdate:modify 2013-08-13T07:50:42+02:00 kUE IEND®B`‚ Nuitka-0.5.0.1/bin/ 0000755 0001750 0001750 00000000000 12265271051 014111 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/bin/nuitka-run 0000777 0001750 0001750 00000000000 12265044767 017360 2nuitka ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/bin/nuitka 0000755 0001750 0001750 00000010234 12265264105 015334 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
This is the main program of Nuitka, it checks the options and then translates
one or more modules to a C++ source code using Python C/API in a build directory
compiles it to either an executable or an extension module that can contain
other modules.
"""
# Import as little as possible initially, because we might be re-executing
# soon.
import logging
import os
import sys
import warnings
# LIBDIR trick start (marker for removal on platforms that don't need it)
libdir = '@LIBDIR@'
# Two cases:
if libdir != '@' 'LIBDIR' '@':
# Changed by our distutils hook, then use the given path.
if not os.path.isabs(libdir):
libdir = os.path.join(
os.path.dirname(os.path.realpath(__file__)),
libdir
)
libdir = os.path.abspath(libdir)
sys.path.insert(
0,
libdir
)
else:
# Unchanged, running from checkout, use the parent directory, the nuitka
# package ought be there.
sys.path.insert(
0,
os.path.normpath(
os.path.join(
os.path.dirname(__file__),
".."
)
)
)
# LIBDIR trick end (marker for removal on platforms that don't need it)
from nuitka import Options, Utils # isort:skip
logging.basicConfig(format = 'Nuitka:%(levelname)s:%(message)s')
# We don't care, and these are triggered by run time calculations of "range" and
# others, while on python2.7 they are disabled by default.
warnings.simplefilter("ignore", DeprecationWarning)
# We will run with the Python configuration as specified by the user, if it does
# not match, we restart ourselves with matching configuration.
needs_reexec = False
current_version = "%d.%d" % (sys.version_info[0], sys.version_info[1])
# We support to execute with a specified version.
intended_version = Options.getIntendedPythonVersion()
if intended_version is None:
intended_version = current_version
# If it's a different version, we find it by guessing it, otherwise we use the
# one previously used.
if current_version != intended_version:
assert intended_version in ("2.6", "2.7", "3.2", "3.3", "3.4")
if Utils.getOS() == "Windows":
python_binary = r"C:\Python%s\python.exe" % \
intended_version.replace( ".", "" )
else:
python_binary = "/usr/bin/python" + intended_version
needs_reexec = True
else:
python_binary = sys.executable
python_flags = Options.getPythonFlags()
if sys.flags.no_site == 0 and "no_site" in python_flags:
needs_reexec = True
# The hash randomization totally destroys the created source code created,
# changing it every single time Nuitka is run. This kills any attempt at
# caching it, and comparing generated source code. While the created binary
# actually should use it, we don't want to. So lets disable it.
if os.environ.get( "PYTHONHASHSEED", "-1" ) != "0":
os.environ[ "PYTHONHASHSEED" ] = "0"
needs_reexec = True
# In case we need to re-execute.
if needs_reexec:
# Execute with full path as the process name, so it can find itself and its
# libraries.
args = [
python_binary,
python_binary,
]
# Potentially give Python command line flags as necessary.
if "no_site" in python_flags:
args.append("-S")
# Same arguments as before.
args += sys.argv
Utils.callExec(args)
# Now the main program.
from nuitka import MainControl # isort:skip
MainControl.main()
Nuitka-0.5.0.1/bin/compare_with_cpython 0000755 0001750 0001750 00000032200 12265264105 020263 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import print_function
import difflib
import os
import re
import subprocess
import sys
import tempfile
filename = sys.argv[1]
args = sys.argv[2:]
def hasArg(arg):
if arg in args:
args.remove(arg)
return True
else:
return False
# For output keep it
arguments = list(args)
silent_mode = hasArg("silent")
ignore_stderr = hasArg("ignore_stderr")
ignore_warnings = hasArg("ignore_warnings")
expect_success = hasArg("expect_success")
expect_failure = hasArg("expect_failure")
python_debug = hasArg("python_debug")
module_mode = hasArg("module_mode")
two_step_execution = hasArg("two_step_execution")
binary_python_path = hasArg("binary_python_path")
trace_command = hasArg("trace_command")
remove_output = hasArg("remove_output")
standalone_mode = hasArg("standalone")
no_site = hasArg("no_site")
assert not standalone_mode or not module_mode
if args:
sys.exit("Error, non understood mode(s) '%s'," % ",".join(args))
if "PYTHONHASHSEED" not in os.environ:
os.environ["PYTHONHASHSEED"] = "0"
if "PYTHON" not in os.environ:
os.environ["PYTHON"] = sys.executable
if python_debug and os.path.exists(os.path.join("/usr/bin/",
os.environ["PYTHON"] + "-dbg")):
os.environ["PYTHON"] += "-dbg"
print(
"""\
Comparing output of '{filename}' using '{python}' with flags {args} ...""".
format(
filename = filename,
python = os.environ["PYTHON"],
args = ", ".join(arguments)
)
)
if not silent_mode:
print("*" * 80)
print("CPython:")
print("*" * 80)
if two_step_execution:
filename = os.path.abspath(filename)
if module_mode:
cpython_cmd = [
os.environ[ "PYTHON" ],
"-W", "ignore",
"-c", "import sys; sys.path.append(%s); import %s" % (
repr(os.path.dirname(filename)),
os.path.basename(filename)
)
]
else:
cpython_cmd = [
os.environ[ "PYTHON" ],
"-W", "ignore",
filename
]
if no_site:
cpython_cmd.insert(1,"-S")
if "NUITKA" in os.environ:
nuitka_call = [os.environ["NUITKA"]]
else:
nuitka_call = [
os.environ["PYTHON"],
os.path.abspath(os.path.join(os.path.dirname(__file__), "nuitka"))
]
extra_options = os.environ.get("NUITKA_EXTRA_OPTIONS", "")
if python_debug:
extra_options += " --python-debug"
if remove_output:
extra_options += " --remove-output"
if binary_python_path:
extra_options += " --keep-pythonpath"
if os.name == "nt":
python_path_sep = ";"
else:
python_path_sep = ":"
python_path = os.environ.get("PYTHONPATH","")
os.environ["PYTHONPATH"] = python_path_sep.join(
python_path.split(python_path_sep)+\
[os.path.dirname(os.path.abspath(filename))]
)
if not two_step_execution:
if module_mode:
nuitka_cmd = nuitka_call + extra_options.split() + \
["--module", "--execute", filename]
elif standalone_mode:
nuitka_cmd = nuitka_call + extra_options.split() + \
["--standalone", "--execute", filename]
else:
nuitka_cmd = nuitka_call + extra_options.split() + \
["--execute", filename]
if no_site:
nuitka_cmd.insert(len(nuitka_cmd)-1,"--python-flag=-S")
else:
if module_mode:
nuitka_cmd1 = nuitka_call + extra_options.split() + \
["--module", os.path.abspath(filename)]
elif standalone_mode:
nuitka_cmd1 = nuitka_call + extra_options.split() + \
["--standalone", filename]
else:
nuitka_cmd1 = nuitka_call + extra_options.split() + \
[filename]
if no_site:
nuitka_cmd1.insert(len(nuitka_cmd1)-1,"--python-flag=-S")
dir_match = re.search(r"--output-dir=(.*?)(\s|$)", extra_options)
if dir_match:
output_dir = dir_match.group(1)
else:
output_dir = "."
if module_mode:
nuitka_cmd2 = [
os.environ["PYTHON"],
"-W", "ignore",
"-c", "import %s" % os.path.basename(filename)
]
else:
exe_filename = os.path.basename(filename)
if filename.endswith(".py"):
exe_filename = exe_filename[:-3]
exe_filename += ".exe"
nuitka_cmd2 = [
os.path.join(output_dir, exe_filename)
]
process = subprocess.Popen(
args = cpython_cmd,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
stdout_cpython, stderr_cpython = process.communicate()
exit_cpython = process.returncode
def displayCPython():
print(stdout_cpython, end=' ')
if stderr_cpython:
print(stderr_cpython)
if not silent_mode:
displayCPython()
if not silent_mode:
print("*" * 80)
print("Nuitka:")
print("*" * 80)
if two_step_execution:
if output_dir:
os.chdir(output_dir)
else:
tmp_dir = tempfile.gettempdir()
# Try to avoid RAM disk /tmp and use the disk one instead.
if tmp_dir == "/tmp" and os.path.exists("/var/tmp"):
tmp_dir = "/var/tmp"
os.chdir(tmp_dir)
if trace_command:
print("Going to output directory", os.getcwd())
if not two_step_execution:
if trace_command:
print("Nuitka command:", nuitka_cmd)
process = subprocess.Popen(
args = nuitka_cmd,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
stdout_nuitka, stderr_nuitka = process.communicate()
exit_nuitka = process.returncode
else:
if trace_command:
print("Nuitka command 1:", nuitka_cmd1)
process = subprocess.Popen(
args = nuitka_cmd1,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
stdout_nuitka1, stderr_nuitka1 = process.communicate()
exit_nuitka1 = process.returncode
if exit_nuitka1 != 0:
exit_nuitka = exit_nuitka1
stdout_nuitka, stderr_nuitka = stdout_nuitka1, stderr_nuitka1
else:
if trace_command:
print("Nuitka command 2:", nuitka_cmd2)
process = subprocess.Popen(
args = nuitka_cmd2,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
stdout_nuitka2, stderr_nuitka2 = process.communicate()
stdout_nuitka = stdout_nuitka1 + stdout_nuitka2
stderr_nuitka = stderr_nuitka1 + stderr_nuitka2
exit_nuitka = process.returncode
if not silent_mode:
print(stdout_nuitka, end=' ')
if stderr_nuitka:
print(stderr_nuitka)
ran_re = re.compile(r"^(Ran \d+ tests? in )\d+\.\d+s$")
instance_re = re.compile(r"at (?:0x)?[0-9a-fA-F]+")
compiled_function_re = re.compile(r"\")
compiled_generator_re = re.compile(r"\)")
global_name_error_re = re.compile(
r"global (name ')(.*?)(' is not defined)"
)
non_ascii_error_rt = re.compile(
r"(SyntaxError: Non-ASCII character.*? on line) \d+"
)
python_win_lib_re = re.compile(
r"[a-zA-Z]:\\\\?[Pp]ython(.*\\\\?)[Ll]ib"
)
def makeDiffable(output):
result = []
# fix import readline cause output sometimes startswith \x1b[?1034h
m = re.match(b'\\x1b\\[[^h]+h', output)
if m:
output = output[len(m.group()):]
for line in output.split(b"\n"):
if type(line) is not str:
line = line.decode("utf-8" if os.name != "nt" else "cp850")
if line.endswith("\r"):
line = line[:-1]
if line.startswith("[") and line.endswith("refs]"):
continue
if ignore_warnings and line.startswith("Nuitka:WARNING"):
continue
if line.startswith("Nuitka:WARNING:Cannot recurse to import"):
continue
line = instance_re.sub(r"at 0xxxxxxxxx", line)
line = compiled_function_re.sub(r"", line)
line = compiled_generator_re.sub(r" ignored":
continue
# This is also a bug potentially, but only visible under
# CPython
line = python_win_lib_re.sub(r"C:\Python\1Lib", line)
# This is a bug with clang potentially, can't find out why it says that.
if line == "/usr/bin/ld: warning: .init_array section has zero size":
continue
# This is for NetBSD, which seems to build libpython so it gives such
# warnings.
if "() possibly used unsafely, use mkstemp() or mkdtemp()" in line:
continue
result.append(line)
return result
def compareOutput(kind, out_cpython, out_nuitka):
fromdate = None
todate = None
diff = difflib.unified_diff(
makeDiffable(out_cpython),
makeDiffable(out_nuitka),
"{program} ({detail})".format(
program = os.environ[ "PYTHON" ],
detail = kind
),
"{program} ({detail})".format(
program = "nuitka",
detail = kind
),
fromdate,
todate,
n=3
)
result = list(diff)
if result:
for line in result:
print(line, end = "\n" if not line.startswith("---") else "")
return 1
else:
return 0
exit_code_stdout = compareOutput("stdout", stdout_cpython, stdout_nuitka)
if ignore_stderr:
exit_code_stderr = 0
else:
exit_code_stderr = compareOutput("stderr", stderr_cpython, stderr_nuitka)
exit_code_return = exit_cpython != exit_nuitka
if exit_code_return:
print(
"""\
Exit codes {exit_cpython:d} (CPython) != {exit_nuitka:d} (Nuitka)""".format(
exit_cpython = exit_cpython,
exit_nuitka = exit_nuitka
)
)
exit_code = exit_code_stdout or exit_code_stderr or exit_code_return
if exit_code:
sys.exit("Error, outputs differed.")
if expect_success and exit_cpython != 0:
if silent_mode:
displayCPython()
sys.exit("Unexpected error exit from CPython.")
if expect_failure and exit_cpython == 0:
sys.exit("Unexpected success exit from CPython.")
if remove_output:
if not module_mode:
if os.path.exists(nuitka_cmd2[0]):
if os.name == "nt":
# It appears there is a tiny lock race that we randonly cause,
# likely because --execute spawns a subprocess that might still
# be doing the cleanup work.
os.rename(nuitka_cmd2[0], nuitka_cmd2[0]+".away")
os.unlink(nuitka_cmd2[0]+".away")
else:
os.unlink(nuitka_cmd2[0])
else:
if os.name == "nt":
module_filename = os.path.basename(filename) + ".pyd"
else:
module_filename = os.path.basename(filename) + ".so"
if os.path.exists(module_filename):
os.unlink(module_filename)
if not silent_mode:
print("OK, same outputs.")
Nuitka-0.5.0.1/bin/compare_with_xml 0000755 0001750 0001750 00000006302 12265264105 017403 0 ustar hayen hayen 0000000 0000000 #!/usr/bin/env python
# Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import print_function
import difflib
import os
import re
import subprocess
import sys
import tempfile
nuitka1 = sys.argv[1]
nuitka2 = sys.argv[2]
filename = sys.argv[3]
print(
"""\
Comparing output of '{filename}' using '{nuitka1}' <-> '{nuitka2}' ...""".
format(
filename = filename,
nuitka1 = nuitka1,
nuitka2 = nuitka2
)
)
extra_options = os.environ.get("NUITKA_EXTRA_OPTIONS", "")
nuitka1_cmd = "{nuitka1} --dump-xml {filename}".format(
nuitka1 = nuitka1,
filename = filename
)
process = subprocess.Popen(
args = nuitka1_cmd,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE,
shell = True
)
stdout_nuitka1, stderr_nuitka1 = process.communicate()
exit_nuitka1 = process.returncode
nuitka2_cmd = "{nuitka2} --dump-xml {filename}".format(
nuitka2 = nuitka2,
filename = filename
)
process = subprocess.Popen(
args = nuitka2_cmd,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE,
shell = True
)
stdout_nuitka2, stderr_nuitka2 = process.communicate()
exit_nuitka2 = process.returncode
def makeDiffable(output):
result = []
for line in output.split(b"\n"):
line = str(line)
result.append(line)
return result
fromdate = None
todate = None
def compareOutput(kind, out1, out2):
diff = difflib.unified_diff(
makeDiffable(out1),
makeDiffable(out2),
"{program} ({detail})".format(
program = "nuitka1 " + filename,
detail = kind
),
"{program} ({detail})".format(
program = "nuitka2 " + filename,
detail = kind
),
fromdate,
todate,
n=3
)
result = list(diff)
if result:
for line in result:
print(line, end = "\n" if not line.startswith( "---" ) else "")
return 1
else:
return 0
exit_code_stdout = compareOutput("stdout", stdout_nuitka1, stdout_nuitka2)
exit_code_return = exit_nuitka1 != exit_nuitka2
if exit_code_return:
print(
"""\
Exit codes {exit_nuitka1:d} ({nuitka1}) != {exit_nuitka2:d} ({nuitka2})""".
format(
exit_nuitka1 = exit_nuitka1,
nuitka1 = nuitka1,
exit_nuitka2 = exit_nuitka2,
nuitka2 = nuitka2
)
)
exit_code = exit_code_stdout or exit_code_return
if exit_code:
sys.exit("Error, outputs differed.")
print("OK, same outputs.")
Nuitka-0.5.0.1/setup.py 0000644 0001750 0001750 00000013307 12265264105 015061 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys, os
scripts = [ "bin/nuitka", "bin/nuitka-run" ]
if os.name == "nt":
scripts += [ "misc/nuitka.bat", "misc/nuitka-run.bat" ]
def detectVersion():
version_line, = [
line
for line in
open( "nuitka/Options.py" )
if line.startswith( "Nuitka V" )
]
return version_line.split( "V" )[1].strip()
version = detectVersion()
if os.name == "nt" and "bdist_msi" in sys.argv:
# The MSI enforces a 3 digit version number, which is stupid, but no way
# around it, so we map our number to it, in some way.
# Prereleases are always smaller.
middle = 1 if "pre" not in version else 0
version = version.replace( "pre", "" )
parts = version.split(".")
major, first, last = parts[:3]
hotfix = parts[4] if len(parts) > 3 else 0
version = ".".join(
"%s" % value
for value in
(
int(major)*10+int(first),
middle,
int(last)*10+int(hotfix)
)
)
def find_packages():
result = []
for root, _dirnames, filenames in os.walk( "nuitka" ):
# Ignore the inline copy of scons, these are not packages of Nuitka.
if "scons-" in root:
continue
# Packages must contain "__init__.py" or they are merely directories.
if "__init__.py" not in filenames:
continue
result.append( root.replace( os.path.sep, "." ) )
return result
package = find_packages()
from distutils.core import setup, Command, Extension
from distutils.command.install_scripts import install_scripts
class nuitka_installscripts( install_scripts ):
"""
This is a specialization of install_scripts that replaces the @LIBDIR@ with
the configured directory for modules. If possible, the path is made relative
to the directory for scripts.
"""
def initialize_options( self ):
install_scripts.initialize_options( self )
self.install_lib = None
def finalize_options( self ):
install_scripts.finalize_options(self)
self.set_undefined_options( "install", ( "install_lib", "install_lib" ) )
def run( self ):
install_scripts.run( self )
if ( os.path.splitdrive( self.install_dir )[0] != os.path.splitdrive( self.install_lib )[0] ):
# can't make relative paths from one drive to another, so use an
# absolute path instead
libdir = self.install_lib
else:
common = os.path.commonprefix( (self. install_dir, self.install_lib ) )
rest = self.install_dir[ len(common) : ]
uplevel = len( [n for n in os.path.split( rest ) if n ] )
libdir = uplevel * ( ".." + os.sep ) + self.install_lib[ len(common) : ]
for outfile in self.outfiles:
fp = open( outfile, "rb" )
data = fp.read()
fp.close()
# skip binary files
if b'\0' in data:
continue
data = data.replace( b"@LIBDIR@", libdir.encode( "unicode_escape" ) )
fp = open( outfile, "wb" )
fp.write( data )
fp.close()
cmdclass = {
"install_scripts": nuitka_installscripts
}
def findSources():
result = []
for root, _dirnames, filenames in os.walk( "src" ):
for filename in filenames:
if filename.endswith( ".cpp" ) or filename.endswith( ".h" ) or filename.endswith( ".asm" ) or filename.endswith( ".S" ):
result.append( os.path.join( root, filename ) )
return result
if os.path.exists( "/usr/bin/scons" ) and \
"sdist" not in sys.argv and \
"bdist_wininst" not in sys.argv and \
"bdist_msi" not in sys.argv:
scons_files = []
else:
scons_files = [
"inline_copy/*/*.py",
"inline_copy/*/*/*.py",
"inline_copy/*/*/*/*.py",
"inline_copy/*/*/*/*/*.py",
"inline_copy/*/*/*/*/*/*.py",
]
if "bdist_msi" in sys.argv:
project_name = "Nuitka%s" % (64 if "AMD64" in sys.version else 32)
else:
project_name = "Nuitka"
setup(
name = project_name,
license = "Apache License, Version 2.0",
version = version,
packages = find_packages(),
scripts = scripts,
cmdclass = cmdclass,
package_data = {
# Include extra files
"" : ['*.txt', '*.rst', '*.cpp', '*.hpp', '*.ui' ],
"nuitka.build" : [
"SingleExe.scons",
"static_src/*.cpp",
"static_src/*/*.cpp",
"static_src/*/*.h",
"static_src/*/*.asm",
"static_src/*/*.S",
"include/*.hpp",
"include/*/*.hpp",
"include/*/*/*.hpp",
] + scons_files,
"nuitka.gui" : [
"dialogs/*.ui",
],
},
# metadata for upload to PyPI
author = "Kay Hayen",
author_email = "Kay.Hayen@gmail.com",
url = "http://nuitka.net",
description = "Python compiler with full language support and CPython compatibility",
keywords = "compiler,python,nuitka",
)
Nuitka-0.5.0.1/README.txt 0000644 0001750 0001750 00000056240 12265044767 015062 0 ustar hayen hayen 0000000 0000000 Nuitka User Manual
~~~~~~~~~~~~~~~~~~
.. image:: images/Nuitka-Logo-Symbol.png
.. contents::
.. raw:: pdf
PageBreak oneColumn
SetPageCounter 1
Overview
========
Nuitka is the Python compiler. It is a good replacement for the Python
interpreter and compiles **every** construct that CPython 2.6, 2.7, 3.2 and 3.3
offer. It translates the Python into a C++ program that then uses "libpython" to
execute in the same way as CPython does, in a very compatible way.
This document is the recommended first read if you are interested in using
Nuitka, understand its use cases, check what you can expect, license,
requirements, credits, etc.
Usage
=====
Requirements
------------
- C++ Compiler: You need a compiler with support for C++03
Currently this means, you need to use either of these compilers:
* GNU g++ compiler of at least version 4.4
* The clang compiler on MacOS X or FreeBSD, based on LLVM version 3.2
* The MinGW compiler on Windows
* Visual Studion 2008 or higher on Windows
- Python: Version 2.6, 2.7 or 3.2, 3.3 (support for upcoming 3.4 exists
partially)
You need the standard Python implementation, called CPython, to execute
Nuitka, because it is closely tied to using it.
.. note::
The created binaries can be made executable independent of the Python
installation, with ``--standalone`` option.
- Operating System: Linux, FreeBSD, NetBSD, MacOS X, and Windows (32/64 bits),
Others may work as well. The portability is expected to be generally good, but
the Scons usage may have to be adapted.
- Architectures: x86, x86_64 (amd64), and arm.
Other architectures may also work, these are just the only ones
tested. Feedback is welcome.
Command Line
------------
No environment variable changes are needed, you can call the ``nuitka`` and
``nuitka-run`` scripts directly without any changes to the environment. You may
want to add the ``bin`` directory to your ``PATH`` for your convenience, but
that step is optional.
Nuitka has a ``--help`` option to output what it can do:
.. code-block:: bash
nuitka --help
The ``nuitka-run`` command is the same as ``nuitka``, but with different
default. It tries to compile and directly execute a Python script:
.. code-block:: bash
nuitka-run --help
These option that is different is ``--execute``, and passing on arguments after
the first non-option to the created binary, so it is somewhat more similar to
what plain ``python`` will do.
License
-------
Nuitka is licensed under the Apache License, Version 2.0; you may not use
it except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
Use Cases
=========
Use Case 1 - Program compilation with all modules embedded
----------------------------------------------------------
If you want to compile a whole program recursively, and not only the single file
that is the main program, do it like this:
.. code-block:: bash
nuitka --recurse-all program.py
.. note::
The is more fine grained control than ``--recurse-all`` available. Consider
the output of ``nuitka --help``.
In case you have a plugin directory, i.e. one which is not found by recursing
after normal import statements (recommended way), you can always require that a
given directory shall also be included in the executable:
.. code-block:: bash
nuitka --recurse-all --recurse-directory=plugin_dir program.py
.. note::
If you don't do any dynamic imports, simply setting your ``PYTHONPATH`` at
compilation time will be sufficient for all your needs normally.
Use ``--recurse-directory`` only if you make ``__import__()`` calls that
Nuitka cannot predict, because they e.g. depend on command line
parameters. Nuitka also warns about these, and point to the option.
.. note::
The resulting binary still depends on CPython and used C extension modules
being installed.
If you want to be able to copy it to another machine, use ``--standalone``
and copy the created ``program.dist`` directory and execute the
``program.exe`` put inside.
Use Case 2 - Extension Module compilation
-----------------------------------------
If you want to compile a single extension module, all you have to do is this:
.. code-block:: bash
nuitka --module some_module.py
The resulting file "some_module.so" can then be used instead of
"some_module.py". It's left as an exercise to the reader, what happens if both
are present.
.. note::
The option ``--recurse-all`` and other variants work as well.
Use Case 3 - Package compilation
--------------------------------
If you need to compile a whole package and embedded all modules, that is also
feasible, use Nuitka like this:
.. code-block:: bash
nuitka --module some_package --recurse-directory=some_package
.. note::
The recursion into the package directory needs to be provided manually,
otherwise the package is empty. Data files located inside the package will
not be embedded yet.
Where to go next
================
Remember, this project is not completed yet. Although the CPython test suite
works near perfect, there is still more work needed, to make it do more
optimization. Try it out.
Subscribe to its mailing lists
------------------------------
Please visit the `mailing list page
`__ in order to subscribe the
relatively low volume mailing list. All Nuitka issues can be discussed there.
Report issues or bugs
---------------------
Should you encounter any issues, bugs, or ideas, please visit the `Nuitka bug
tracker `__ and report them.
Contact me via email with your questions
----------------------------------------
You are welcome to `contact me via email `__ with
your questions.
Word of Warning
---------------
Consider using this software with caution. Your feedback and patches to Nuitka
are very welcome.
Especially report it please, if you find that anything doesn't work, because the
project is now at the stage that this should not happen.
Join Nuitka
===========
You are more than welcome to join Nuitka development and help to complete the
project in all minor and major ways.
The development of Nuitka occurs in git. We currently have these 2 branches:
- `master
`__:
This branch contains the stable release to which only hotfixes for bugs will
be done. It is supposed to work at all times and is supported.
- `develop
`__:
This branch contains the ongoing development. It may at times contain little
regressions, but also new features. On this branch the integration work is
done, whereas new features might be developed on feature branches.
- `factory
`__:
This branch contains potentially unfinished and incomplete work. It is very
frequently subject ``git rebase`` and the public staging ground, where my work
for develop branch lives first. It is intended for testing only and
recommended to base any of your own development on.
.. note::
I accept patch files, git formatted patch queues (use ``git format-patch
origin`` command), or if you prefer git pull on the social code platforms.
I will do the integration work. If you base your work on "master" or
"develop" at any given time, I will do any re-basing required and keep your
authorship intact.
.. note::
The `Developer Manual `__
explains the coding rules, branching model used, with feature branches and
hotfix releases, the Nuitka design and much more. Consider reading it to
become a contributor. This document is intended for Nuitka users.
Donations
=========
Should you feel that you cannot help Nuitka directly, but still want to support,
please consider `making a donation `__
and help this way.
Unsupported functionality
=========================
The ``co_code`` attribute of code objects
-----------------------------------------
The code objects are empty for for native compiled functions. There is no
bytecode with Nuitka's compiled function objects, so there is no way to provide
it.
Optimization
============
Constant Folding
----------------
The most important form of optimization is the constant folding. This is when an
operation can be predicted. Currently Nuitka does these for some built-ins (but
not all yet), and it does it for binary/unary operations and comparisons.
Constants currently recognized:
.. code-block:: python
5 + 6 # operations
5 < 6 # comparisons
range(3) # built-ins
Literals are the one obvious source of constants, but also most likely other
optimization steps like constant propagation or function inlining will be. So
this one should not be underestimated and a very important step of successful
optimizations. Every option to produce a constant may impact the generated code
quality a lot.
Status: The folding of constants is considered implemented, but it might be
incomplete. Please report it as a bug when you find an operation in Nuitka that
has only constants are input and is not folded.
Constant Propagation
--------------------
At the core of optimizations there is an attempt to determine values of
variables at run time and predictions of assignments. It determines if their
inputs are constants or of similar values. An expression, e.g. a module variable
access, an expensive operation, may be constant across the module of the
function scope and then there needs to be none, or no repeated module variable
look-up.
Consider e.g. the module attribute ``__name__`` which likely is only ever read,
so its value could be predicted to a constant string known at compile time. This
can then be used as input to the constant folding.
.. code-block:: python
if __name__ == "__main__":
# Your test code might be here
use_something_not_use_by_program()
From modules attributes, only ``__name__`` is currently actually optimized. Also
possible would be at least ``__doc__``.
Also built-in exception name references are optimized if they are uses as module
level read only variables:
.. code-block:: python
try:
something()
except ValueError: # The ValueError is a slow global name lookup normally.
pass
Builtin Call Prediction
-----------------------
For builtin calls like ``type``, ``len``, or ``range`` it is often possible to
predict the result at compile time, esp. for constant inputs the resulting value
often can be precomputed by Nuitka. It can simply determine the result or the
raised exception and replace the builtin call with it allowing for more constant
folding or code path folding.
.. code-block:: python
type( "string" ) # predictable result, builtin type str.
len( [ 1, 2 ] ) # predictable result
range( 3, 9, 2 ) # predictable result
range( 3, 9, 0 ) # predictable exception, range hates that 0.
The builtin call prediction is considered implemented. We can simply during
compile time emulate the call and use its result or raised exception. But we may
not cover all the built-ins there are yet.
Sometimes the result of a built-in should not be predicted when the result is
big. A ``range()`` call e.g. may give too big values to include the result in
the binary. Then it is not done.
.. code-block:: python
range( 100000 ) # We do not want this one to be expanded
Status: This is considered mostly implemented. Please file bugs for built-ins
that are predictable but are not computed by Nuitka at compile time.
Conditional Statement Prediction
--------------------------------
For conditional statements, some branches may not ever be taken, because of the
conditions being possible to predict. In these cases, the branch not taken and
the condition check is removed.
This can typically predict code like this:
.. code-block:: python
if __name__ == "__main__":
# Your test code might be here
use_something_not_use_by_program()
or
.. code-block:: python
if False:
# Your deactivated code might be here
It will also benefit from constant propagations, or enable them because once
some branches have been removed, other things may become more predictable, so
this can trigger other optimization to become possible.
Every branch removed makes optimization more likely. With some code branches
removed, access patterns may be more friendly. Imagine e.g. that a function is
only called in a removed branch. It may be possible to remove it entirely, and
that may have other consequences too.
Status: This is considered implemented, but for the maximum benefit, more
constants needs to be determined at compile time.
Exception Propagation
---------------------
For exceptions that are determined at compile time, there is an expression that
will simply do raise the exception. These can be propagated, collecting
potentially "side effects", i.e. parts of expressions that must still be
executed.
Consider the following code:
.. code-block:: python
print side_effect_having() + (1 / 0)
print something_else()
The ``(1 / 0)`` can be predicted to raise a ``ZeroDivisionError`` exception,
which will be propagated through the ``+`` operation. That part is just Constant
Propagation as normal.
The call to ``side_effect_having`` will have to be retained though, but the
print statement, can be turned into an explicit raise. The statement sequence
can then be aborted and as such the ``something_else`` call needs no code
generation or consideration anymore.
To that end, Nuitka works with a special node that raises an exception and has
so called "side_effects" children, yet can be used in generated code as an
expression.
Status: The propagation of exceptions is implemented on a very basic level. It
works, but exceptions will not propagate through all different expression and
statement types. As work progresses or examples arise, the coverage will be
extended.
Exception Scope Reduction
-------------------------
Consider the following code:
.. code-block:: python
try:
b = 8
print range(3, b, 0)
print "Will not be executed"
except ValueError, e:
print e
The try block is bigger than it needs to be. The statement ``b = 8`` cannot
cause a ``ValueError`` to be raised. As such it can be moved to outside the try
without any risk.
.. code-block:: python
b = 8
try:
print range(3, b, 0)
print "Will not be executed"
except ValueError, e:
print e
Status: Not yet done yet. The infrastructure is in place, but until exception
block inlining works perfectly, there is not much of a point.
Exception Block Inlining
------------------------
With the exception propagation it is then possible to transform this code:
.. code-block:: python
try:
b = 8
print range(3, b, 0)
print "Will not be executed"
except ValueError, e:
print e
.. code-block:: python
try:
raise ValueError, "range() step argument must not be zero"
except ValueError, e:
print e
Which then can be reduced by avoiding the raise and catch of the exception,
making it:
.. code-block:: python
e = ValueError( "range() step argument must not be zero" )
print e
Status: This is not implemented yet.
Empty Branch Removal
--------------------
For loops and conditional statements that contain only code without effect, it
should be possible to remove the whole construct:
.. code-block:: python
for i in range(1000):
pass
The loop could be removed, at maximum it should be considered an assignment of
variable ``i`` to ``999`` and no more.
Another example:
.. code-block:: python
if side_effect_free:
pass
The condition should be removed in this case, as its evaluation is not
needed. It may be difficult to predict that ``side_effect_free`` has no side
effects, but many times this might be possible.
Status: This is not implemented yet.
Unpacking Prediction
--------------------
When the length of the right hand side of an assignment to a sequence can be
predicted, the unpacking can be replaced with multiple assignments.
.. code-block:: python
a, b, c = 1, side_effect_free(), 3
.. code-block:: python
a = 1
b = side_effect_free()
c = 3
This is of course only really safe if the left hand side cannot raise an
exception while building the assignment targets.
We do this now, but only for constants, because we currently have no ability to
predict if an expression can raise an exception or not.
Status: Not really implemented, and should use ``mayHaveSideEffect()`` to be
actually good at things.
Builtin Type Inference
----------------------
When a construct like ``in xrange()`` or ``in range()`` is used, it is possible
to know what the iteration does and represent that, so that iterator users can
use that instead.
I consider that:
.. code-block:: python
for i in xrange(1000):
something(i)
could translate ``xrange(1000)`` into an object of a special class that does the
integer looping more efficiently. In case ``i`` is only assigned from there,
this could be a nice case for a dedicated class.
Status: Future work, not even started.
Quicker Function Calls
----------------------
Functions are structured so that their parameter parsing and ``tp_call``
interface is separate from the actual function code. This way the call can be
optimized away. One problem is that the evaluation order can differ.
.. code-block:: python
def f(a, b, c):
return a, b, c
f( c = get1(), b = get2(), a = get3() )
This will evaluate first get1(), then get2() and then get3() and then make the
call.
In C++ whatever way the signature is written, its order is fixed.
Therefore it will be necessary to have a staging of the parameters before making
the actual call, to avoid an re-ordering of the calls to get1(), get2() and
get3().
To solve this, we may have to create wrapper functions that allow different
order of parameters to C++.
Status: Not even started.
Lowering of iterated Container Types
------------------------------------
In some cases, accesses to ``list`` constants can become ``tuple`` constants
instead.
Consider that:
.. code-block:: python
for x in [ 1, 2, 7 ]:
something( x )
Can be optimized into this:
.. code-block:: python
for x in ( 1, 2, 7 ):
something( x )
This allows for simpler code to be generated, and less checks needed, because
e.g. the ``tuple`` is clearly immutable, whereas the ``list`` needs a check to
assert that.
Something similar is possible for ``set`` and in theory also for ``dict``. For
the later it will be non-trivial though to maintain the order of execution
without temporary values introduced. The same thing is done for pure constants
of these types, they change to ``tuple`` values when iterated.
Status: Implemented, needs other optimization to become generally useful, will
help others to become possible.
Credits
=======
Contributors to Nuitka
----------------------
Thanks go to these individuals for their much valued contributions to
Nuitka. Contributors have the license to use Nuitka for their own code even if
Closed Source.
The order is sorted by time.
- Li Xuan Ji: Contributed patches for general portability issue and enhancements
to the environment variable settings.
- Nicolas Dumazet: Found and fixed reference counting issues, ``import``
packages work, improved some of the English and generally made good code
contributions all over the place, solved code generation TODOs, did tree
building cleanups, core stuff.
- Khalid Abu Bakr: Submitted patches for his work to support MinGW and Windows,
debugged the issues, and helped me to get cross compile with MinGW from Linux
to Windows. This was quite a difficult stuff.
- Liu Zhenhai: Submitted patches for Windows support, making the inline Scons
copy actually work on Windows as well. Also reported import related bugs, and
generally helped me make the Windows port more usable through his testing and
information.
- Christopher Tott: Submitted patches for Windows, and general as well as
structural cleanups.
- Pete Hunt: Submitted patches for MacOS X support.
- "ownssh": Submitted patches for built-ins module guarding, and made massive
efforts to make high quality bug reports. Also the initial "standalone" mode
implementation was created by him.
- Juan Carlos Paco: Submitted cleanup patches, creator of the `Nuitka GUI
`__, creator of the `Ninja IDE
plugin `__ for Nuitka.
- "dr. Equivalent": Submitted the Nuitka Logo.
- Johan Holmberg: Submitted patch for Python3 support on MacOS X.
- Umbra: Submitted patches to make the Windows port more usable, adding user
provided application icons, as well as MSVC support for large constants and
console applications.
Projects used by Nuitka
-----------------------
* The `CPython project `__
Thanks for giving us CPython, which is the base of Nuitka. We are nothing
without it.
* The `GCC project `__
Thanks for not only the best compiler suite, but also thanks for supporting
C++11 which helped to get Nuitka off the ground. Your compiler was the first
usable for Nuitka and with little effort.
* The `Scons project `__
Thanks for tackling the difficult points and providing a Python environment to
make the build results. This is such a perfect fit to Nuitka and a dependency
that will likely remain.
* The `valgrind project `__
Luckily we can use Valgrind to determine if something is an actual improvement
without the noise. And it's also helpful to determine what's actually
happening when comparing.
* The `NeuroDebian project `__
Thanks for hosting the build infrastructure that the Debian and sponsor
Yaroslav Halchenko uses to provide packages for all Ubuntu versions.
* The `openSUSE Buildservice `__
Thanks for hosting this excellent service that allows us to provide RPMs for a
large variety of platforms and make them available immediately nearly at
release time.
* The `MinGW project `__
Thanks for porting the gcc to Windows. This allowed portability of Nuitka with
relatively little effort. Unfortunately this is currently limited to compiling
CPython with 32 bits, and 64 bits requires MSVC compiler.
* The `Buildbot project `__
Thanks for creating an easy to deploy and use continous integration framework
that also runs on Windows and written and configured in Python. This allows to
run the Nuitka tests long before release time.
Updates for this Manual
=======================
This document is written in REST. That is an ASCII format which is readable as
ASCII, but used to generate PDF or HTML documents.
You will find the current source under:
http://nuitka.net/gitweb/?p=Nuitka.git;a=blob_plain;f=README.txt
And the current PDF under:
http://nuitka.net/doc/README.pdf
Nuitka-0.5.0.1/lib/ 0000755 0001750 0001750 00000000000 12265271051 014107 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/lib/hints.py 0000644 0001750 0001750 00000001414 12265264105 015610 0 ustar hayen hayen 0000000 0000000 # Copyright 2013, Kay Hayen, mailto:kay.hayen@gmail.com
#
# Part of "Nuitka", an optimizing Python compiler that is compatible and
# integrates with CPython, but also works on its own.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
Nuitka-0.5.0.1/doc/ 0000755 0001750 0001750 00000000000 12265271051 014106 5 ustar hayen hayen 0000000 0000000 Nuitka-0.5.0.1/doc/nuitka-run.1 0000644 0001750 0001750 00000013150 12265271041 016264 0 ustar hayen hayen 0000000 0000000 .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.43.3.
.TH NUITKA-RUN "1" "January 2014" "nuitka-run 0.5.0.1" "User Commands"
.SH NAME
nuitka-run \- the Python compiler
.SH SYNOPSIS
.B nuitka-run
[\fIoptions\fR] \fImain_module.py\fR
.SH OPTIONS
.TP
\fB\-\-version\fR
show program's version number and exit
.TP
\fB\-h\fR, \fB\-\-help\fR
show this help message and exit
.TP
\fB\-\-module\fR
Create an extension module executable instead of a
program. Defaults to off.
.TP
\fB\-\-standalone\fR, \fB\-\-portable\fR
Enable standalone mode in build. This allows you to
transfer the created binary to other machines without
it relying on an existing Python installation. It
implies these options: "\-\-recurse\-all \fB\-\-recursestdlib\fR". Defaults to off.
.TP
\fB\-\-python\-version\fR=\fIPYTHON_VERSION\fR
Major version of Python to be used, one of '2.6',
\&'2.7', '3.2', or '3.3'.
.TP
\fB\-\-python\-debug\fR
Use debug version or not. Default uses what you are
using to run Nuitka, most likely a non\-debug version.
.TP
\fB\-\-python\-flag\fR=\fIPYTHON_FLAGS\fR
Python flags to use. Default uses what you are using
to run Nuitka, this enforces a specific mode. These
are options that also exist to standard Python
executable. Currently supported "\-S" (alias nosite).
Default empty.
.TP
\fB\-\-windows\-disable\-console\fR
When compiling for windows, disable the console
window. Defaults to off.
.TP
\fB\-\-lto\fR
Use link time optimizations if available and usable
(g++ 4.6 and higher). Defaults to off.
.TP
\fB\-\-clang\fR
Enforce the use of clang (clang 3.0 or higher).
Defaults to off.
.TP
\fB\-\-mingw\fR
Enforce the use of MinGW on Windows. Defaults to off.
.TP
\fB\-j\fR N, \fB\-\-jobs\fR=\fIN\fR
Specify the allowed number of parallel C++ compiler
jobs. Defaults to the system CPU count.
.TP
\fB\-\-warn\-implicit\-exceptions\fR
Given warnings for implicit exceptions detected at
compile time.
.TP
\fB\-\-icon\fR=\fIICON_PATH\fR
Add executable icon (windows only).
.SS Control the recursion into imported modules:
.BR
.TP
\fB\-\-recurse\-stdlib\fR
Also descend into imported modules from standard
library. Defaults to off.
.TP
\fB\-\-recurse\-none\fR
When \fB\-\-recurse\-none\fR is used, do not descend into any
imported modules at all, overrides all other recursion
options. Defaults to off.
.TP
\fB\-\-recurse\-all\fR, \fB\-\-recurse\-on\fR
When \fB\-\-recurse\-all\fR is used, attempt to descend into
all imported modules. Defaults to off.
.TP
\fB\-\-recurse\-to\fR=\fIMODULE\fR/PACKAGE
Recurse to that module, or if a package, to the whole
package. Can be given multiple times. Default empty.
.TP
\fB\-\-recurse\-not\-to\fR=\fIMODULE\fR/PACKAGE
Do not recurse to that module, or if a package, to the
whole package in any case, overrides all other
options. Can be given multiple times. Default empty.
.TP
\fB\-\-recurse\-plugins\fR=\fIMODULE\fR/PACKAGE, \fB\-\-recurse\-directory\fR=\fIMODULE\fR/PACKAGE
Recurse into that directory, no matter if it's used by
the given main program in a visible form. Overrides
all other options. Can be given multiple times.
Default empty.
.SS Immediate execution after compilation:
.BR
.TP
\fB\-\-execute\fR
Execute immediately the created binary (or import the
compiled module). Defaults to on.
.TP
\fB\-\-execute\-with\-pythonpath\fR, \fB\-\-keep\-pythonpath\fR
When immediately executing the created binary
(\fB\-\-execute\fR), don't reset PYTHONPATH. When all modules
are successfully included, you ought to not need
PYTHONPATH anymore.
.SS Dump options for internal tree:
.BR
.TP
\fB\-\-dump\-xml\fR
Dump the final result of optimization as XML, then
exit.
.TP
\fB\-\-dump\-tree\fR
Dump the final result of optimization as text, then
exit.
.TP
\fB\-\-display\-tree\fR
Display the final result of optimization in a GUI,
then exit.
.SS Code generation choices:
.BR
.TP
\fB\-\-improved\fR, \fB\-\-enhanced\fR
Allow minor devitations from CPython behaviour, e.g.
better tracebacks, which are not really incompatible,
but different.
.TP
\fB\-\-code\-gen\-no\-statement\-lines\fR
Statements shall have their line numbers set. Disable
this for less precise exceptions and slightly faster
code. Not recommended. Defaults to off.
.SS Output directory choices:
.BR
.TP
\fB\-\-output\-dir\fR=\fIDIRECTORY\fR
Specify where intermediate and final output files
should be put. DIRECTORY will be populated with C++
files, object files, etc. Defaults to current
directory.
.TP
\fB\-\-remove\-output\fR
Removes the build directory after producing the module
or exe file. Defaults to off.
.SS Debug features:
.BR
.TP
\fB\-\-debug\fR
Executing all self checks possible to find errors in
Nuitka, do not use for production. Defaults to off.
.TP
\fB\-\-unstripped\fR, \fB\-\-no\-strip\fR, \fB\-\-unstriped\fR
Keep debug info in the resulting object file for
better gdb interaction. Defaults to off.
.TP
\fB\-\-trace\-execution\fR
Traced execution output, output the line of code
before executing it. Defaults to off.
.TP
\fB\-\-c\fR++\-only
Compile the would\-be regenerated source file. Allows
compiling edited C++ files with the C++ compiler for
quick debugging changes to the generated source.
Defaults to off.
.TP
\fB\-\-experimental\fR
Use features declared as 'experimental'. May have no
effect if no experimental features are present in the
code. Defaults to off.
.SS Tracing features:
.BR
.TP
\fB\-\-show\-scons\fR
Operate Scons in non\-quiet mode, showing the executed
commands. Defaults to off.
.TP
\fB\-\-show\-progress\fR
Provide progress information and statistics. Defaults
to off.
.TP
\fB\-\-show\-modules\fR
Provide a final summary on included modules. Defaults
to off.
.TP
\fB\-\-verbose\fR
Output details of actions take, esp. in optimizations.
Can become a lot. Defaults to off.
Nuitka-0.5.0.1/doc/nuitka.1 0000644 0001750 0001750 00000017376 12265271041 015500 0 ustar hayen hayen 0000000 0000000 .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.43.3.
.TH NUITKA "1" "January 2014" "nuitka 0.5.0.1" "User Commands"
.SH NAME
nuitka \- the Python compiler
.SH SYNOPSIS
.B nuitka
[\fI--module\fR] [\fI--execute\fR] [\fIoptions\fR] \fImain_module.py\fR
.SH OPTIONS
.TP
\fB\-\-version\fR
show program's version number and exit
.TP
\fB\-h\fR, \fB\-\-help\fR
show this help message and exit
.TP
\fB\-\-module\fR
Create an extension module executable instead of a
program. Defaults to off.
.TP
\fB\-\-standalone\fR, \fB\-\-portable\fR
Enable standalone mode in build. This allows you to
transfer the created binary to other machines without
it relying on an existing Python installation. It
implies these options: "\-\-recurse\-all \fB\-\-recursestdlib\fR". Defaults to off.
.TP
\fB\-\-python\-version\fR=\fIPYTHON_VERSION\fR
Major version of Python to be used, one of '2.6',
\&'2.7', '3.2', or '3.3'.
.TP
\fB\-\-python\-debug\fR
Use debug version or not. Default uses what you are
using to run Nuitka, most likely a non\-debug version.
.TP
\fB\-\-python\-flag\fR=\fIPYTHON_FLAGS\fR
Python flags to use. Default uses what you are using
to run Nuitka, this enforces a specific mode. These
are options that also exist to standard Python
executable. Currently supported "\-S" (alias nosite).
Default empty.
.TP
\fB\-\-windows\-disable\-console\fR
When compiling for windows, disable the console
window. Defaults to off.
.TP
\fB\-\-lto\fR
Use link time optimizations if available and usable
(g++ 4.6 and higher). Defaults to off.
.TP
\fB\-\-clang\fR
Enforce the use of clang (clang 3.0 or higher).
Defaults to off.
.TP
\fB\-\-mingw\fR
Enforce the use of MinGW on Windows. Defaults to off.
.TP
\fB\-j\fR N, \fB\-\-jobs\fR=\fIN\fR
Specify the allowed number of parallel C++ compiler
jobs. Defaults to the system CPU count.
.TP
\fB\-\-warn\-implicit\-exceptions\fR
Given warnings for implicit exceptions detected at
compile time.
.TP
\fB\-\-icon\fR=\fIICON_PATH\fR
Add executable icon (windows only).
.SS Control the recursion into imported modules:
.BR
.TP
\fB\-\-recurse\-stdlib\fR
Also descend into imported modules from standard
library. Defaults to off.
.TP
\fB\-\-recurse\-none\fR
When \fB\-\-recurse\-none\fR is used, do not descend into any
imported modules at all, overrides all other recursion
options. Defaults to off.
.TP
\fB\-\-recurse\-all\fR, \fB\-\-recurse\-on\fR
When \fB\-\-recurse\-all\fR is used, attempt to descend into
all imported modules. Defaults to off.
.TP
\fB\-\-recurse\-to\fR=\fIMODULE\fR/PACKAGE
Recurse to that module, or if a package, to the whole
package. Can be given multiple times. Default empty.
.TP
\fB\-\-recurse\-not\-to\fR=\fIMODULE\fR/PACKAGE
Do not recurse to that module, or if a package, to the
whole package in any case, overrides all other
options. Can be given multiple times. Default empty.
.TP
\fB\-\-recurse\-plugins\fR=\fIMODULE\fR/PACKAGE, \fB\-\-recurse\-directory\fR=\fIMODULE\fR/PACKAGE
Recurse into that directory, no matter if it's used by
the given main program in a visible form. Overrides
all other options. Can be given multiple times.
Default empty.
.SS Immediate execution after compilation:
.BR
.TP
\fB\-\-execute\fR
Execute immediately the created binary (or import the
compiled module). Defaults to off.
.TP
\fB\-\-execute\-with\-pythonpath\fR, \fB\-\-keep\-pythonpath\fR
When immediately executing the created binary
(\fB\-\-execute\fR), don't reset PYTHONPATH. When all modules
are successfully included, you ought to not need
PYTHONPATH anymore.
.SS Dump options for internal tree:
.BR
.TP
\fB\-\-dump\-xml\fR
Dump the final result of optimization as XML, then
exit.
.TP
\fB\-\-dump\-tree\fR
Dump the final result of optimization as text, then
exit.
.TP
\fB\-\-display\-tree\fR
Display the final result of optimization in a GUI,
then exit.
.SS Code generation choices:
.BR
.TP
\fB\-\-improved\fR, \fB\-\-enhanced\fR
Allow minor devitations from CPython behaviour, e.g.
better tracebacks, which are not really incompatible,
but different.
.TP
\fB\-\-code\-gen\-no\-statement\-lines\fR
Statements shall have their line numbers set. Disable
this for less precise exceptions and slightly faster
code. Not recommended. Defaults to off.
.SS Output directory choices:
.BR
.TP
\fB\-\-output\-dir\fR=\fIDIRECTORY\fR
Specify where intermediate and final output files
should be put. DIRECTORY will be populated with C++
files, object files, etc. Defaults to current
directory.
.TP
\fB\-\-remove\-output\fR
Removes the build directory after producing the module
or exe file. Defaults to off.
.SS Debug features:
.BR
.TP
\fB\-\-debug\fR
Executing all self checks possible to find errors in
Nuitka, do not use for production. Defaults to off.
.TP
\fB\-\-unstripped\fR, \fB\-\-no\-strip\fR, \fB\-\-unstriped\fR
Keep debug info in the resulting object file for
better gdb interaction. Defaults to off.
.TP
\fB\-\-trace\-execution\fR
Traced execution output, output the line of code
before executing it. Defaults to off.
.TP
\fB\-\-c\fR++\-only
Compile the would\-be regenerated source file. Allows
compiling edited C++ files with the C++ compiler for
quick debugging changes to the generated source.
Defaults to off.
.TP
\fB\-\-experimental\fR
Use features declared as 'experimental'. May have no
effect if no experimental features are present in the
code. Defaults to off.
.SS Tracing features:
.BR
.TP
\fB\-\-show\-scons\fR
Operate Scons in non\-quiet mode, showing the executed
commands. Defaults to off.
.TP
\fB\-\-show\-progress\fR
Provide progress information and statistics. Defaults
to off.
.TP
\fB\-\-show\-modules\fR
Provide a final summary on included modules. Defaults
to off.
.TP
\fB\-\-verbose\fR
Output details of actions take, esp. in optimizations.
Can become a lot. Defaults to off.
.SH EXAMPLES
Compile a python file "some_module.py" to a module "some_module.so":
.IP
\f(CW$ nuitka some_module.py\fR
.PP
Compile a python program "some_program.py" to an executable "some_program.exe":
.IP
\f(CW$ nuitka \-\-exe some_program.py\fR
.PP
Compile a python program "some_program.py" and the package "some_package" it
uses to an executable "some_program.exe":
.IP
\f(CW$ nuitka \-\-exe \-\-recurse\-to=some_package some_program.py\fR
.PP
Compile a python program "some_program.py" and all the modules it uses to an executable "some_program.exe". Then execute it immediately when ready:
.IP
\f(CW$ nuitka \-\-exe \-\-execute \-\-recurse\-all some_program.py\fR
.PP
Compile a python program "some_program.py" and the modules it uses (even standard library) to an executable "some_program.exe":
.IP
\f(CW$ nuitka \-\-recurse\-all \-\-recurse\-stdlib some_program.py \-\-exe\fR
.PP
Compile a python program "some_program.py" and the modules it uses to an executable "some_program.exe". Keep the debug information, so valrind, gdb, etc. work
nice.
Note: This will *not* degrade performance:
.IP
\f(CW$ nuitka \-\-unstriped \-\-recurse\-all some_program.py \-\-exe\fR
.PP
Compile a python program "some_program.py" and the modules it uses to an executable "some_program.exe". Perform all kinds of checks about correctness of the generated
C++ and run\-time checks.
Note: This will degrade performance and should only be used to debug Nuitka:
.IP
\f(CW$ nuitka \-\-debug \-\-recurse\-all some_program.py \-\-exe\fR
.PP
Compile a python program "some_program.py" and the modules it uses to an executable "some_program.exe". Perform all kinds of checks about correctness of the generated
C++ and run\-time checks. Also use the debug Python library, which does its own checks.
Note: This will degrade performance and should only be used to debug Nuitka:
.IP
\f(CW$ nuitka \-\-debug \-\-python-debug \-\-recurse\-all some_program.py \-\-exe\fR
.PP
Compile a python program "some_program.py" and the plugins modules it loads at run time to an executable "some_program.exe":
.IP
\f(CW$ nuitka \-\-recurse\-all \-\-recurse\-directory=plugins_dir some_program.py \-\-exe\fR
.PP