debian/0000755000000000000000000000000012222122167007163 5ustar debian/control0000644000000000000000000000221212222056211010557 0ustar Source: parallelpython Section: python Priority: optional Maintainer: Debian Python Modules Team Uploaders: Sandro Tosi Build-Depends: debhelper (>= 5.0.38), python-all (>= 2.5.4-1~) Build-Depends-Indep: python-support (>= 0.4) Homepage: http://www.parallelpython.com/ Standards-Version: 3.9.4 Vcs-Svn: svn://anonscm.debian.org/python-modules/packages/parallelpython/trunk/ Vcs-Browser: http://anonscm.debian.org/viewvc/python-modules/packages/parallelpython/trunk/ XS-Python-Version: all Package: python-pp Architecture: all Depends: ${shlibs:Depends}, ${misc:Depends}, ${python:Depends} XB-Python-Version: ${python:Versions} Description: parallel and distributed programming toolkit for Python Parallel Python module (pp) provides an easy and efficient way to create parallel-enabled applications for SMP computers and clusters. pp module features cross-platform portability and dynamic load balancing. Thus application written with PP will parallelize efficiently even on heterogeneous and multi-platform clusters (including clusters running other application with variable CPU loads). debian/source/0000755000000000000000000000000012222122167010463 5ustar debian/source/format0000644000000000000000000000001411616076523011703 0ustar 3.0 (quilt) debian/README.source0000644000000000000000000000023111124164211011332 0ustar This package uses dpatch to handle patches against upstream source code; you can find additional information about dpatch at /usr/share/doc/dpatch/ debian/rules0000755000000000000000000000075112222072500010241 0ustar #!/usr/bin/make -f %: dh $@ override_dh_installexamples: dh_installexamples examples/* override_dh_installchangelogs: dh_installchangelogs CHANGELOG override_dh_installdocs: dh_installdocs doc/ppdoc.html doc/example.config override_dh_installman: dh_installman doc/ppserver.1 override_dh_auto_install: dh_auto_install # Policy 10.4 - removing extension mv $(CURDIR)/debian/python-pp/usr/bin/ppserver.py $(CURDIR)/debian/python-pp/usr/bin/ppserver debian/python-pp.doc-base0000644000000000000000000000041411263707504012527 0ustar Document: python-pp Title: Parallel Python documentation Author: Vitalii Vanovschi Abstract: Documentation for Parallel Python module (pp) Section: Programming/Python Format: HTML Index: /usr/share/doc/python-pp/ppdoc.html Files: /usr/share/doc/python-pp/ppdoc.html debian/copyright0000644000000000000000000000367012100337105011117 0ustar This package was debianized by Sandro Tosi on Sun, 03 Feb 2008 18:44:13 +0100. It was downloaded from http://www.parallelpython.com Upstream Author: Vitalii Vanovschi Copyright: Copyright (c) 2005-2012 Vitalii Vanovschi License: Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the author nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The Debian packaging is Copyright (C) 2008-2013, Sandro Tosi and is licensed under the same license as upstream code. debian/changelog0000644000000000000000000001322612222122150011031 0ustar parallelpython (1.6.4-1) unstable; urgency=low [ Sandro Tosi ] * New upstream release; thanks to Yaroslav Halchenko for the report; Closes: #698716 * debian/copyright - extend packaging copyright years * debian/patches/10_remove_default_password.patch - updated to new upstream code * debian/patches/30_ppworker_invoke_via-m.patch - removed, merged upstream * debian/control - bump Standards-Version to 3.9.4 (no changes needed) * debian/rules - convert to dh sequencer [ Jakub Wilk ] * Use canonical URIs for Vcs-* fields. -- Sandro Tosi Mon, 30 Sep 2013 00:06:26 +0200 parallelpython (1.6.2-2) unstable; urgency=low * Invoke ppworker via "python -m" avoiding addition of /usr/share/pyshared to sys.path; thanks to Zbigniew JÄ™drzejewski-Szmek for the report and to Yaroslav Halchenko, Jakub Wilk for the fix; Closes: #620551 -- Sandro Tosi Wed, 23 Jan 2013 20:09:51 +0100 parallelpython (1.6.2-1) unstable; urgency=low * New upstream release * debian/copyright - updated upstream copyright years and extended packaging ones * debian/patches/10_remove_default_password.patch - updated to new upstream code * debian/control - bump Standards-Version to 3.9.3 (no changes needed) -- Sandro Tosi Thu, 07 Jun 2012 22:49:51 +0200 parallelpython (1.6.1-1) unstable; urgency=low * New upstream release * debian/copyright - updated both upstream and packaging copyright years * debian/control - bump Standards-Version to 3.9.2 (no changes needed) * Converted to '3.0 (quilt)' source format (so also dpatch -> quilt) -- Sandro Tosi Wed, 03 Aug 2011 00:57:04 +0200 parallelpython (1.6.0-1) unstable; urgency=low * New upstream release * debian/copyright - updated both upstream and packaging copyright years - added the complete license text, as suggested by lintian for BSD license * debian/patches/10_remove_default_password.dpatch - adapted to new upstream code * debian/control - bump Standards-Version to 3.9.0 (no changes needed) -- Sandro Tosi Thu, 15 Jul 2010 20:05:26 +0200 parallelpython (1.5.7-1) unstable; urgency=low [ Sandro Tosi ] * New upstream release * debian/rules - updated manpage location - updated html doc location, adding example.config * debian/copyright - extended upstream copyright years * debian/patches/10_remove_default_password.dpatch - adapted to new upstream code * debian/control - bump Standards-Version to 3.8.3 (no changes needed) * debian/python-pp.doc-base - added doc-base * debian/patches/20_hyphen-used-as-minus-sign.dpatch - added to fix hyphen-used-as-minus-sign lintian warning [ Julien Lavergne ] * From Ubuntu, prepare for the future Python 2.6 transition: - debian/rules: + include /usr/share/python/python.mk + add $(py_setup_install_args) to setup.py install - debian/control: + bump build-depends to python-all (>= 2.5.4-1~) + bump build-depends to debhelper (>= 5.0.38) -- Sandro Tosi Sat, 31 Oct 2009 22:07:44 +0100 parallelpython (1.5.6-2) unstable; urgency=low * Uploading to unstable, since Lenny has been released * debian/control - switch Vcs-Browser field to viewsvn - updated my email address - removed DM-Upload-Allowed field * debian/README.source - added (as requested by Policy 3.8.0) * debian/copyright - updated my email address - updated packaging copyright notice, extending years - clarified the packaging is licensed under the same terms as upstream code * debian/rules - moved removal of 'build' dir after python clean - merged 'rm' call into 'dh_clean' one -- Sandro Tosi Sun, 15 Feb 2009 20:27:10 +0100 parallelpython (1.5.6-1) experimental; urgency=low * New upstream release -- Sandro Tosi Sun, 14 Sep 2008 01:24:34 +0200 parallelpython (1.5.5-1) experimental; urgency=low * New upstream release * debian/patches/10_remove_default_password.dpatch - adapted to new upstream code * debian/patches/20_backport_segfault_fix.dpatch - removed, it was backported from this release -- Sandro Tosi Fri, 22 Aug 2008 16:31:35 +0200 parallelpython (1.5.4-2) unstable; urgency=low * debian/patches/20_backport_segfault_fix.dpatch - backporting from 1.5.5: it fixed a segfault when calling destroy() method -- Sandro Tosi Wed, 20 Aug 2008 01:18:40 +0200 parallelpython (1.5.4-1) unstable; urgency=low * New upstream release * debian/rules - added removal of objects Python files * debian/patches/10_remove_default_password.dpatch - updated to match new upstream code - little formatting changes both in html and manpage files - minus sings are escaped in the manpage - added description * debian/control - added "DM-Upload-Allowed: yes" - bump Standard-Version to 3.8.0 (no changes needed) * debian/copyright - removed the license text, leaving only the reference on a Debian system and a short notice about it. -- Sandro Tosi Sat, 07 Jun 2008 16:24:58 +0200 parallelpython (1.5.3-2) unstable; urgency=low * debian/patches/10_remove_default_password.dpatch - updated to uniform documentation about security issues - updated to uniform ppserver name - better execption handling in ppserver -- Sandro Tosi Sat, 08 Mar 2008 20:13:49 +0100 parallelpython (1.5.3-1) unstable; urgency=low * Initial release (Closes: #448093) -- Sandro Tosi Mon, 25 Feb 2008 23:35:10 +0100 debian/patches/0000755000000000000000000000000012222122167010612 5ustar debian/patches/10_remove_default_password.patch0000644000000000000000000002521112100342646017060 0ustar Author: Sandro Tosi Description: This patch removes the default password from pp code, requiring users to explicitly write one; it updates the documentation along the source code --- a/doc/ppdoc.html +++ b/doc/ppdoc.html @@ -158,13 +158,11 @@ to the number of pro and list of ppservers to connect with :

    job_server = pp.Server(ppservers=ppservers) 

5) Submit all the tasks for parallel execution:

    f1 = job_server.submit(func1, args1, depfuncs1, modules1)

    f2 = job_server.submit(func1, args2, depfuncs1, modules1)

    f3 = job_server.submit(func2, args3, depfuncs2, modules2)

   ...etc...

6) Retrieve the results as needed:

    r1 = f1()

    r2 = f2()

    r3 = f3() 

    ...etc...

 To find out how to achieve efficient parallelization with pp please take a look at examples 

    Advanced guide, clusters 

On the nodes  

1) Start parallel python execution server on all your remote computational nodes (listen to a given port 35000,
and local network interface only, accept only connections which know correct secret):

    node-1> ./ppserver.py -p 35000 -i 192.168.0.101 -s "mysecret"

    node-2> ./ppserver.py -p 35000 -i 192.168.0.102 -s "mysecret"

    node-3> ./ppserver.py -p 35000 -i 192.168.0.103 -s "mysecret"

On the client

2) Import pp module:

    import pp

3)  Create a list of all the nodes in your cluster (computers where you've run ppserver.py)

    ppservers=("node-1:35000", "node-2:35000", "node-3:35000")

4) Start pp execution server with the number of workers set to the number of processors in the system, -
list of ppservers to connect with and secret key to authorize the connection:

    job_server = pp.Server(ppservers=ppservers, secret="mysecret") 

5) Submit all the tasks for parallel execution:

    f1 = job_server.submit(func1, args1, depfuncs1, modules1)

    f2 = job_server.submit(func1, args2, depfuncs1, modules1)

    f3 = job_server.submit(func2, args3, depfuncs2, modules2)

   ...etc...

6) Retrieve the results as needed:

    r1 = f1()

    r2 = f2()

    r3 = f3() 

    ...etc...

 7) Print the execution statistics:

    job_server.print_stats()

To find out how to achieve efficient parallelization with pp please take a look at examples

  Command line options, ppserver.py 

Usage: ppserver.py [-hda] [-i interface] [-b broadcast] [-p port] [-w nworkers] [-s secret] [-t seconds]
 Options:
 -h                 : this help message
 -d                 : debug
 -a                 : enable auto-discovery service
 -i interface       : interface to listen
 -b broadcast       : broadcast address for auto-discovery service
 -p port            : port to listen
 -w nworkers        : number of workers to start
 -s secret          : secret for authentication
 -t seconds         : timeout to exit if no connections with clients exist
 -k seconds         : socket timeout in seconds 
 -P pid_file          : file to write PID to

  Security and secret key

 Due - to the security concerns it is highly recommended to run ppserver.py -with an non-trivial secret key (-s command line argument) which should -be paired with the matching secret keyword of PP Server class constructor. Since PP 1.5.3 it is possible to set secret key by assigning pp_secret variable in the configuration file .pythonrc.py - which should be located in the user home directory (please make this -file readable and writable only by user). The key set in .pythonrc.py -could be overridden by command line argument (for ppserver.py) and secret keyword (for PP Server class constructor).

 

  ppserver.py stats and PID file example

To print job execution statistics for ppserver.py send a SIGUSR1 signal to its main process.
For instance on UNIX platform following commands will start a server and print its stats:
ppserver.py  -P /tmp/ppserver.pid

kill -s SIGUSR1 `cat /tmp/ppserver.pid`

+
list of ppservers to connect with and secret key to authorize the connection:

    job_server = pp.Server(ppservers=ppservers, secret="mysecret") 

5) Submit all the tasks for parallel execution:

    f1 = job_server.submit(func1, args1, depfuncs1, modules1)

    f2 = job_server.submit(func1, args2, depfuncs1, modules1)

    f3 = job_server.submit(func2, args3, depfuncs2, modules2)

   ...etc...

6) Retrieve the results as needed:

    r1 = f1()

    r2 = f2()

    r3 = f3() 

    ...etc...

 7) Print the execution statistics:

    job_server.print_stats()

To find out how to achieve efficient parallelization with pp please take a look at examples

  Command line options, ppserver.py 

Usage: ppserver.py [-hda] [-i interface] [-b broadcast] [-p port] [-w nworkers] [-s secret] [-t seconds]
 Options:
 -h                 : this help message
 -d                 : debug
 -a                 : enable auto-discovery service
 -i interface       : interface to listen
 -b broadcast       : broadcast address for auto-discovery service
 -p port            : port to listen
 -w nworkers        : number of workers to start
 -s secret          : secret for authentication
 -t seconds         : timeout to exit if no connections with clients exist
 -k seconds         : socket timeout in seconds 
 -P pid_file          : file to write PID to

  Security and secret key + +

 Due to the security concerns, in Debian we decided to disable default password authentication, and require to run ppserver with a non-trivial secret key (-s command line argument) which should be paired with the matching secret keyword of PP Server class constructor. An alternative way to set a secret key is by assigning pp_secret variable in the configuration file .pythonrc.py which should be located in the user home directory (please make this file readable and writable only by user). The secret key set in .pythonrc.py could be overridden by command line argument (for ppserver) and secret keyword (for PP Server class constructor). Note that passing the password on the command line allows every user to see it (e.g. using ps(1)), and that running it on an open/untrusted network can be a security problem as traf fic, including the password, is not encrypted.

+ +

 

  ppserver.py stats and PID file example

To print job execution statistics for ppserver.py send a SIGUSR1 signal to its main process.
For instance on UNIX platform following commands will start a server and print its stats:
ppserver.py  -P /tmp/ppserver.pid

kill -s SIGUSR1 `cat /tmp/ppserver.pid`

--- a/pp.py +++ b/pp.py @@ -277,7 +277,6 @@ class Server(object): """ default_port = 60000 - default_secret = "epo20pdosl;dksldkmm" def __init__(self, ncpus="autodetect", ppservers=(), secret=None, restart=False, proto=2, socket_timeout=3600): @@ -288,9 +287,8 @@ class Server(object): the number of processors in the system ppservers - list of active parallel python execution servers to connect with - secret - passphrase for network connections, if omitted a default - passphrase will be used. It's highly recommended to use a - custom passphrase for all network connections. + secret - passphrase for network connections; it can be set via + command-line or configuration file restart - whether to restart worker process after each task completion proto - protocol number for pickle module socket_timeout - socket timeout in seconds which is also the maximum @@ -372,7 +370,7 @@ class Server(object): raise TypeError("secret must be of a string type") self.secret = str(secret) else: - self.secret = Server.default_secret + raise ValueError("secret must be set using command-line option or configuration file") self.__connect() self.__creation_time = time.time() self.logger.info("pp local server started with %d workers" debian/patches/20_hyphen-used-as-minus-sign.patch0000644000000000000000000000154711616076523017076 0ustar Author: Sandro Tosi Description: escape minus sign where needed --- parallelpython-1.6.1.orig/doc/ppserver.1 +++ parallelpython-1.6.1/doc/ppserver.1 @@ -49,7 +49,7 @@ documentation, examples and support forums .br .SH SECURITY -Due to the security concerns it is highly recommended to run ppserver.py with an non-trivial secret key (-s command line argument) which should be paired with the matching secret keyword of PP Server class constructor. An alternative way to set a secret key is by assigning +Due to the security concerns it is highly recommended to run ppserver.py with an non-trivial secret key (\-s command line argument) which should be paired with the matching secret keyword of PP Server class constructor. An alternative way to set a secret key is by assigning .B pp_secret variable in the configuration file .B .pythonrc.py debian/patches/series0000644000000000000000000000010412222056124012021 0ustar 10_remove_default_password.patch 20_hyphen-used-as-minus-sign.patch debian/watch0000644000000000000000000000021310752120126010207 0ustar # Compulsory line, this is a version 3 file version=3 http://www.parallelpython.com/content/view/18/32/ downloads/pp/pp-([\d.]*)\.tar\.gz debian/compat0000644000000000000000000000000210752120126010360 0ustar 5