psycogreen-1.0/0000775000175000017500000000000012035402203013465 5ustar piropiro00000000000000psycogreen-1.0/README.rst0000664000175000017500000000667612035400365015202 0ustar piropiro00000000000000psycogreen -- psycopg2 integration with coroutine libraries =========================================================== The `psycogreen`_ package enables psycopg2 to work with coroutine libraries, using asynchronous calls internally but offering a blocking interface so that regular code can run unmodified. `Psycopg`_ offers `coroutines support`__ since release 2.2. Because the main module is a C extension it cannot be monkey-patched to become coroutine-friendly. Instead it exposes `a hook`__ that coroutine libraries can use to install a function integrating with their event scheduler. Psycopg will call the function whenever it executes a libpq call that may block. `psycogreen` is a collection of "wait callbacks" useful to integrate Psycopg with different coroutine libraries. .. _psycogreen: https://bitbucket.org/dvarrazzo/psycogreen .. _Psycopg: http://initd.org/psycopg/ .. __: http://initd.org/psycopg/docs/advanced.html#support-to-coroutine-libraries .. __: http://initd.org/psycopg/docs/extensions.html#psycopg2.extensions.set_wait_callback Module ``psycogreen.eventlet`` ------------------------------ `Eventlet`_ currently supports Psycopg out-of-the-box and ``psycogreen`` is not necessary. See `the documentation`__ for patching instruction. .. _Eventlet: http://eventlet.net/ .. __: http://eventlet.net/doc/patching.html#monkeypatching-the-standard-library If for any reason you want to avoid using Eventlet monkeypatching you can use ``psycogreen.eventlet.patch_psycopg()``. Function ``psycogreen.eventlet.patch_psycopg()`` Enable async processing in Psycopg integrated with the Eventlet events loop. It is performed by registering ``eventlet_wait_callback()`` as psycopg2 wait callback. Function ``psycogreen.eventlet.eventlet_wait_callback(conn)`` A wait callback integrating with Eventlet events loop. An example script showing concurrent usage of ``psycopg2`` with ``urlopen()`` with Eventlet is available in |tests/test_eventlet.py|__. .. |tests/test_eventlet.py| replace:: ``tests/test_eventlet.py`` .. __: https://bitbucket.org/dvarrazzo/psycogreen/src/master/tests/test_eventlet.py Module ``psycogreen.gevent`` ---------------------------- In order to use psycopg2 asynchronously with `gevent`_ you can use ``psycogreen.gevent.patch_psycopg()``. Function ``psycogreen.gevent.patch_psycopg()`` Enable async processing in Psycopg integrated with the gevent events loop. It is performed by registering ``gevent_wait_callback()`` as psycopg2 wait callback. Function ``psycogreen.gevent.gevent_wait_callback(conn)`` A wait callback integrating with gevent events loop. An example script showing concurrent usage of ``psycopg2`` with ``urlopen()`` with gevent is available in |tests/test_gevent.py|__. .. _gevent: http://www.gevent.org/ .. |tests/test_gevent.py| replace:: ``tests/test_gevent.py`` .. __: https://bitbucket.org/dvarrazzo/psycogreen/src/master/tests/test_gevent.py uWSGI green threads ------------------- Roberto De Ioris is writing uGreen__, a green thread implementation on top of the `uWSGI async platform`__. .. __: http://projects.unbit.it/uwsgi/wiki/uGreen .. __: http://projects.unbit.it/uwsgi/ He has performed some tests using both `psycopg2 async support`__ and `psycopg2 green support`__ and has reported no problem in their stress tests with both the async styles. .. __: http://projects.unbit.it/uwsgi/browser/tests/psycopg2_green.py .. __: http://projects.unbit.it/uwsgi/browser/tests/psycogreen_green.py psycogreen-1.0/COPYING0000664000175000017500000000271412031645405014535 0ustar piropiro00000000000000Copyright (c) 2010-2012, Daniele Varrazzo All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * The name of Daniele Varrazzo may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. psycogreen-1.0/setup.py0000664000175000017500000000155612035401623015213 0ustar piropiro00000000000000#!/usr/bin/env python """ psycogreen -- setup script """ # Copyright (C) 2010-2012 Daniele Varrazzo from setuptools import setup from psycogreen import __version__ classifiers = """ Development Status :: 5 - Production/Stable License :: OSI Approved :: BSD License Operating System :: OS Independent Programming Language :: Python :: 2 Topic :: Database """ kwargs = {} try: kwargs['long_description'] = open('README.rst').read() except: pass setup( name = 'psycogreen', description = 'psycopg2 integration with coroutine libraries', author = 'Daniele Varrazzo', author_email = 'daniele.varrazzo@gmail.com', url = 'https://bitbucket.org/dvarrazzo/psycogreen', license = 'BSD', packages = ['psycogreen'], classifiers = [x for x in classifiers.split('\n') if x], version = __version__, **kwargs) psycogreen-1.0/PKG-INFO0000664000175000017500000001114612035402203014565 0ustar piropiro00000000000000Metadata-Version: 1.0 Name: psycogreen Version: 1.0 Summary: psycopg2 integration with coroutine libraries Home-page: https://bitbucket.org/dvarrazzo/psycogreen Author: Daniele Varrazzo Author-email: daniele.varrazzo@gmail.com License: BSD Description: psycogreen -- psycopg2 integration with coroutine libraries =========================================================== The `psycogreen`_ package enables psycopg2 to work with coroutine libraries, using asynchronous calls internally but offering a blocking interface so that regular code can run unmodified. `Psycopg`_ offers `coroutines support`__ since release 2.2. Because the main module is a C extension it cannot be monkey-patched to become coroutine-friendly. Instead it exposes `a hook`__ that coroutine libraries can use to install a function integrating with their event scheduler. Psycopg will call the function whenever it executes a libpq call that may block. `psycogreen` is a collection of "wait callbacks" useful to integrate Psycopg with different coroutine libraries. .. _psycogreen: https://bitbucket.org/dvarrazzo/psycogreen .. _Psycopg: http://initd.org/psycopg/ .. __: http://initd.org/psycopg/docs/advanced.html#support-to-coroutine-libraries .. __: http://initd.org/psycopg/docs/extensions.html#psycopg2.extensions.set_wait_callback Module ``psycogreen.eventlet`` ------------------------------ `Eventlet`_ currently supports Psycopg out-of-the-box and ``psycogreen`` is not necessary. See `the documentation`__ for patching instruction. .. _Eventlet: http://eventlet.net/ .. __: http://eventlet.net/doc/patching.html#monkeypatching-the-standard-library If for any reason you want to avoid using Eventlet monkeypatching you can use ``psycogreen.eventlet.patch_psycopg()``. Function ``psycogreen.eventlet.patch_psycopg()`` Enable async processing in Psycopg integrated with the Eventlet events loop. It is performed by registering ``eventlet_wait_callback()`` as psycopg2 wait callback. Function ``psycogreen.eventlet.eventlet_wait_callback(conn)`` A wait callback integrating with Eventlet events loop. An example script showing concurrent usage of ``psycopg2`` with ``urlopen()`` with Eventlet is available in |tests/test_eventlet.py|__. .. |tests/test_eventlet.py| replace:: ``tests/test_eventlet.py`` .. __: https://bitbucket.org/dvarrazzo/psycogreen/src/master/tests/test_eventlet.py Module ``psycogreen.gevent`` ---------------------------- In order to use psycopg2 asynchronously with `gevent`_ you can use ``psycogreen.gevent.patch_psycopg()``. Function ``psycogreen.gevent.patch_psycopg()`` Enable async processing in Psycopg integrated with the gevent events loop. It is performed by registering ``gevent_wait_callback()`` as psycopg2 wait callback. Function ``psycogreen.gevent.gevent_wait_callback(conn)`` A wait callback integrating with gevent events loop. An example script showing concurrent usage of ``psycopg2`` with ``urlopen()`` with gevent is available in |tests/test_gevent.py|__. .. _gevent: http://www.gevent.org/ .. |tests/test_gevent.py| replace:: ``tests/test_gevent.py`` .. __: https://bitbucket.org/dvarrazzo/psycogreen/src/master/tests/test_gevent.py uWSGI green threads ------------------- Roberto De Ioris is writing uGreen__, a green thread implementation on top of the `uWSGI async platform`__. .. __: http://projects.unbit.it/uwsgi/wiki/uGreen .. __: http://projects.unbit.it/uwsgi/ He has performed some tests using both `psycopg2 async support`__ and `psycopg2 green support`__ and has reported no problem in their stress tests with both the async styles. .. __: http://projects.unbit.it/uwsgi/browser/tests/psycopg2_green.py .. __: http://projects.unbit.it/uwsgi/browser/tests/psycogreen_green.py Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python :: 2 Classifier: Topic :: Database psycogreen-1.0/psycogreen.egg-info/0000775000175000017500000000000012035402203017335 5ustar piropiro00000000000000psycogreen-1.0/psycogreen.egg-info/SOURCES.txt0000664000175000017500000000046312035402203021224 0ustar piropiro00000000000000COPYING MANIFEST.in README.rst setup.py psycogreen/__init__.py psycogreen/eventlet.py psycogreen/gevent.py psycogreen.egg-info/PKG-INFO psycogreen.egg-info/SOURCES.txt psycogreen.egg-info/dependency_links.txt psycogreen.egg-info/top_level.txt tests/test_eventlet.py tests/test_gevent.py tools/wait_server.pypsycogreen-1.0/psycogreen.egg-info/PKG-INFO0000664000175000017500000001114612035402203020435 0ustar piropiro00000000000000Metadata-Version: 1.0 Name: psycogreen Version: 1.0 Summary: psycopg2 integration with coroutine libraries Home-page: https://bitbucket.org/dvarrazzo/psycogreen Author: Daniele Varrazzo Author-email: daniele.varrazzo@gmail.com License: BSD Description: psycogreen -- psycopg2 integration with coroutine libraries =========================================================== The `psycogreen`_ package enables psycopg2 to work with coroutine libraries, using asynchronous calls internally but offering a blocking interface so that regular code can run unmodified. `Psycopg`_ offers `coroutines support`__ since release 2.2. Because the main module is a C extension it cannot be monkey-patched to become coroutine-friendly. Instead it exposes `a hook`__ that coroutine libraries can use to install a function integrating with their event scheduler. Psycopg will call the function whenever it executes a libpq call that may block. `psycogreen` is a collection of "wait callbacks" useful to integrate Psycopg with different coroutine libraries. .. _psycogreen: https://bitbucket.org/dvarrazzo/psycogreen .. _Psycopg: http://initd.org/psycopg/ .. __: http://initd.org/psycopg/docs/advanced.html#support-to-coroutine-libraries .. __: http://initd.org/psycopg/docs/extensions.html#psycopg2.extensions.set_wait_callback Module ``psycogreen.eventlet`` ------------------------------ `Eventlet`_ currently supports Psycopg out-of-the-box and ``psycogreen`` is not necessary. See `the documentation`__ for patching instruction. .. _Eventlet: http://eventlet.net/ .. __: http://eventlet.net/doc/patching.html#monkeypatching-the-standard-library If for any reason you want to avoid using Eventlet monkeypatching you can use ``psycogreen.eventlet.patch_psycopg()``. Function ``psycogreen.eventlet.patch_psycopg()`` Enable async processing in Psycopg integrated with the Eventlet events loop. It is performed by registering ``eventlet_wait_callback()`` as psycopg2 wait callback. Function ``psycogreen.eventlet.eventlet_wait_callback(conn)`` A wait callback integrating with Eventlet events loop. An example script showing concurrent usage of ``psycopg2`` with ``urlopen()`` with Eventlet is available in |tests/test_eventlet.py|__. .. |tests/test_eventlet.py| replace:: ``tests/test_eventlet.py`` .. __: https://bitbucket.org/dvarrazzo/psycogreen/src/master/tests/test_eventlet.py Module ``psycogreen.gevent`` ---------------------------- In order to use psycopg2 asynchronously with `gevent`_ you can use ``psycogreen.gevent.patch_psycopg()``. Function ``psycogreen.gevent.patch_psycopg()`` Enable async processing in Psycopg integrated with the gevent events loop. It is performed by registering ``gevent_wait_callback()`` as psycopg2 wait callback. Function ``psycogreen.gevent.gevent_wait_callback(conn)`` A wait callback integrating with gevent events loop. An example script showing concurrent usage of ``psycopg2`` with ``urlopen()`` with gevent is available in |tests/test_gevent.py|__. .. _gevent: http://www.gevent.org/ .. |tests/test_gevent.py| replace:: ``tests/test_gevent.py`` .. __: https://bitbucket.org/dvarrazzo/psycogreen/src/master/tests/test_gevent.py uWSGI green threads ------------------- Roberto De Ioris is writing uGreen__, a green thread implementation on top of the `uWSGI async platform`__. .. __: http://projects.unbit.it/uwsgi/wiki/uGreen .. __: http://projects.unbit.it/uwsgi/ He has performed some tests using both `psycopg2 async support`__ and `psycopg2 green support`__ and has reported no problem in their stress tests with both the async styles. .. __: http://projects.unbit.it/uwsgi/browser/tests/psycopg2_green.py .. __: http://projects.unbit.it/uwsgi/browser/tests/psycogreen_green.py Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python :: 2 Classifier: Topic :: Database psycogreen-1.0/psycogreen.egg-info/top_level.txt0000664000175000017500000000001312035402203022061 0ustar piropiro00000000000000psycogreen psycogreen-1.0/psycogreen.egg-info/dependency_links.txt0000664000175000017500000000000112035402203023403 0ustar piropiro00000000000000 psycogreen-1.0/tests/0000775000175000017500000000000012035402203014627 5ustar piropiro00000000000000psycogreen-1.0/tests/test_gevent.py0000775000175000017500000000252712035400435017546 0ustar piropiro00000000000000#!/usr/bin/env python """A test to verify Psycopg collaboration with other blocking I/O. Please run the script ``tools/wait_server.py`` in a separate shell to make the test work. If the test works you should see download tasks overlapping query tasks. """ # Copyright (C) 2010-2012 Daniele Varrazzo # All rights reserved. See COPYING file for details. import gevent import gevent.monkey gevent.monkey.patch_all() import psycogreen.gevent psycogreen.gevent.patch_psycopg() import urllib2 # green import psycopg2 import logging logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s") logger = logging.getLogger() logger.info("testing psycopg2 with gevent") conn = psycopg2.connect("dbname=postgres") def download(num, secs): url = "http://localhost:8000/%d/" % secs for i in range(num): logger.info("download %d start", i) data = urllib2.urlopen(url).read() logger.info("download %d end", i) def fetch(num, secs): cur = conn.cursor() for i in range(num): logger.info("query %d start", i) cur.execute("select pg_sleep(%s)", (secs,)) logger.info("query %d end", i) logger.info("making jobs") jobs = [ gevent.spawn(download, 2, 3), gevent.spawn(fetch, 3, 2), ] logger.info("join begin") gevent.joinall(jobs) logger.info("join end") psycogreen-1.0/tests/test_eventlet.py0000775000175000017500000000250312035400426020076 0ustar piropiro00000000000000#!/usr/bin/env python """A test to verify Psycopg collaboration with other blocking I/O. Please run the script ``tools/wait_server.py`` in a separate shell to make the test work. If the test works you should see download tasks overlapping query tasks. """ # Copyright (C) 2010-2012 Daniele Varrazzo # All rights reserved. See COPYING file for details. import eventlet eventlet.monkey_patch() import psycogreen.eventlet psycogreen.eventlet.patch_psycopg() import urllib2 # green import psycopg2 import logging logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s") logger = logging.getLogger() logger.info("testing psycopg2 with eventlet") conn = psycopg2.connect("dbname=postgres") def download(num, secs): url = "http://localhost:8000/%d/" % secs for i in range(num): logger.info("download %d start", i) data = urllib2.urlopen(url).read() logger.info("download %d end", i) def fetch(num, secs): cur = conn.cursor() for i in range(num): logger.info("query %d start", i) cur.execute("select pg_sleep(%s)", (secs,)) logger.info("query %d end", i) logger.info("making jobs") pool = eventlet.GreenPool() pool.spawn(download, 2, 3), pool.spawn(fetch, 3, 2), logger.info("join begin") pool.waitall() logger.info("join end") psycogreen-1.0/setup.cfg0000664000175000017500000000007312035402203015306 0ustar piropiro00000000000000[egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 psycogreen-1.0/tools/0000775000175000017500000000000012035402203014625 5ustar piropiro00000000000000psycogreen-1.0/tools/wait_server.py0000775000175000017500000000125412031645405017547 0ustar piropiro00000000000000#!/usr/bin/env python """A server to test with blocking I/O.""" # Copyright (C) 2010-2012 Daniele Varrazzo # All rights reserved. See COPYING file for details. import time from wsgiref.simple_server import make_server def wait_app(environ, start_response): """An application serving blocking pages.""" status = '200 OK' headers = [('Content-type', 'text/plain')] start_response(status, headers) try: secs = int(environ['PATH_INFO'].replace('/', '')) except: secs = 0 time.sleep(secs) return [ str(secs) ] httpd = make_server('', 8000, wait_app) print "Serving on port 8000..." httpd.serve_forever() psycogreen-1.0/psycogreen/0000775000175000017500000000000012035402203015643 5ustar piropiro00000000000000psycogreen-1.0/psycogreen/gevent.py0000664000175000017500000000236012035400452017512 0ustar piropiro00000000000000"""A wait callback to allow psycopg2 cooperation with gevent. Use `patch_psycopg()` to enable gevent support in Psycopg. """ # Copyright (C) 2010-2012 Daniele Varrazzo # All rights reserved. See COPYING file for details. from __future__ import absolute_import import psycopg2 from psycopg2 import extensions from gevent.socket import wait_read, wait_write def patch_psycopg(): """Configure Psycopg to be used with gevent in non-blocking way.""" if not hasattr(extensions, 'set_wait_callback'): raise ImportError( "support for coroutines not available in this Psycopg version (%s)" % psycopg2.__version__) extensions.set_wait_callback(gevent_wait_callback) def gevent_wait_callback(conn, timeout=None): """A wait callback useful to allow gevent to work with Psycopg.""" while 1: state = conn.poll() if state == extensions.POLL_OK: break elif state == extensions.POLL_READ: wait_read(conn.fileno(), timeout=timeout) elif state == extensions.POLL_WRITE: wait_write(conn.fileno(), timeout=timeout) else: raise psycopg2.OperationalError( "Bad result from poll: %r" % state) psycogreen-1.0/psycogreen/__init__.py0000664000175000017500000000031612035401342017757 0ustar piropiro00000000000000"""Integration of psycopg2 with coroutine framework """ # Copyright (C) 2010-2012 Daniele Varrazzo # All rights reserved. See COPYING file for details. __version__ = '1.0' psycogreen-1.0/psycogreen/eventlet.py0000664000175000017500000000234512035400442020052 0ustar piropiro00000000000000"""A wait callback to allow psycopg2 cooperation with eventlet. Use `patch_psycopg()` to enable eventlet support in Psycopg. """ # Copyright (C) 2010-2012 Daniele Varrazzo # All rights reserved. See COPYING file for details. from __future__ import absolute_import import psycopg2 from psycopg2 import extensions from eventlet.hubs import trampoline def patch_psycopg(): """Configure Psycopg to be used with eventlet in non-blocking way.""" if not hasattr(extensions, 'set_wait_callback'): raise ImportError( "support for coroutines not available in this Psycopg version (%s)" % psycopg2.__version__) extensions.set_wait_callback(eventlet_wait_callback) def eventlet_wait_callback(conn, timeout=-1): """A wait callback useful to allow eventlet to work with Psycopg.""" while 1: state = conn.poll() if state == extensions.POLL_OK: break elif state == extensions.POLL_READ: trampoline(conn.fileno(), read=True) elif state == extensions.POLL_WRITE: trampoline(conn.fileno(), write=True) else: raise psycopg2.OperationalError( "Bad result from poll: %r" % state) psycogreen-1.0/MANIFEST.in0000664000175000017500000000021412035377050015233 0ustar piropiro00000000000000include COPYING MANIFEST.in README.rst setup.py recursive-include psycogreen *.py recursive-include tests *.py recursive-include tools *.py