oops-twisted-0.0.6/0000755000175000017500000000000011670323604014420 5ustar roaksoaxroaksoaxoops-twisted-0.0.6/oops_twisted/0000755000175000017500000000000011670323604017143 5ustar roaksoaxroaksoaxoops-twisted-0.0.6/oops_twisted/createhooks.py0000644000175000017500000000235211670323103022020 0ustar roaksoaxroaksoax# Copyright (c) 2011, Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 only. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . # GNU Lesser General Public License version 3 (see the file LICENSE). """Extensions to permit creatings OOPS reports with twisted types.""" __metaclass__ = type __all__ = [ 'failure_to_context', ] def failure_to_context(report, context): """If a twisted_failure key is present, use it to set context['exc_info']. This permits using regular python hooks with a twisted failure. """ failure = context.get('twisted_failure') if not failure: return exc_info=(failure.type, failure.value, failure.getTraceback()) context['exc_info'] = exc_info del exc_info # prevent cycles oops-twisted-0.0.6/oops_twisted/log.py0000644000175000017500000000623411670323334020303 0ustar roaksoaxroaksoax# Copyright (c) 2010, 2011, Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 only. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . # GNU Lesser General Public License version 3 (see the file LICENSE). """twisted.log observer to create OOPSes for failures.""" __metaclass__ = type import datetime from pytz import utc from twisted.python.log import ( ILogObserver, textFromEventDict, ) from zope.interface import implements __all__ = [ 'OOPSObserver', ] class OOPSObserver: """Convert twisted log events to OOPSes if they are failures.""" implements(ILogObserver) def __init__(self, config, fallback=None): """"Create an OOPSObserver. :param config: An oops_twisted.Config. :param fallback: If supplied, a callable to pass non-failure log events to, and to inform (as non-failures) when an OOPS has occurred. """ self.config = config assert fallback is None or callable(fallback) self.fallback = fallback def emit(self, eventDict): """Handle a twisted log entry. Note that you should generally pass the actual observer to twisted functions, as the observer instance 'implements' ILogObserver, but the emit method does not (and cannot). :return: For testing convenience returns the oops report and a deferred which fires after all publication and fallback forwarding has occured, though the twisted logging protocol does not need (or examine) the return value. """ if not eventDict.get('isError'): if self.fallback: self.fallback(eventDict) return None, None context = {} context['twisted_failure'] = eventDict.get('failure') report = self.config.create(context) report['time'] = datetime.datetime.fromtimestamp( eventDict['time'], utc) report['tb_text'] = textFromEventDict(eventDict) d = self.config.publish(report) if self.fallback: d.addCallback(self._fallback_report, report, dict(eventDict)) return report, d __call__ = emit def _fallback_report(self, ids, report, event): # If ids is empty, no publication occured so there is no id to forward: # don't alter the event at all in this case. if ids: event['isError'] = False event.pop('failure', None) event['message'] = ["Logged OOPS id %s: %s: %s" % ( report['id'], report.get('type', 'No exception type'), report.get('value', 'No exception value'))] if ids is not None: self.fallback(event) oops-twisted-0.0.6/oops_twisted/__init__.py0000644000175000017500000001352611670323420021257 0ustar roaksoaxroaksoax# # Copyright (c) 2011, Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 only. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . # GNU Lesser General Public License version 3 (see the file LICENSE). """OOPS error reports for Twisted. The oops_twisted package provides integration glue between logged twisted errors (via the twisted.log api) and the oops error reporting system (http://pypi.python.org/pypi/oops). Dependencies ============ * Python 2.6+ * oops (http://pypi.python.org/pypi/oops) * Twisted Testing Dependencies ==================== * subunit (http://pypi.python.org/pypi/python-subunit) (optional) * testtools (http://pypi.python.org/pypi/testtools) Usage ===== OOPS Configuration ++++++++++++++++++ * Setup your configuration:: >>> from oops_twisted import Config >>> config = Config() Note that you will probably want at least one publisher, or your reports will be silently discarded. * When adding publishers, either wrap 'normal' OOPS publishers in deferToThread or similar, or use native Twisted publishers. For instance:: >>> from functools import partial >>> config.publishers.append(partial(deferToThread, blocking_publisher)) A helper 'defer_publisher' is supplied to do this for your convenience. Catching log.err calls ++++++++++++++++++++++ * create an OOPS log observer:: >>> from oops_twisted import OOPSObserver >>> observer = OOPSObserver(config) * And enable it:: >>> from twisted.log import addObserver >>> addObserver(observer) * This is typically used to supplement regular logging, e.g. you might initialize normal logging to a file first:: >>> twisted.log.startLogging(logfile) The OOPSObserver will discard all non-error log messages, and convert error log messages into OOPSes using the oops config. Optionally, you can provide OOPSObserver with a second observer to delegate too. Any event that is not converted into an OOPS is passed through unaltered. Events that are converted to OOPSes have a new event second to the second observer which provides the OOPS id and the failure name and value:: >>> observer = OOPSObserver(config, twisted.log.PythonLoggingObserver().emit) If the OOPS config has no publishers, the fallback will receive unaltered failure events (this stops them getting lost). If there are publishers but the OOPS is filtered, the fallback will not be invoked at all (this permits e.g. rate limiting of failutes via filters). Creating OOPSes without using log.err +++++++++++++++++++++++++++++++++++++ You can directly create an OOPS if you have a twisted failure object:: >>> from twisted.python.failure import Failure >>> report = config.create(dict(twisted_failure=Failure())) >>> config.publish(report) Extending WSGI ++++++++++++++ oops_twisted supports an extended WSGI contract where if the returned iterator for the body implements t.w.i.IBodyProducer, then the iterator that oops_twisted's WSGI wrapper returns will also implement IBodyProducer. This is useful with a customised Twisted WSGI resource that runs IBodyProducer iterators in the IO loop, rather than using up a threadpool thread. To use this pass tracker=oops_twisted.wsgi.body_producer_tracker when calling oops_wsgi.make_app. Note that a non-twisted OOPS Config is assumed because the WSGI protocol is synchronous: be sure to provide the oops_wsgi make_app with a non-twisted OOPS Config. If you are publishing with native OOPS publishers you may want to write a small synchronous publish-to-an-internal queue as you cannot use t.i.t.blockingCallFromThread: the WSGI protocol permits start_response to be called quite late, which may happen after an IBodyProducer has been returned to the WSGI gateway and all further code will be executing in the reactor thread. Specifically the call to startProducing may trigger start_response being called before the first write() occurs; and the call to start_response may trigger an OOPS being published if: - it contains an exc_info value - it has a status code matching the oops-on-status code values - (in future) the response takes too long to start flowing Another route to exceptions is in startProducing itself, which may error, with similar consequences in that the oops config must be called into, and it has to be compatible with the config for start_response handling. If there is a need to address this, oops_twisted could take responsibility for exception handling in the IBodyProducer code path, with the cost of needing a second OOPS config - a native Twisted one. """ # same format as sys.version_info: "A tuple containing the five components of # the version number: major, minor, micro, releaselevel, and serial. All # values except releaselevel are integers; the release level is 'alpha', # 'beta', 'candidate', or 'final'. The version_info value corresponding to the # Python version 2.0 is (2, 0, 0, 'final', 0)." Additionally we use a # releaselevel of 'dev' for unreleased under-development code. # # If the releaselevel is 'alpha' then the major/minor/micro components are not # established at this point, and setup.py will use a version of next-$(revno). # If the releaselevel is 'final', then the tarball will be major.minor.micro. # Otherwise it is major.minor.micro~$(revno). __version__ = (0, 0, 6, 'beta', 0) __all__ = [ 'Config', 'defer_publisher', 'OOPSObserver', ] from oops_twisted.config import ( Config, defer_publisher, ) from oops_twisted.log import OOPSObserver oops-twisted-0.0.6/oops_twisted/wsgi.py0000644000175000017500000001250711670323103020465 0ustar roaksoaxroaksoax# Copyright (c) 2011, Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 only. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . # GNU Lesser General Public License version 3 (see the file LICENSE). """Extended WSGI support for Twisted.""" from oops_wsgi.middleware import generator_tracker from twisted.internet.defer import Deferred from twisted.web.iweb import IBodyProducer from zope.interface import implements __all__ = [ 'body_producer_tracker', ] def body_producer_tracker(on_first_bytes, on_finish, on_error, app_body): """A wrapper for IBodyProducer that calls the OOPS hooks as needed. :seealso: generator_tracker which defines the contract for this factory function. :param on_first_bytes: Called as on_first_bytes() when the first bytes from the app body are available but before they are delivered. :param on_finish: Called as on_finish() when the app body is fully consumed. :param on_error: Called as on_error(sys.exc_info()) if a handleable error has occured while consuming the generator. Errors like GeneratorExit are not handleable. The return value from this is written to the consumer. :param app_body: The iterable body for the WSGI app. This may be a simple list or a generator or an IBodyProducer. If it is not an IBodyProducer then oops_wsgi.middleware.generator_tracker will be used. """ if not IBodyProducer.providedBy(app_body): return generator_tracker( on_first_bytes, on_finish, on_error, app_body) return ProducerWrapper(on_first_bytes, on_finish, on_error, app_body) class ProducerWrapper: """Wrap an IBodyProducer and call callbacks at key points. :seealso: body_producer_tracker - the main user of ProducerWrapper. """ implements(IBodyProducer) def __init__(self, on_first_bytes, on_finish, on_error, app_body): self.on_first_bytes = on_first_bytes self.on_finish = on_finish self.on_error = on_error self.app_body = app_body # deferred returned from startProducing. If None the producer has # finished its work (or never been started). self.result = None # We wrap the consumer as well so that we can track the first write. self.consumer = None # Error page from OOPS self.error_data = None @property def length(self): if self.error_data is not None: return len(self.error_data) return self.app_body.length def startProducing(self, consumer): self.consumer = consumer self.written = False # Track production state self.paused = False # Error page from OOPS self.error_data = None # The deferred we return. Because we eat errors when an OOPS error page # is generated, we cannot return the underlying producers deferred. result = Deferred() self.result = result # Tell the wrapped producer to write to us. wrapped_result = self.app_body.startProducing(self) # We need a callback at the end to fire on_finish. wrapped_result.addCallback(self.wrapped_finished) # If an exception happens, we want to fire on_error. wrapped_result.addErrback(self.wrapped_failure) return result def stopProducing(self): # The deferred from startProducing must not fire. self.result = None self.app_body.stopProducing() def pauseProducing(self): self.paused = True self.app_body.pauseProducing() def resumeProducing(self): if self.error_data is not None: self.consumer.write(self.error_data) result = self.result self.result = None result.callback(None) else: self.app_body.resumeProducing() def write(self, data): if not self.written: self.written = True self.on_first_bytes() self.consumer.write(data) def wrapped_finished(self, ignored): result = self.result self.result = None try: self.on_finish() except: result.errback() else: result.callback(None) def wrapped_failure(self, failure): try: exc_info = ( failure.type, failure.value, failure.getTracebackObject()) self.error_data = self.on_error(exc_info) except: result = self.result self.result = None result.errback() else: if not self.paused: self.consumer.write(self.error_data) result = self.result self.result = None result.callback(None) # Received an error from the producer while it was paused, and OOPS # generated an error page. This will be buffered until we are unpaused. oops-twisted-0.0.6/oops_twisted/config.py0000644000175000017500000000547711670323103020771 0ustar roaksoaxroaksoax# Copyright (c) 2010, 2011, Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 only. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . # GNU Lesser General Public License version 3 (see the file LICENSE). """OOPS config and publishing for Twisted. This module acts as an adapter for the oops.Config module - see `pydoc oops.Config` for the primary documentation. The only change is that Config.publish works with deferreds. A helper function defer_publisher is supplied which can wrap a non-deferred publisher. """ from functools import partial from twisted.internet import defer from twisted.internet.threads import deferToThread import oops from createhooks import failure_to_context __all__ = [ 'Config', 'defer_publisher', ] class Config(oops.Config): """Twisted version of oops.Config. The only difference is that the publish method, which could block now expects each publisher to return a deferred. For more information see the oops.Config documentation. """ def __init__(self, *args, **kwargs): oops.Config.__init__(self) self.on_create.insert(0, failure_to_context) def publish(self, report): """Publish a report. The report is first synchronously run past self.filters, then fired asynchronously at all of self.publishers. See `pydoc oops.Config.publish` for more documentation. :return: a twisted.internet.defer.Deferred. On success this deferred will return the list of oops ids allocated by the publishers (a direct translation of the oops.Config.publish result). """ for report_filter in self.filters: if report_filter(report): return defer.succeed(None) if not self.publishers: return defer.succeed([]) d = self.publishers[0](report) result = [] def stash_id(id): report['id'] = id result.append(id) return report d.addCallback(stash_id) for publisher in self.publishers[1:]: d.addCallback(publisher) d.addCallback(stash_id) def return_result(_): return result d.addCallback(return_result) return d def defer_publisher(publisher): """Wrap publisher in deferToThread.""" return partial(deferToThread, publisher) oops-twisted-0.0.6/setup.py0000755000175000017500000000342611670323427016145 0ustar roaksoaxroaksoax#!/usr/bin/env python # # Copyright (c) 2011, Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 only. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . # GNU Lesser General Public License version 3 (see the file LICENSE). from distutils.core import setup import os.path description = file( os.path.join(os.path.dirname(__file__), 'README'), 'rb').read() setup(name="oops_twisted", version="0.0.6", description=\ "Translate twisted error logs into OOPS error reports.", long_description=description, maintainer="Launchpad Developers", maintainer_email="launchpad-dev@lists.launchpad.net", url="https://launchpad.net/python-oops-twisted", packages=['oops_twisted'], package_dir = {'':'.'}, classifiers = [ 'Development Status :: 2 - Pre-Alpha', 'Intended Audience :: Developers', 'License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)', 'Operating System :: OS Independent', 'Programming Language :: Python', ], install_requires = [ 'oops', 'oops_wsgi', 'pytz', 'Twisted', ], extras_require = dict( test=[ 'testtools', ] ), ) oops-twisted-0.0.6/README0000644000175000017500000001012211670323334015274 0ustar roaksoaxroaksoax********************************************************** python-oops-twisted: Error report integration with twisted ********************************************************** Copyright (c) 2011, Canonical Ltd This program is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, version 3 only. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this program. If not, see . GNU Lesser General Public License version 3 (see the file LICENSE). The oops_twisted package provides integration glue between logged twisted errors (via the twisted.log api) and the oops error reporting system (http://pypi.python.org/pypi/oops). Dependencies ============ * Python 2.6+ * oops (http://pypi.python.org/pypi/oops) * Twisted Testing Dependencies ==================== * subunit (http://pypi.python.org/pypi/python-subunit) (optional) * testtools (http://pypi.python.org/pypi/testtools) Usage ===== OOPS Configuration ++++++++++++++++++ * Setup your configuration:: >>> from oops_twisted import Config >>> config = Config() Note that you will probably want at least one publisher, or your reports will be silently discarded. * When adding publishers, either wrap 'normal' OOPS publishers in deferToThread or similar, or use native Twisted publishers. For instance:: >>> from functools import partial >>> config.publishers.append(partial(deferToThread, blocking_publisher)) A helper 'defer_publisher' is supplied to do this for your convenience. Catching log.err calls ++++++++++++++++++++++ * create an OOPS log observer:: >>> from oops_twisted import OOPSObserver >>> observer = OOPSObserver(config) * And enable it:: >>> from twisted.log import addObserver >>> addObserver(observer) * This is typically used to supplement regular logging, e.g. you might initialize normal logging to a file first:: >>> twisted.log.startLogging(logfile) The OOPSObserver will discard all non-error log messages, and convert error log messages into OOPSes using the oops config. Optionally, you can provide OOPSObserver with a second observer to delegate too. Any event that is not converted into an OOPS is passed through unaltered. Events that are converted to OOPSes have a new event second to the second observer which provides the OOPS id and the failure name and value:: >>> observer = OOPSObserver(config, twisted.log.PythonLoggingObserver().emit) Extending WSGI ++++++++++++++ oops_twisted supports an extended WSGI contract where if the returned iterator for the body implements t.w.i.IBodyProducer, then the iterator that oops_twisted's WSGI wrapper returns will also implement IBodyProducer. This is useful with a customised Twisted WSGI resource that runs IBodyProducer iterators in the IO loop, rather than using up a threadpool thread. To use this pass tracker=oops_twisted.wsgi.body_producer_tracker when calling oops_wsgi.make_app. Note that a non-twisted OOPS Config is assumed because the WSGI protocol is synchronous: be sure to provide the oops_wsgi make_app with a non-twisted OOPS Config. For more information see pydoc oops_twisted. Installation ============ Either run setup.py in an environment with all the dependencies available, or add the working directory to your PYTHONPATH. Development =========== Upstream development takes place at https://launchpad.net/python-oops-twisted. To setup a working area for development, if the dependencies are not immediately available, you can use ./bootstrap.py to create bin/buildout, then bin/py to get a python interpreter with the dependencies available. To run the tests use the runner of your choice, the test suite is oops_twisted.tests.test_suite. For instance:: $ bin/py -m testtools.run oops_twisted.tests.test_suite oops-twisted-0.0.6/PKG-INFO0000644000175000017500000001315511670323604015522 0ustar roaksoaxroaksoaxMetadata-Version: 1.0 Name: oops_twisted Version: 0.0.6 Summary: Translate twisted error logs into OOPS error reports. Home-page: https://launchpad.net/python-oops-twisted Author: Launchpad Developers Author-email: launchpad-dev@lists.launchpad.net License: UNKNOWN Description: ********************************************************** python-oops-twisted: Error report integration with twisted ********************************************************** Copyright (c) 2011, Canonical Ltd This program is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, version 3 only. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this program. If not, see . GNU Lesser General Public License version 3 (see the file LICENSE). The oops_twisted package provides integration glue between logged twisted errors (via the twisted.log api) and the oops error reporting system (http://pypi.python.org/pypi/oops). Dependencies ============ * Python 2.6+ * oops (http://pypi.python.org/pypi/oops) * Twisted Testing Dependencies ==================== * subunit (http://pypi.python.org/pypi/python-subunit) (optional) * testtools (http://pypi.python.org/pypi/testtools) Usage ===== OOPS Configuration ++++++++++++++++++ * Setup your configuration:: >>> from oops_twisted import Config >>> config = Config() Note that you will probably want at least one publisher, or your reports will be silently discarded. * When adding publishers, either wrap 'normal' OOPS publishers in deferToThread or similar, or use native Twisted publishers. For instance:: >>> from functools import partial >>> config.publishers.append(partial(deferToThread, blocking_publisher)) A helper 'defer_publisher' is supplied to do this for your convenience. Catching log.err calls ++++++++++++++++++++++ * create an OOPS log observer:: >>> from oops_twisted import OOPSObserver >>> observer = OOPSObserver(config) * And enable it:: >>> from twisted.log import addObserver >>> addObserver(observer) * This is typically used to supplement regular logging, e.g. you might initialize normal logging to a file first:: >>> twisted.log.startLogging(logfile) The OOPSObserver will discard all non-error log messages, and convert error log messages into OOPSes using the oops config. Optionally, you can provide OOPSObserver with a second observer to delegate too. Any event that is not converted into an OOPS is passed through unaltered. Events that are converted to OOPSes have a new event second to the second observer which provides the OOPS id and the failure name and value:: >>> observer = OOPSObserver(config, twisted.log.PythonLoggingObserver().emit) Extending WSGI ++++++++++++++ oops_twisted supports an extended WSGI contract where if the returned iterator for the body implements t.w.i.IBodyProducer, then the iterator that oops_twisted's WSGI wrapper returns will also implement IBodyProducer. This is useful with a customised Twisted WSGI resource that runs IBodyProducer iterators in the IO loop, rather than using up a threadpool thread. To use this pass tracker=oops_twisted.wsgi.body_producer_tracker when calling oops_wsgi.make_app. Note that a non-twisted OOPS Config is assumed because the WSGI protocol is synchronous: be sure to provide the oops_wsgi make_app with a non-twisted OOPS Config. For more information see pydoc oops_twisted. Installation ============ Either run setup.py in an environment with all the dependencies available, or add the working directory to your PYTHONPATH. Development =========== Upstream development takes place at https://launchpad.net/python-oops-twisted. To setup a working area for development, if the dependencies are not immediately available, you can use ./bootstrap.py to create bin/buildout, then bin/py to get a python interpreter with the dependencies available. To run the tests use the runner of your choice, the test suite is oops_twisted.tests.test_suite. For instance:: $ bin/py -m testtools.run oops_twisted.tests.test_suite Platform: UNKNOWN Classifier: Development Status :: 2 - Pre-Alpha Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL) Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python