oops-0.0.14/0000755000175000017500000000000013272572665014275 5ustar cjwatsoncjwatson00000000000000oops-0.0.14/PKG-INFO0000644000175000017500000001463713272572665015405 0ustar cjwatsoncjwatson00000000000000Metadata-Version: 1.1 Name: oops Version: 0.0.14 Summary: OOPS report model and default allocation/[de]serialization. Home-page: https://launchpad.net/python-oops Author: Launchpad Developers Author-email: launchpad-dev@lists.launchpad.net License: UNKNOWN Description: ************************** python-oops: Error reports ************************** Copyright (c) 2011, Canonical Ltd This program is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, version 3 only. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this program. If not, see . GNU Lesser General Public License version 3 (see the file LICENSE). The oops project provides an in-memory model and basic serialisation, deserialisation and allocation of OOPS reports. An OOPS report is a report about something going wrong in a piece of software... thus, an 'oops' :) This core package is rarely used directly: most programs or services that want to generate OOPS reports will do so via a framework specific adapter (e.g. python-oops_wsgi). Dependencies ============ * Python 2.6+ or 3.3+ * pytz (http://pypi.python.org/pypi/pytz/) Testing Dependencies ==================== * subunit (http://pypi.python.org/pypi/python-subunit) (optional) * testtools (http://pypi.python.org/pypi/testtools) Usage ===== In Python, OOPS reports are dicts with some well known keys, but extensible simply by adding as many additional keys asneeded. The only constraint is that the resulting dict must be bson serializable : this is the standard to which new serializers are held. Some existing serializers cannot handle this degree of extensability and will ignore additional keys, and/or raise an error on keys they expect but which contain unexpected data. Typical usage: * When initializing your script/app/server, create a Config object:: >>> from oops import Config >>> config = Config() * New reports will be based on the template report:: >>> config.template {} * You can edit the template report (which like all reports is just a dict):: >>> config.template['branch_nick'] = 'mybranch' >>> config.template['appname'] = 'demo' * You can supply a callback (for instance, to capture your process memory usage when the oops is created, or to override / tweak the information gathered by an earlier callback):: >>> mycallback = lambda report, context: None >>> config.on_create.append(mycallback) The context parameter is also just dict, and is passed to all the on_create callbacks similarly to the report. This is used to support passing information to the on_create hooks. For instance, the exc_info key is used to pass in information about the exception being logged (if one was caught). * Later on, when something has gone wrong and you want to create an OOPS report:: >>> report = config.create(context=dict(exc_info=sys.exc_info())) >>> report {'appname': 'demo', 'branch_nick': 'mybranch'} * And then send it off for storage:: >>> config.publish(report) [] * Note that publish returns a list - each item in the list is the id allocated by the particular repository that recieved the report. (Id allocation is up to the repository). Publishers should try to use report['id'] for the id, if it is set. This is automatically set to the id returned by the previous publisher. If publish returns None, then the report was filtered and not passed to any publisher (see the api docs for more information). >>> 'id' in report False >>> def demo_publish(report): ... return ['id 1'] >>> config.publisher = demo_publish >>> config.publish(report) ['id 1'] >>> report['id'] 'id 1' * The pprint_to_stream publisher will print out reports to a stream after pprinting them. This can be very helpful for interactive use. * The related project oops_datedir_repo contains a local disk based repository which can be used as a publisher. More coming soon. Installation ============ Either run setup.py in an environment with all the dependencies available, or add the working directory to your PYTHONPATH. Development =========== Upstream development takes place at https://launchpad.net/python-oops. To setup a working area for development, if the dependencies are not immediately available, you can use ./bootstrap.py to create bin/buildout, then bin/py to get a python interpreter with the dependencies available. To run the tests use the runner of your choice, the test suite is oops.tests.test_suite. For instance:: $ bin/py -m testtools.run oops.tests.test_suite Alternative you can use the testr command from testrepository:: $ testr run Platform: UNKNOWN Classifier: Development Status :: 2 - Pre-Alpha Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL) Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 3 oops-0.0.14/setup.py0000755000175000017500000000352313237011463015776 0ustar cjwatsoncjwatson00000000000000#!/usr/bin/env python # # Copyright (c) 2011, Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 only. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . # GNU Lesser General Public License version 3 (see the file LICENSE). from distutils.core import setup import os.path with open(os.path.join(os.path.dirname(__file__), 'README')) as f: description = f.read() setup(name="oops", version="0.0.14", description=\ "OOPS report model and default allocation/[de]serialization.", long_description=description, maintainer="Launchpad Developers", maintainer_email="launchpad-dev@lists.launchpad.net", url="https://launchpad.net/python-oops", packages=['oops'], package_dir = {'':'.'}, classifiers = [ 'Development Status :: 2 - Pre-Alpha', 'Intended Audience :: Developers', 'License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 3', ], install_requires = [ 'pytz', 'six', ], extras_require = dict( test=[ 'fixtures', 'testtools', ] ), ) oops-0.0.14/oops/0000755000175000017500000000000013272572665015255 5ustar cjwatsoncjwatson00000000000000oops-0.0.14/oops/createhooks.py0000644000175000017500000000744613237011463020132 0ustar cjwatsoncjwatson00000000000000# Copyright (c) 2010, 2011, Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 only. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . # GNU Lesser General Public License version 3 (see the file LICENSE). """Various hooks that can be used to populate OOPS reports. The default_hooks list contains some innocuous hooks which most reporters will want. """ from __future__ import absolute_import, print_function __all__ = [ 'attach_exc_info', 'attach_date', 'attach_hostname', 'copy_reporter', 'copy_topic', 'copy_url', 'default_hooks', 'safe_unicode', ] __metaclass__ = type import datetime import socket import traceback from pytz import utc import six # Used to detect missing keys. _sentinel = object() def _simple_copy(key): """Curry a simple hook that copies a key from context to report.""" def copy_key(report, context): value = context.get(key, _sentinel) if value is not _sentinel: report[key] = value copy_key.__doc__ = ( "Copy the %s field from context to report, if present." % key) return copy_key copy_reporter = _simple_copy('reporter') copy_topic = _simple_copy('topic') copy_url = _simple_copy('url') def safe_unicode(obj): """Used to reliably get *a* string for an object. This is called on objects like exceptions, where bson won't be able to serialize it, but a representation is needed for the report. It is exposed a convenience for other on_create hook authors. """ if isinstance(obj, six.text_type): return obj # A call to str(obj) could raise anything at all. # We'll ignore these errors, and print something # useful instead, but also log the error. # We disable the pylint warning for the blank except. try: value = six.text_type(obj) except: value = u'' % ( six.text_type(type(obj).__name__)) # Some objects give back bytestrings to __unicode__... if isinstance(value, six.binary_type): value = value.decode('latin-1') return value def attach_date(report, context): """Set the time key in report to a datetime of now.""" report['time'] = datetime.datetime.now(utc) def attach_exc_info(report, context): """Attach exception info to the report. This reads the 'exc_info' key from the context and sets the: * type * value * tb_text keys in the report. exc_info must be a tuple, but it can contain either live exception information or simple strings (allowing exceptions that have been serialised and received over the network to be reported). """ info = context.get('exc_info') if info is None: return report['type'] = getattr(info[0], '__name__', info[0]) report['value'] = safe_unicode(info[1]) if isinstance(info[2], six.string_types): tb_text = info[2] else: tb_text = u''.join(map(safe_unicode, traceback.format_tb(info[2]))) report['tb_text'] = tb_text def attach_hostname(report, context): """Add the machine's hostname to report in the 'hostname' key.""" report['hostname'] = socket.gethostname() # hooks that are installed into Config objects by default. default_hooks = [ attach_exc_info, attach_date, copy_reporter, copy_topic, copy_url, attach_hostname, ] oops-0.0.14/oops/config.py0000644000175000017500000002242313237011463017060 0ustar cjwatsoncjwatson00000000000000# # Copyright (c) 2010, 2011, Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 only. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . # GNU Lesser General Public License version 3 (see the file LICENSE). """The primary interface for clients creating OOPS reports. Typical usage: * Configure the library:: >>> from oops import Config >>> config = Config() >>> def demo_publish(report): ... return 'id 1' >>> config.publisher = demo_publish This allows aggregation of oops reports from different programs into one oops-tools install. >>> config.template['reporter'] = 'myprogram' * Create a report:: >>> report = config.create() * And then send it off for storage:: >>> config.publish(report) ['id 1'] >>> report {'id': 'id 1', 'template': 'myprogram'} * See the Config object pydoc for more information. The OOPS report is a dictionary, and must be bson serializable. This permits the inclusion of binary data in the report, and provides cross-language compatibility. A minimal report can be empty, but this is fairly useless and may even be rejected by some repositories. Some well known keys used by Launchpad in its OOPS reports:: * id: The name of this error report. * type: The type of the exception that occurred. * value: The value of the exception that occurred. * time: The time at which the exception occurred. * hostname: The hostname of the machine the oops was created on. (Set by default) * branch_nick: The branch nickname. * revno: The revision number of the branch. * tb_text: A text version of the traceback. * username: The user associated with the request. * url: The URL for the failed request. * req_vars: The request variables. This should be a dict of simple string -> string mappings. The strings and their values should be either unicode or url escaped ascii bytestrings. Older versions of the oops toolchain emitted this variable as a list of two-tuples e.g. (key, value). Code expecting to receive or process old reports should accept both dicts and tuples. Modern or new code can just expect a dict. * branch_nick: A name for the branch of code that was running when the report was triggered. * revno: The revision that the branch was at. * reporter: Describes the program creating the report. For instance you might put the name of the program, or its website - as long as its distinct from other reporters being sent to a single analysis server. For dynamically scaled services with multiple instances, the reporter will usually be the same for a single set of identical instances. e.g. all the instances in one Amazon EC2 availability zone might be given the same reporter. Differentiated backend services for the same front end site would usually get different reporters as well. (e.g. auth, cache, render, ...) * topic: The subject or context for the report. With a command line tool you might put the subcommand here, with a web site you might put the template (as opposed to the url). This is used as a weak correlation hint: reports from the same topic are more likely to have the same cause than reports from different topics. * timeline: A sequence of (start, stop, category, detail) tuples describing the events leading up to the OOPS. One way to populate this is the oops-timeline package. Consumers should not assume the length of the tuple to be fixed - additional fields may be added in future to the right hand side (e.g. backtraces). """ from __future__ import absolute_import, print_function __all__ = [ 'Config', ] __metaclass__ = type from copy import deepcopy import warnings from oops.createhooks import default_hooks from oops.publishers import ( convert_result_to_list, publish_to_many, ) class Config: """The configuration for the OOPS system. :ivar on_create: A list of callables to call when making a new report. Each will be called in series with the new report and a creation context dict. The return value of the callbacks is ignored. :ivar filters: A list of callables to call when filtering a report. Each will be called in series with a report that is about to be published. If the filter returns true (that is not None, 0, '' or False), then the report will not be published, and the call to publish will return None to the user. :ivar publisher: A callable to call when publishing a report. It will be called in series with the report to publish. It is expected to return a list of ids. See the publish() method for more information. :ivar publishers: A list of callables to call when publishing a report. Each will be called in series with the report to publish. Their return value will be assigned to the reports 'id' key : if a publisher allocates a different id than a prior publisher, only the last publisher in the list will have its id present in the report at the end. See the publish() method for more information. This attribute is deprecated, Use the `publisher` attribute instead, and see `oops.publishers.publish_to_many` if you want to publish to multiple publishers. """ def __init__(self): self.filters = [] self.on_create = list(default_hooks) self.template = {} self.publisher = None self.publishers = [] def create(self, context=None): """Create an OOPS. The current template is copied to make the new report, and the new report is then passed to all the on_create callbacks for population. If a callback raises an exception, that will propgate to the caller. :param context: A dict of information that the on_create callbacks can use in populating the report. For instance, the attach_exception callback looks for an exc_info key in the context and uses that to add information to the report. If context is None, an empty dict is created and used with the callbacks. :return: A fresh OOPS. """ if context is None: context = {} result = deepcopy(self.template) [callback(result, context) for callback in self.on_create] return result def publish(self, report): """Publish a report. The report will be passed through any filters, and then handed off to the callable assigned to the `publisher` instance variable, if any, and then passed through any publishers in the `publishers` list instance variable. The return value will be the list returned by `publisher` method, with the ids returned by the `publishers` appended. The `publishers` list is deprecated, and the `publisher` attribute should be used instead, with `oops.publishers.publish_to_many` used if needed. The `publisher` should return a list of ids that were allocated-or-used for the report. The `publishers` should each return any id that was allocated-or-used for the report. The return value of the callables in the `publishers` list will be assigned to the `id` key of the report. If a publisher in the `publishers` list returns anything non-True (that is None, 0, False, ''), it indicates that the publisher did not publish the report. The last entry in the list of ids, if any, will be assigned to the 'id' key of the report before returning, so that clients that only care to deal with one id don't have to pick one themselves. The whole list of ids will be returned to the caller to allow them to handle the case where multiple ids were used for a report. If any publisher raises an exception, that will propagate to the caller. :return: A list of the allocated ids. """ for report_filter in self.filters: if report_filter(report): return None # XXX: james_w 2012-06-19 bug=1015293: Deprecated code path, # this should be removed once users have had a chance # to migrate. The constructor and docstrings should # also be cleaned up at the same time. if self.publishers: warnings.warn( "Using the oops.Config.publishers attribute is " "deprecated. Use the oops.Config.publisher attribute " "instead, with an oops.publishers.publish_to_many object " "if multiple publishers are needed", DeprecationWarning, stacklevel=2) old_publishers = map(convert_result_to_list, self.publishers) if self.publisher: publisher = publish_to_many(self.publisher, *old_publishers) else: publisher = publish_to_many(*old_publishers) ret = publisher(report) if ret: report['id'] = ret[-1] return ret oops-0.0.14/oops/publishers.py0000644000175000017500000001040713237011463017772 0ustar cjwatsoncjwatson00000000000000# Copyright (c) 2011, Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 only. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . # GNU Lesser General Public License version 3 (see the file LICENSE). """Generic publisher support and utility code.""" from __future__ import absolute_import, print_function __metaclass__ = type __all__ = [ 'pprint_to_stream', 'publish_with_fallback', 'publish_to_many', ] from hashlib import md5 from pprint import pformat def pprint_to_stream(stream): """Pretty print reports to text stream. Reports will be given an id by hashing the report if none is present. """ def pprinter(report): report = dict(report) output = pformat(report) if not report.get('id'): report['id'] = md5(output.encode('UTF-8')).hexdigest() output = pformat(report) stream.write(output) stream.write('\n') stream.flush() return [report['id']] return pprinter def publish_new_only(publisher): """Wraps a publisher with a check that the report has not had an id set. This permits having fallback publishers that only publish if the earlier one failed. For instance: >>> config.publishers.append(amqp_publisher) >>> config.publishers.append(publish_new_only(datedir_repo.publish)) This function is deprecated. Instead please use publish_with_fallback. """ def result(report): if report.get('id'): return None return publisher(report) return result def publish_with_fallback(*publishers): """A publisher to fallback publishing through a list of publishers This is a publisher, see Config.publish for the calling and return conventions. This publisher delegates to the supplied publishers by calling them all until one reports that it has published the report, and aggregates the results. :param *publishers: a list of callables to publish oopses to. :return: a callable that will publish a report to each of the publishers when called. """ def result(report): ret = [] for publisher in publishers: ret.extend(publisher(report)) if ret: break return ret return result def publish_to_many(*publishers): """A fan-out publisher of oops reports. This is a publisher, see Config.publish for the calling and return conventions. This publisher delegates to the supplied publishers by calling them all, and aggregates the results. If a publisher returns a non-emtpy list (indicating that the report was published) then the last item of this list will be set as the 'id' key in the report before the report is passed to the next publisher. This makes it possible for publishers later in the chain to re-use the id. :param *publishers: a list of callables to publish oopses to. :return: a callable that will publish a report to each of the publishers when called. """ def result(report): ret = [] for publisher in publishers: if ret: report['id'] = ret[-1] ret.extend(publisher(report)) return ret return result def convert_result_to_list(publisher): """Ensure that a publisher returns a list. The old protocol for publisher callables was to return an id, or a False value if the report was not published. The new protocol is to return a list, which is empty if the report was not published. This function coverts a publisher using the old protocol in to one that uses the new protocol, translating values as needed. """ def publish(report): ret = publisher(report) if ret: return [ret] else: return [] return publish oops-0.0.14/oops/__init__.py0000644000175000017500000000343113272572337017363 0ustar cjwatsoncjwatson00000000000000# # Copyright (c) 2011, Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, version 3 only. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . # GNU Lesser General Public License version 3 (see the file LICENSE). # same format as sys.version_info: "A tuple containing the five components of # the version number: major, minor, micro, releaselevel, and serial. All # values except releaselevel are integers; the release level is 'alpha', # 'beta', 'candidate', or 'final'. The version_info value corresponding to the # Python version 2.0 is (2, 0, 0, 'final', 0)." Additionally we use a # releaselevel of 'dev' for unreleased under-development code. # # If the releaselevel is 'alpha' then the major/minor/micro components are not # established at this point, and setup.py will use a version of next-$(revno). # If the releaselevel is 'final', then the tarball will be major.minor.micro. # Otherwise it is major.minor.micro~$(revno). __version__ = (0, 0, 14, 'final', 0) __all__ = [ 'Config', 'convert_result_to_list', 'pprint_to_stream', 'publish_new_only', 'publish_to_many', 'publish_with_fallback', ] from oops.config import Config from oops.publishers import ( convert_result_to_list, pprint_to_stream, publish_new_only, publish_to_many, publish_with_fallback, ) oops-0.0.14/README0000644000175000017500000001123513237011463015140 0ustar cjwatsoncjwatson00000000000000************************** python-oops: Error reports ************************** Copyright (c) 2011, Canonical Ltd This program is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, version 3 only. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this program. If not, see . GNU Lesser General Public License version 3 (see the file LICENSE). The oops project provides an in-memory model and basic serialisation, deserialisation and allocation of OOPS reports. An OOPS report is a report about something going wrong in a piece of software... thus, an 'oops' :) This core package is rarely used directly: most programs or services that want to generate OOPS reports will do so via a framework specific adapter (e.g. python-oops_wsgi). Dependencies ============ * Python 2.6+ or 3.3+ * pytz (http://pypi.python.org/pypi/pytz/) Testing Dependencies ==================== * subunit (http://pypi.python.org/pypi/python-subunit) (optional) * testtools (http://pypi.python.org/pypi/testtools) Usage ===== In Python, OOPS reports are dicts with some well known keys, but extensible simply by adding as many additional keys asneeded. The only constraint is that the resulting dict must be bson serializable : this is the standard to which new serializers are held. Some existing serializers cannot handle this degree of extensability and will ignore additional keys, and/or raise an error on keys they expect but which contain unexpected data. Typical usage: * When initializing your script/app/server, create a Config object:: >>> from oops import Config >>> config = Config() * New reports will be based on the template report:: >>> config.template {} * You can edit the template report (which like all reports is just a dict):: >>> config.template['branch_nick'] = 'mybranch' >>> config.template['appname'] = 'demo' * You can supply a callback (for instance, to capture your process memory usage when the oops is created, or to override / tweak the information gathered by an earlier callback):: >>> mycallback = lambda report, context: None >>> config.on_create.append(mycallback) The context parameter is also just dict, and is passed to all the on_create callbacks similarly to the report. This is used to support passing information to the on_create hooks. For instance, the exc_info key is used to pass in information about the exception being logged (if one was caught). * Later on, when something has gone wrong and you want to create an OOPS report:: >>> report = config.create(context=dict(exc_info=sys.exc_info())) >>> report {'appname': 'demo', 'branch_nick': 'mybranch'} * And then send it off for storage:: >>> config.publish(report) [] * Note that publish returns a list - each item in the list is the id allocated by the particular repository that recieved the report. (Id allocation is up to the repository). Publishers should try to use report['id'] for the id, if it is set. This is automatically set to the id returned by the previous publisher. If publish returns None, then the report was filtered and not passed to any publisher (see the api docs for more information). >>> 'id' in report False >>> def demo_publish(report): ... return ['id 1'] >>> config.publisher = demo_publish >>> config.publish(report) ['id 1'] >>> report['id'] 'id 1' * The pprint_to_stream publisher will print out reports to a stream after pprinting them. This can be very helpful for interactive use. * The related project oops_datedir_repo contains a local disk based repository which can be used as a publisher. More coming soon. Installation ============ Either run setup.py in an environment with all the dependencies available, or add the working directory to your PYTHONPATH. Development =========== Upstream development takes place at https://launchpad.net/python-oops. To setup a working area for development, if the dependencies are not immediately available, you can use ./bootstrap.py to create bin/buildout, then bin/py to get a python interpreter with the dependencies available. To run the tests use the runner of your choice, the test suite is oops.tests.test_suite. For instance:: $ bin/py -m testtools.run oops.tests.test_suite Alternative you can use the testr command from testrepository:: $ testr run