structlog-15.2.0/0000755000076500000240000000000012536025152014067 5ustar hynekstaff00000000000000structlog-15.2.0/.coveragerc0000644000076500000240000000014512522574421016213 0ustar hynekstaff00000000000000[run] branch = True source = structlog [report] show_missing = True omit = structlog/_compat.py structlog-15.2.0/.travis.yml0000644000076500000240000000101312516146537016204 0ustar hynekstaff00000000000000language: python sudo: false python: 2.7 env: - TOX_ENV=py26-threads - TOX_ENV=py27-threads - TOX_ENV=py33-threads - TOX_ENV=py34-threads - TOX_ENV=pypy-threads - TOX_ENV=py26-greenlets - TOX_ENV=py27-greenlets - TOX_ENV=py33-greenlets - TOX_ENV=py34-greenlets - TOX_ENV=pypy-greenlets - TOX_ENV=docs - TOX_ENV=flake8-py2 - TOX_ENV=flake8-py3 - TOX_ENV=manifest install: - pip install tox coveralls script: - tox -e $TOX_ENV after_success: - coveralls structlog-15.2.0/AUTHORS.rst0000644000076500000240000000334412501224114015741 0ustar hynekstaff00000000000000Authors ------- ``structlog`` is written and maintained by `Hynek Schlawack `_. It’s inspired by previous work done by `Jean-Paul Calderone `_ and `David Reid `_. The development is kindly supported by `Variomedia AG `_. The following folks helped forming ``structlog`` into what it is now: - `Alex Gaynor `_ - `Christopher Armstrong `_ - `Daniel Lindsley `_ - `David Reid `_ - `Donald Stufft `_ - `George-Cristian Bîrzan `_ - `Glyph `_ - `Holger Krekel `_ - `Itamar Turner-Trauring `_ - `Jack Pearkes `_ - `Jean-Paul Calderone `_ - `Lakshmi Kannan `_ - `Lynn Root `_ - `Mathieu Leplatre `_ - `Noah Kantrowitz `_ - `Tarek Ziadé `_ - `Thomas Heinrichsdobler `_ - `Tom Lazar `_ - `Wouter Bolsterlee `_ Some of them disapprove of the addition of thread local context data. :) Third Party Code ^^^^^^^^^^^^^^^^ The compatibility code that makes this software run on both Python 2 and 3 is heavily inspired and partly copy and pasted from the MIT-licensed six_ by Benjamin Peterson. The only reason why it’s not used as a dependency is to avoid any runtime dependency in the first place. .. _six: https://bitbucket.org/gutworth/six/ structlog-15.2.0/CONTRIBUTING.rst0000644000076500000240000000434212460465545016545 0ustar hynekstaff00000000000000How To Contribute ================= Every open source project lives from the generous help by contributors that sacrifice their time and structlog is no different. To make participation as pleasant as possible, this project adheres to the `Code of Conduct`_ by the Python Software Foundation. Here are a few hints and rules to get you started: - Add yourself to the AUTHORS.rst_ file in an alphabetical fashion. Every contribution is valuable and shall be credited. - If your change is noteworthy, add an entry to the changelog_. - No contribution is too small; please submit as many fixes for typos and grammar bloopers as you can! - Don’t *ever* break backward compatibility. ``structlog`` is an infrastructure library people rely on; therefore highest care must be put into avoiding breakage on updates. If it ever *has* to happen for higher reasons, structlog will follow the proven procedures_ of the Twisted project. - *Always* add tests and docs for your code. This is a hard rule; patches with missing tests or documentation won’t be merged – if a feature is not tested or documented, it doesn’t exist. - Obey `PEP 8`_ and `PEP 257`_. Twisted-specific modules use CamelCase. - Write `good commit messages`_. .. note:: If you have something great but aren’t sure whether it adheres -- or even can adhere -- to the rules above: **please submit a pull request anyway**! In the best case, we can mold it into something, in the worst case the pull request gets politely closed. There’s absolutely nothing to fear. Thank you for considering to contribute to structlog! If you have any question or concerns, feel free to reach out to me -- there is also a ``#structlog`` channel on freenode_. .. _`PEP 8`: https://www.python.org/dev/peps/pep-0008/ .. _`PEP 257`: https://www.python.org/dev/peps/pep-0257/ .. _`good commit messages`: http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html .. _`Code of Conduct`: https://www.python.org/psf/codeofconduct/ .. _changelog: https://github.com/hynek/structlog/blob/master/docs/changelog.rst .. _AUTHORS.rst: https://github.com/hynek/structlog/blob/master/AUTHORS.rst .. _procedures: https://twistedmatrix.com/trac/wiki/CompatibilityPolicy .. _`freenode`: https://freenode.net structlog-15.2.0/dev-requirements.txt0000644000076500000240000000013012455675145020136 0ustar hynekstaff00000000000000-r docs-requirements.txt check-manifest freezegun pretend pytest-cov pytest twine wheel structlog-15.2.0/docs/0000755000076500000240000000000012536025152015017 5ustar hynekstaff00000000000000structlog-15.2.0/docs/_static/0000755000076500000240000000000012536025152016445 5ustar hynekstaff00000000000000structlog-15.2.0/docs/_static/.keep0000644000076500000240000000000012204213705017353 0ustar hynekstaff00000000000000structlog-15.2.0/docs/api.rst0000644000076500000240000000405312460460063016324 0ustar hynekstaff00000000000000.. _api: structlog Package ================= .. module:: structlog :mod:`structlog` Package ------------------------ .. autofunction:: get_logger .. autofunction:: getLogger .. autofunction:: wrap_logger .. autofunction:: configure .. autofunction:: configure_once .. autofunction:: reset_defaults .. autoclass:: BoundLogger :members: new, bind, unbind .. autoclass:: PrintLogger :members: msg, err, debug, info, warning, error, critical, log .. autoclass:: PrintLoggerFactory .. autoclass:: ReturnLogger :members: msg, err, debug, info, warning, error, critical, log .. autoclass:: ReturnLoggerFactory .. autoexception:: DropEvent .. autoclass:: BoundLoggerBase :members: new, bind, unbind, _logger, _process_event, _proxy_to_logger :mod:`threadlocal` Module ------------------------- .. automodule:: structlog.threadlocal .. autofunction:: wrap_dict .. autofunction:: tmp_bind(logger, **tmp_values) .. autofunction:: as_immutable .. _procs: :mod:`processors` Module ------------------------ .. automodule:: structlog.processors .. autoclass:: JSONRenderer .. autoclass:: KeyValueRenderer .. autoclass:: UnicodeEncoder .. autofunction:: format_exc_info .. autoclass:: StackInfoRenderer .. autoclass:: ExceptionPrettyPrinter .. autoclass:: TimeStamper(fmt=None, utc=True) :mod:`stdlib` Module -------------------- .. automodule:: structlog.stdlib .. autoclass:: BoundLogger :members: bind, unbind, new, debug, info, warning, warn, error, critical, exception, log .. autoclass:: LoggerFactory :members: __call__ .. autofunction:: filter_by_level .. autofunction:: add_log_level .. autofunction:: add_logger_name .. autoclass:: PositionalArgumentsFormatter :mod:`twisted` Module --------------------- .. automodule:: structlog.twisted .. autoclass:: BoundLogger :members: bind, unbind, new, msg, err .. autoclass:: LoggerFactory :members: __call__ .. autoclass:: EventAdapter .. autoclass:: JSONRenderer .. autofunction:: plainJSONStdOutLogger .. autofunction:: JSONLogObserverWrapper .. autoclass:: PlainFileLogObserver structlog-15.2.0/docs/changelog.rst0000644000076500000240000001244712536006150017505 0ustar hynekstaff00000000000000========= Changelog ========= - :release:`15.2.0 <2015-06-10>` - :bug:`- major` Allow empty lists of processors. This is a valid use case since `#26 `_ has been merged. Before, supplying an empty list resulted in the defaults being used. - :bug:`- major` Prevent Twisted's ``log.err`` from quoting strings rendered by :class:`structlog.twisted.JSONRenderer`. - :feature:`52` Better support of :meth:`logging.Logger.exception` within ``structlog``. - :feature:`51` Add option to specify target key in :class:`structlog.processors.TimeStamper` processor. - :release:`15.1.0 <2015-02-24>` - :bug:`- major` Tolerate frames without a ``__name__``. - :release:`15.0.0 <2015-01-23>` - :feature:`44` Add :func:`structlog.stdlib.add_log_level` and :func:`structlog.stdlib.add_logger_name` processors. - :feature:`42` Add :func:`structlog.stdlib.BoundLogger.log`. - :feature:`19` Pass positional arguments to stdlib wrapped loggers that use string formatting. - :feature:`28` structlog is now dually licensed under the `Apache License, Version 2 `_ and the `MIT `_ license. Therefore it is now legal to use structlog with `GPLv2 `_-licensed projects. - :feature:`22` Add :func:`structlog.stdlib.BoundLogger.exception`. - :release:`0.4.2 <2014-07-26>` - :bug:`8` Fixed a memory leak in greenlet code that emulates thread locals. It shouldn't matter in practice unless you use multiple wrapped dicts within one program that is rather unlikely. - :feature:`-` :class:`structlog.PrintLogger` now is thread-safe. - :feature:`-` Test Twisted-related code on Python 3 (with some caveats). - :feature:`-` Drop support for Python 3.2. There is no justification to add complexity for a Python version that nobody uses. If you are one of the `0.350% `_ that use Python 3.2, please stick to the 0.4 branch; critical bugs will still be fixed. - :feature:`-` Officially support Python 3.4. - :feature:`26` Allow final processor to return a dictionary. See :ref:`adapting`. - :bug:`-` ``from structlog import *`` works now (but you still shouldn't use it). - :release:`0.4.1 <2013-12-19>` - :bug:`-` Don't cache proxied methods in :class:`structlog.threadlocal._ThreadLocalDictWrapper`. This doesn't affect regular users. - :bug:`-` Various doc fixes. - :release:`0.4.0 <2013-11-10>` - :feature:`6` Add :class:`structlog.processors.StackInfoRenderer` for adding stack information to log entries without involving exceptions. Also added it to default processor chain. - :feature:`12` Allow optional positional arguments for :func:`structlog.get_logger` that are passed to logger factories. The standard library factory uses this for explicit logger naming. - :feature:`-` Add :class:`structlog.processors.ExceptionPrettyPrinter` for development and testing when multiline log entries aren't just acceptable but even helpful. - :feature:`-` Allow the standard library name guesser to ignore certain frame names. This is useful together with frameworks. - :feature:`5` Add meta data (e.g. function names, line numbers) extraction for wrapped stdlib loggers. - :release:`0.3.2 <2013-09-27>` - :bug:`-` Fix stdlib's name guessing. - :release:`0.3.1 <2013-09-26>` - :bug:`-` Add forgotten :class:`structlog.processors.TimeStamper` to API documentation. - :release:`0.3.0 <2013-09-23>` - :support:`-` Greatly enhanced and polished the documentation and added a new theme based on Write The Docs, requests, and Flask. See :doc:`license`. - :feature:`-` Add Python Standard Library-specific BoundLogger that has an explicit API instead of intercepting unknown method calls. See :class:`structlog.stdlib.BoundLogger`. - :feature:`-` :class:`structlog.ReturnLogger` now allows arbitrary positional and keyword arguments. - :feature:`-` Add Twisted-specific BoundLogger that has an explicit API instead of intercepting unknown method calls. See :class:`structlog.twisted.BoundLogger`. - :feature:`-` Allow logger proxies that are returned by :func:`structlog.get_logger` and :func:`structlog.wrap_logger` to cache the BoundLogger they assemble according to configuration on first use. See :doc:`performance` and the `cache_logger_on_first_use` of :func:`structlog.configure` and :func:`structlog.wrap_logger`. - :feature:`-` Extract a common base class for loggers that does nothing except keeping the context state. This makes writing custom loggers much easier and more straight-forward. See :class:`structlog.BoundLoggerBase`. - :release:`0.2.0 <2013-09-17>` - :feature:`-` Promote to stable, thus henceforth a strict backward compatibility policy is put into effect. See :ref:`contributing`. - :feature:`-` Add `key_order` option to :class:`structlog.processors.KeyValueRenderer` for more predictable log entries with any `dict` class. - :feature:`-` :class:`structlog.PrintLogger` now uses proper I/O routines and is thus viable not only for examples but also for production. - :feature:`-` :doc:`Enhance Twisted support ` by offering JSONification of non-structlog log entries. - :feature:`-` Allow for custom serialization in :class:`structlog.twisted.JSONRenderer` without abusing ``__repr__``. - :release:`0.1.0 <2013-09-16>` - :feature:`-` Initial work. structlog-15.2.0/docs/code_examples/0000755000076500000240000000000012536025152017627 5ustar hynekstaff00000000000000structlog-15.2.0/docs/code_examples/flask_/0000755000076500000240000000000012536025152021066 5ustar hynekstaff00000000000000structlog-15.2.0/docs/code_examples/flask_/__init__.py0000644000076500000240000000000012215562225023166 0ustar hynekstaff00000000000000structlog-15.2.0/docs/code_examples/flask_/some_module.py0000644000076500000240000000044012217075173023752 0ustar hynekstaff00000000000000from structlog import get_logger logger = get_logger() def some_function(): # later then: logger.error('user did something', something='shot_in_foot') # gives you: # event='user did something 'request_id='ffcdc44f-b952-4b5f-95e6-0f1f3a9ee5fd' something='shot_in_foot' structlog-15.2.0/docs/code_examples/flask_/webapp.py0000644000076500000240000000152012217075173022720 0ustar hynekstaff00000000000000import uuid import flask import structlog from .some_module import some_function logger = structlog.get_logger() app = flask.Flask(__name__) @app.route('/login', methods=['POST', 'GET']) def some_route(): log = logger.new( request_id=str(uuid.uuid4()), ) # do something # ... log.info('user logged in', user='test-user') # gives you: # event='user logged in' request_id='ffcdc44f-b952-4b5f-95e6-0f1f3a9ee5fd' user='test-user' # ... some_function() # ... if __name__ == "__main__": structlog.configure( processors=[ structlog.processors.KeyValueRenderer( key_order=['event', 'request_id'], ), ], context_class=structlog.threadlocal.wrap_dict(dict), logger_factory=structlog.stdlib.LoggerFactory(), ) app.run() structlog-15.2.0/docs/code_examples/getting-started/0000755000076500000240000000000012536025152022734 5ustar hynekstaff00000000000000structlog-15.2.0/docs/code_examples/getting-started/imaginary_web.py0000644000076500000240000000077412215562225026134 0ustar hynekstaff00000000000000from structlog import get_logger log = get_logger() def view(request): user_agent = request.get('HTTP_USER_AGENT', 'UNKNOWN') peer_ip = request.client_addr if something: log.msg('something', user_agent=user_agent, peer_ip=peer_ip) return 'something' elif something_else: log.msg('something_else', user_agent=user_agent, peer_ip=peer_ip) return 'something_else' else: log.msg('else', user_agent=user_agent, peer_ip=peer_ip) return 'else' structlog-15.2.0/docs/code_examples/getting-started/imaginary_web_better.py0000644000076500000240000000076312215562225027477 0ustar hynekstaff00000000000000from structlog import get_logger logger = get_logger() def view(request): log = logger.bind( user_agent=request.get('HTTP_USER_AGENT', 'UNKNOWN'), peer_ip=request.client_addr, ) foo = request.get('foo') if foo: log = log.bind(foo=foo) if something: log.msg('something') return 'something' elif something_else: log.msg('something_else') return 'something_else' else: log.msg('else') return 'else' structlog-15.2.0/docs/code_examples/processors/0000755000076500000240000000000012536025152022031 5ustar hynekstaff00000000000000structlog-15.2.0/docs/code_examples/processors/conditional_dropper.py0000644000076500000240000000123112215562225026437 0ustar hynekstaff00000000000000from structlog import DropEvent class ConditionalDropper(object): def __init__(self, peer_to_ignore): self._peer_to_ignore = peer_to_ignore def __call__(self, logger, method_name, event_dict): """ >>> cd = ConditionalDropper('127.0.0.1') >>> cd(None, None, {'event': 'foo', 'peer': '10.0.0.1'}) {'peer': '10.0.0.1', 'event': 'foo'} >>> cd(None, None, {'event': 'foo', 'peer': '127.0.0.1'}) Traceback (most recent call last): ... DropEvent """ if event_dict.get('peer') == self._peer_to_ignore: raise DropEvent else: return event_dict structlog-15.2.0/docs/code_examples/processors/dropper.py0000644000076500000240000000014412215562225024056 0ustar hynekstaff00000000000000from structlog import DropEvent def dropper(logger, method_name, event_dict): raise DropEvent structlog-15.2.0/docs/code_examples/processors/timestamper.py0000644000076500000240000000024212215562225024734 0ustar hynekstaff00000000000000import calendar import time def timestamper(logger, log_method, event_dict): event_dict['timestamp'] = calendar.timegm(time.gmtime()) return event_dict structlog-15.2.0/docs/code_examples/twisted_echo.py0000644000076500000240000000175112241217552022666 0ustar hynekstaff00000000000000import sys import uuid import structlog import twisted from twisted.internet import protocol, reactor logger = structlog.getLogger() class Counter(object): i = 0 def inc(self): self.i += 1 def __repr__(self): return str(self.i) class Echo(protocol.Protocol): def connectionMade(self): self._counter = Counter() self._log = logger.new( connection_id=str(uuid.uuid4()), peer=self.transport.getPeer().host, count=self._counter, ) def dataReceived(self, data): self._counter.inc() log = self._log.bind(data=data) self.transport.write(data) log.msg('echoed data!') if __name__ == "__main__": structlog.configure( processors=[structlog.twisted.EventAdapter()], logger_factory=structlog.twisted.LoggerFactory(), ) twisted.python.log.startLogging(sys.stderr) reactor.listenTCP(1234, protocol.Factory.forProtocol(Echo)) reactor.run() structlog-15.2.0/docs/conf.py0000644000076500000240000002437012460465545016336 0ustar hynekstaff00000000000000# -*- coding: utf-8 -*- # This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. # This file is execfile()d with the current directory set to its containing dir # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import codecs import datetime import os import re try: import sphinx_rtd_theme except ImportError: sphinx_rtd_theme = None here = os.path.abspath(os.path.dirname(__file__)) def read(*parts): return codecs.open(os.path.join(here, *parts), 'r').read() def find_version(*file_paths): version_file = read(*file_paths) version_match = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]", version_file, re.M) if version_match: return version_match.group(1) raise RuntimeError("Unable to find version string.") # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # sys.path.insert(0, os.path.abspath('.')) # -- General configuration ---------------------------------------------------- # If your documentation needs a minimal Sphinx version, state it here. # needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [ 'releases', 'sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.intersphinx', 'sphinx.ext.viewcode', ] # 'releases' (changelog) settings releases_issue_uri = "https://github.com/hynek/structlog/issues/%s" releases_release_uri = "https://github.com/hynek/structlog/tree/%s" # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = u'structlog' author = u"Hynek Schlawack" copyright = u'2013-{0}, {1}'.format(datetime.date.today().year, author) # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = find_version('..', 'structlog', '__init__.py') # The full version, including alpha/beta/rc tags. release = '' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = [ '_build', ] # The reST default role (used for this markup: `text`) to use for all # documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # -- Options for HTML output -------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. if sphinx_rtd_theme: html_theme = "sphinx_rtd_theme" html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] else: html_theme = "default" # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. # html_theme_path = ['_themes'] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'structlogdoc' # -- Options for LaTeX output ------------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass # [howto/manual]). latex_documents = [ ('index', 'structlog.tex', u'structlog Documentation', u'Author', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output ------------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'structlog', u'structlog Documentation', [u'Author'], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ----------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ('index', 'structlog', u'structlog Documentation', u'Author', 'structlog', 'One line description of project.', 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' # -- Options for Epub output -------------------------------------------------- # Bibliographic Dublin Core info. epub_title = project epub_author = author epub_publisher = author epub_copyright = copyright # The language of the text. It defaults to the language option # or en if the language is not set. #epub_language = '' # The scheme of the identifier. Typical schemes are ISBN or URL. #epub_scheme = '' # The unique identifier of the text. This can be a ISBN number # or the project homepage. #epub_identifier = '' # A unique identification for the text. #epub_uid = '' # A tuple containing the cover image and cover page html template filenames. #epub_cover = () # HTML files that should be inserted before the pages created by sphinx. # The format is a list of tuples containing the path and title. #epub_pre_files = [] # HTML files shat should be inserted after the pages created by sphinx. # The format is a list of tuples containing the path and title. #epub_post_files = [] # A list of files that should not be packed into the epub file. #epub_exclude_files = [] # The depth of the table of contents in toc.ncx. #epub_tocdepth = 3 # Allow duplicate toc entries. #epub_tocdup = True linkcheck_ignore = [ # 404s for unknown reasons r'http://graylog2.org.*', # Times out way too often r'http://www.rabbitmq.com', # throws a 406 for unknown reasons r'http://www.elasticsearch.org', ] # Twisted's trac tends to be slow linkcheck_timeout = 300 intersphinx_mapping = { 'https://docs.python.org/2': None, } structlog-15.2.0/docs/configuration.rst0000644000076500000240000001636512471544331020436 0ustar hynekstaff00000000000000.. _configuration: Configuration ============= Global Defaults --------------- To make logging as unintrusive and straight-forward to use as possible, structlog comes with a plethora of configuration options and convenience functions. Let me start at the end and introduce you to the ultimate convenience function that relies purely on configuration: :func:`structlog.get_logger` (and its Twisted-friendly alias :func:`structlog.getLogger`). The goal is to reduce your per-file logging boilerplate to:: from structlog import get_logger logger = get_logger() while still giving you the full power via configuration. To achieve that you'll have to call :func:`structlog.configure` on app initialization (of course, only if you're not content with the defaults). The :ref:`example ` from the previous chapter could thus have been written as following: .. testcleanup:: * import structlog structlog.reset_defaults() .. testsetup:: config_wrap_logger, config_get_logger from structlog import PrintLogger, configure, reset_defaults, wrap_logger, get_logger from structlog.threadlocal import wrap_dict def proc(logger, method_name, event_dict): print 'I got called with', event_dict return repr(event_dict) .. doctest:: config_wrap_logger >>> configure(processors=[proc], context_class=dict) >>> log = wrap_logger(PrintLogger()) >>> log.msg('hello world') I got called with {'event': 'hello world'} {'event': 'hello world'} In fact, it could even be written like .. doctest:: config_get_logger >>> configure(processors=[proc], context_class=dict) >>> log = get_logger() >>> log.msg('hello world') I got called with {'event': 'hello world'} {'event': 'hello world'} because :class:`~structlog.processors.PrintLogger` is the default ``LoggerFactory`` used (see :ref:`logger-factories`). ``structlog`` tries to behave in the least surprising way when it comes to handling defaults and configuration: #. Arguments passed to :func:`structlog.wrap_logger` *always* take the highest precedence over configuration. That means that you can overwrite whatever you've configured for each logger respectively. #. If you leave them on `None`, structlog will check whether you've configured default values using :func:`structlog.configure` and uses them if so. #. If you haven't configured or passed anything at all, the default fallback values are used which means :class:`collections.OrderedDict` for context and ``[``:class:`~structlog.processors.StackInfoRenderer`, :func:`~structlog.processors.format_exc_info`, :class:`~structlog.processors.KeyValueRenderer`\ ``]`` for the processor chain, and `False` for `cache_logger_on_first_use`. If necessary, you can always reset your global configuration back to default values using :func:`structlog.reset_defaults`. That can be handy in tests. .. note:: Since you will call :func:`structlog.wrap_logger` (or one of the ``get_logger()`` functions) most likely at import time and thus before you had a chance to configure structlog, they return a **proxy** that returns a correct wrapped logger on first ``bind()``/``new()``. Therefore, you must not call ``new()`` or ``bind()`` in module scope! Use :func:`~structlog.get_logger`\ 's ``initial_values`` to achieve pre-populated contexts. To enable you to log with the module-global logger, it will create a temporary BoundLogger and relay the log calls to it on *each call*. Therefore if you have nothing to bind but intend to do lots of log calls in a function, it makes sense performance-wise to create a local logger by calling ``bind()`` or ``new()`` without any parameters. See also :doc:`performance`. .. _logger-factories: Logger Factories ---------------- To make :func:`structlog.get_logger` work, one needs one more option that hasn't been discussed yet: ``logger_factory``. It is a callable that returns the logger that gets wrapped and returned. In the simplest case, it's a function that returns a logger -- or just a class. But you can also pass in an instance of a class with a ``__call__`` method for more complicated setups. .. versionadded:: 0.4.0 :func:`structlog.get_logger` can optionally take positional parameters. These will be passed to the logger factories. For example, if you use run ``structlog.get_logger('a name')`` and configure structlog to use the standard library :class:`~structlog.stdlib.LoggerFactory` which has support for positional parameters, the returned logger will have the name ``'a name'``. When writing custom logger factories, they should always accept positional parameters even if they don't use them. That makes sure that loggers are interchangeable. For the common cases of standard library logging and Twisted logging, structlog comes with two factories built right in: - :class:`structlog.stdlib.LoggerFactory` - :class:`structlog.twisted.LoggerFactory` So all it takes to use structlog with standard library logging is this:: >>> from structlog import get_logger, configure >>> from structlog.stdlib import LoggerFactory >>> configure(logger_factory=LoggerFactory()) >>> log = get_logger() >>> log.critical('this is too easy!') event='this is too easy!' By using structlog's :class:`structlog.stdlib.LoggerFactory`, it is also ensured that variables like function names and line numbers are expanded correctly in your log format. The :ref:`Twisted example ` shows how easy it is for Twisted. .. note:: `LoggerFactory()`-style factories always need to get passed as *instances* like in the examples above. While neither allows for customization using parameters yet, they may do so in the future. Calling :func:`structlog.get_logger` without configuration gives you a perfectly useful :class:`structlog.PrintLogger` with the default values exaplained above. I don't believe silent loggers are a sensible default. Where to Configure ------------------ The best place to perform your configuration varies with applications and frameworks. Ideally as late as possible but *before* non-framework (i.e. your) code is executed. If you use standard library's logging, it makes sense to configure them next to each other. **Django** Django has to date unfortunately no concept of an application assembler or "app is done" hooks. Therefore the bottom of your ``settings.py`` will have to do. **Flask** See `Logging Application Errors `_. **Pyramid** `Application constructor `_. **Twisted** The `plugin definition `_ is the best place. If your app is not a plugin, put it into your `tac file `_ (and then `learn `_ about plugins). If you have no choice but *have* to configure on import time in module-global scope, or can't rule out for other reasons that that your :func:`structlog.configure` gets called more than once, structlog offers :func:`structlog.configure_once` that raises a warning if structlog has been configured before (no matter whether using :func:`structlog.configure` or :func:`~structlog.configure_once`) but doesn't change anything. structlog-15.2.0/docs/contributing.rst0000644000076500000240000000006412217075173020264 0ustar hynekstaff00000000000000.. _contributing: .. include:: ../CONTRIBUTING.rst structlog-15.2.0/docs/custom-wrappers.rst0000644000076500000240000000450312217620110020715 0ustar hynekstaff00000000000000Custom Wrappers =============== structlog comes with a generic bound logger called :class:`structlog.BoundLogger` that can be used to wrap any logger class you fancy. It does so by intercepting unknown method names and proxying them to the wrapped logger. This works fine, except that it has a performance penalty and the API of :class:`~structlog.BoundLogger` isn't clear from reading the documentation because large parts depend on the wrapped logger. An additional reason is that you may want to have semantically meaningful log method names that add meta data to log entries as it is fit (see example below). To solve that, structlog offers you to use an own wrapper class which you can configure using :func:`structlog.configure`. And to make it easier for you, it comes with the class :class:`structlog.BoundLoggerBase` which takes care of all data binding duties so you just add your log methods if you choose to sub-class it. .. _wrapper_class-example: Example ------- It's much easier to demonstrate with an example: .. doctest:: >>> from structlog import BoundLoggerBase, PrintLogger, wrap_logger >>> class SemanticLogger(BoundLoggerBase): ... def msg(self, event, **kw): ... if not 'status' in kw: ... return self._proxy_to_logger('msg', event, status='ok', **kw) ... else: ... return self._proxy_to_logger('msg', event, **kw) ... ... def user_error(self, event, **kw): ... self.msg(event, status='user_error', **kw) >>> log = wrap_logger(PrintLogger(), wrapper_class=SemanticLogger) >>> log = log.bind(user='fprefect') >>> log.user_error('user.forgot_towel') user='fprefect' status='user_error' event='user.forgot_towel' You can observe the following: - The wrapped logger can be found in the instance variable :attr:`structlog.BoundLoggerBase._logger`. - The helper method :func:`structlog.BoundLoggerBase._proxy_to_logger` that is a DRY_ convenience function that runs the processor chain, handles possible :exc:`~structlog.DropEvent`\ s and calls a named function on :attr:`~structlog.BoundLoggerBase._logger`. - You can run the chain by hand though using :func:`structlog.BoundLoggerBase._process_event` . These two methods and one attribute is all you need to write own wrapper classes. .. _DRY: http://en.wikipedia.org/wiki/Don%27t_repeat_yourself structlog-15.2.0/docs/examples.rst0000644000076500000240000001005312460465545017400 0ustar hynekstaff00000000000000.. _examples: Examples ======== This chapter is intended to give you a taste of realistic usage of structlog. .. _flask-example: Flask and Thread Local Data --------------------------- In the simplest case, you bind a unique request ID to every incoming request so you can easily see which log entries belong to which request. .. literalinclude:: code_examples/flask_/webapp.py :language: python ``some_module.py`` .. literalinclude:: code_examples/flask_/some_module.py :language: python While wrapped loggers are *immutable* by default, this example demonstrates how to circumvent that using a thread local dict implementation for context data for convenience (hence the requirement for using `new()` for re-initializing the logger). Please note that :class:`structlog.stdlib.LoggerFactory` is a totally magic-free class that just deduces the name of the caller's module and does a :func:`logging.getLogger` with it. It's used by :func:`structlog.get_logger` to rid you of logging boilerplate in application code. If you prefer to name your standard library loggers explicitly, a positional argument to :func:`~structlog.get_logger` gets passed to the factory and used as the name. .. _twisted-example: Twisted, and Logging Out Objects -------------------------------- If you prefer to log less but with more context in each entry, you can bind everything important to your logger and log it out with each log entry. .. literalinclude:: code_examples/twisted_echo.py :language: python gives you something like: .. code:: text ... peer='127.0.0.1' connection_id='1c6c0cb5-...' count=1 data='123\n' event='echoed data!' ... peer='127.0.0.1' connection_id='1c6c0cb5-...' count=2 data='456\n' event='echoed data!' ... peer='127.0.0.1' connection_id='1c6c0cb5-...' count=3 data='foo\n' event='echoed data!' ... peer='10.10.0.1' connection_id='85234511-...' count=1 data='cba\n' event='echoed data!' ... peer='127.0.0.1' connection_id='1c6c0cb5-...' count=4 data='bar\n' event='echoed data!' Since Twisted's logging system is a bit peculiar, structlog ships with an :class:`adapter ` so it keeps behaving like you'd expect it to behave. I'd also like to point out the Counter class that doesn't do anything spectacular but gets bound *once* per connection to the logger and since its repr is the number itself, it's logged out correctly for each event. This shows off the strength of keeping a dict of objects for context instead of passing around serialized strings. .. _processors-examples: Processors ---------- :ref:`Processors` are a both simple and powerful feature of structlog. So you want timestamps as part of the structure of the log entry, censor passwords, filter out log entries below your log level before they even get rendered, and get your output as JSON for convenient parsing? Here you go: .. doctest:: >>> import datetime, logging, sys >>> from structlog import wrap_logger >>> from structlog.processors import JSONRenderer >>> from structlog.stdlib import filter_by_level >>> logging.basicConfig(stream=sys.stdout, format='%(message)s') >>> def add_timestamp(_, __, event_dict): ... event_dict['timestamp'] = datetime.datetime.utcnow() ... return event_dict >>> def censor_password(_, __, event_dict): ... pw = event_dict.get('password') ... if pw: ... event_dict['password'] = '*CENSORED*' ... return event_dict >>> log = wrap_logger( ... logging.getLogger(__name__), ... processors=[ ... filter_by_level, ... add_timestamp, ... censor_password, ... JSONRenderer(indent=1, sort_keys=True) ... ] ... ) >>> log.info('something.filtered') >>> log.warning('something.not_filtered', password='secret') # doctest: +SKIP { "event": "something.not_filtered", "password": "*CENSORED*", "timestamp": "datetime.datetime(..., ..., ..., ..., ...)" } structlog comes with many handy processors build right in -- for a list of shipped processors, check out the :ref:`API documentation `. structlog-15.2.0/docs/getting-started.rst0000644000076500000240000001235012460465545020671 0ustar hynekstaff00000000000000.. _getting-started: Getting Started =============== .. _install: Installation ------------ structlog can be easily installed using:: $ pip install structlog Python 2.6 ^^^^^^^^^^ If you're running Python 2.6 and want to use ``OrderedDict``\ s for your context (which is the default), you also have to install the respective compatibility package:: $ pip install ordereddict If the order of the keys of your context doesn't matter (e.g. if you're logging JSON that gets parsed anyway), simply use a vanilla ``dict`` to avoid this dependency. See :doc:`configuration` on how to achieve that. Your First Log Entry -------------------- A lot of effort went into making structlog accessible without reading pages of documentation. And indeed, the simplest possible usage looks like this: .. doctest:: >>> import structlog >>> log = structlog.get_logger() >>> log.msg('greeted', whom='world', more_than_a_string=[1, 2, 3]) whom='world' more_than_a_string=[1, 2, 3] event='greeted' Here, structlog takes full advantage of its hopefully useful default settings: - Output is sent to `standard out`_ instead of exploding into the user's face. Yes, that seems a rather controversial attitude towards logging. - All keywords are formatted using :class:`structlog.processors.KeyValueRenderer`. That in turn uses `repr()`_ to serialize all values to strings. Thus, it's easy to add support for logging of your own objects\ [*]_. It should be noted that even in most complex logging setups the example would still look just like that thanks to :ref:`configuration`. There you go, structured logging! However, this alone wouldn't warrant its own package. After all, there's even a recipe_ on structured logging for the standard library. So let's go a step further. Building a Context ------------------ Imagine a hypothetical web application that wants to log out all relevant data with just the API from above: .. literalinclude:: code_examples/getting-started/imaginary_web.py :language: python The calls themselves are nice and straight to the point, however you're repeating yourself all over the place. At this point, you'll be tempted to write a closure like :: def log_closure(event): log.msg(event, user_agent=user_agent, peer_ip=peer_ip) inside of the view. Problem solved? Not quite. What if the parameters are introduced step by step? Do you really want to have a logging closure in each of your views? Let's have a look at a better approach: .. literalinclude:: code_examples/getting-started/imaginary_web_better.py :language: python Suddenly your logger becomes your closure! For structlog, a log entry is just a dictionary called *event dict[ionary]*: - You can pre-build a part of the dictionary step by step. These pre-saved values are called the *context*. - As soon as an *event* happens -- which is a dictionary too -- it is merged together with the *context* to an *event dict* and logged out. - To keep as much order of the keys as possible, an :class:`collections.OrderedDict` is used for the context by default. - The recommended way of binding values is the one in these examples: creating new loggers with a new context. If you're okay with giving up immutable local state for convenience, you can also use :ref:`thread/greenlet local storage ` for the context. .. _standard-library-lite: structlog and Standard Library's logging ---------------------------------------- structlog's primary application isn't printing though. Instead, it's intended to wrap your *existing* loggers and **add** *structure* and *incremental context building* to them. For that, structlog is *completely* agnostic of your underlying logger -- you can use it with any logger you like. The most prominent example of such an 'existing logger' is without doubt the logging module in the standard library. To make this common case as simple as possible, structlog comes with some tools to help you: .. doctest:: >>> import logging >>> logging.basicConfig() >>> from structlog import get_logger, configure >>> from structlog.stdlib import LoggerFactory >>> configure(logger_factory=LoggerFactory()) # doctest: +SKIP >>> log = get_logger() >>> log.warn('it works!', difficulty='easy') # doctest: +SKIP WARNING:structlog...:difficulty='easy' event='it works!' In other words, you tell structlog that you would like to use the standard library logger factory and keep calling :func:`~structlog.get_logger` like before. Since structlog is mainly used together with standard library's logging, there's :doc:`more ` goodness to make it as fast and convenient as possible. Liked what you saw? ------------------- Now you're all set for the rest of the user's guide. If you want to see more code, make sure to check out the :ref:`examples`! .. [*] In production, you're more likely to use :class:`~structlog.processors.JSONRenderer` that can also be customized using a ``__structlog__`` method so you don't have to change your repr methods to something they weren't originally intended for. .. _`standard out`: https://en.wikipedia.org/wiki/Standard_out#Standard_output_.28stdout.29 .. _`repr()`: https://docs.python.org/2/reference/datamodel.html#object.__repr__ .. _recipe: https://docs.python.org/2/howto/logging-cookbook.html structlog-15.2.0/docs/index.rst0000644000076500000240000000430312460462207016662 0ustar hynekstaff00000000000000============================= Structured Logging for Python ============================= Release v\ |version| (:doc:`What's new? `). .. include:: ../README.rst :start-after: begin The Pitch ========= ``structlog`` makes structured logging with *incremental context building* and *arbitrary formatting* as easy as: .. doctest:: >>> from structlog import get_logger >>> log = get_logger() >>> log = log.bind(user='anonymous', some_key=23) >>> log = log.bind(user='hynek', another_key=42) >>> log.info('user.logged_in', happy=True) some_key=23 user='hynek' another_key=42 happy=True event='user.logged_in' Please note that this example does *not* use standard library logging (but could so :ref:`easily `). The logger that's returned by :func:`~structlog.get_logger()` is *freely* :doc:`configurable ` and uses a simple :class:`~structlog.PrintLogger` by default. For… - …reasons why structured logging in general and ``structlog`` in particular are the way to go, consult :doc:`why`. - …more realistic examples, peek into :doc:`examples`. - …getting started right away, jump straight into :doc:`getting-started`. Since ``structlog`` avoids monkey-patching and events are fully free-form, you can start using it **today**! User's Guide ============ Basics ------ .. toctree:: :maxdepth: 1 why getting-started loggers configuration thread-local processors examples Integration with Existing Systems --------------------------------- ``structlog`` can be used immediately with any existing logger. However it comes with special wrappers for the Python standard library and Twisted that are optimized for their respective underlying loggers and contain less magic. .. toctree:: :maxdepth: 2 standard-library twisted logging-best-practices Advanced Topics --------------- .. toctree:: :maxdepth: 1 custom-wrappers performance Project Information =================== .. toctree:: :maxdepth: 1 contributing license changelog API Reference ============= .. toctree:: :maxdepth: 4 api Indices and tables ================== - :ref:`genindex` - :ref:`modindex` - :ref:`search` structlog-15.2.0/docs/license.rst0000644000076500000240000000140212431374731017174 0ustar hynekstaff00000000000000License and Hall of Fame ======================== ``structlog`` is licensed both under the `Apache License, Version 2 `_ and the `MIT license `_. The reason for that is to be both protected against patent claims by own contributors and still allow the usage within GPLv2 software. For more legal details, see `this issue `_ on the bug tracker of PyCA's cryptography. The full license texts can be also found in the source code repository: - `Apache `_. - `MIT `_. .. _authors: .. include:: ../AUTHORS.rst structlog-15.2.0/docs/loggers.rst0000644000076500000240000001134312460465545017227 0ustar hynekstaff00000000000000Loggers ======= Bound Loggers ------------- The center of structlog is the immutable log wrapper :class:`~structlog.BoundLogger`. All it does is: - Keep a *context dictionary* and a *logger* that it's wrapping, - recreate itself with (optional) *additional* context data (the :func:`~structlog.BoundLogger.bind` and :func:`~structlog.BoundLogger.new` methods), - recreate itself with *less* data (:func:`~structlog.BoundLogger.unbind`), - and finally relay *all* other method calls to the wrapped logger\ [*]_ after processing the log entry with the configured chain of :ref:`processors `. You won't be instantiating it yourself though. For that there is the :func:`structlog.wrap_logger` function (or the convenience function :func:`structlog.get_logger` we'll discuss in a minute): .. _proc: .. doctest:: >>> from structlog import wrap_logger >>> class PrintLogger(object): ... def msg(self, message): ... print message >>> def proc(logger, method_name, event_dict): ... print 'I got called with', event_dict ... return repr(event_dict) >>> log = wrap_logger(PrintLogger(), processors=[proc], context_class=dict) >>> log2 = log.bind(x=42) >>> log == log2 False >>> log.msg('hello world') I got called with {'event': 'hello world'} {'event': 'hello world'} >>> log2.msg('hello world') I got called with {'x': 42, 'event': 'hello world'} {'x': 42, 'event': 'hello world'} >>> log3 = log2.unbind('x') >>> log == log3 True >>> log3.msg('nothing bound anymore', foo='but you can structure the event too') I got called with {'foo': 'but you can structure the event too', 'event': 'nothing bound anymore'} {'foo': 'but you can structure the event too', 'event': 'nothing bound anymore'} As you can see, it accepts one mandatory and a few optional arguments: **logger** The one and only positional argument is the logger that you want to wrap and to which the log entries will be proxied. If you wish to use a :ref:`configured logger factory `, set it to `None`. **processors** A list of callables that can :ref:`filter, mutate, and format ` the log entry before it gets passed to the wrapped logger. Default is ``[``:func:`~structlog.processors.format_exc_info`, :class:`~structlog.processors.KeyValueRenderer`\ ``]``. **context_class** The class to save your context in. Particularly useful for :ref:`thread local context storage `. Default is :class:`collections.OrderedDict`. Additionally, the following arguments are allowed too: **wrapper_class** A class to use instead of :class:`~structlog.BoundLogger` for wrapping. This is useful if you want to sub-class BoundLogger and add custom logging methods. BoundLogger's bind/new methods are sub-classing friendly so you won't have to re-implement them. Please refer to the :ref:`related example ` for how this may look. **initial_values** The values that new wrapped loggers are automatically constructed with. Useful for example if you want to have the module name as part of the context. .. note:: Free your mind from the preconception that log entries have to be serialized to strings eventually. All structlog cares about is a *dictionary* of *keys* and *values*. What happens to it depends on the logger you wrap and your processors alone. This gives you the power to log directly to databases, log aggregation servers, web services, and whatnot. Printing and Testing -------------------- To save you the hassle of using standard library logging for simple standard out logging, structlog ships a :class:`~structlog.PrintLogger` that can log into arbitrary files -- including standard out (which is the default if no file is passed into the constructor): .. doctest:: >>> from structlog import PrintLogger >>> PrintLogger().info('hello world!') hello world! It's handy for both examples and in combination with tools like `runit `_ or `stdout/stderr-forwarding `_. Additionally -- mostly for unit testing -- structlog also ships with a logger that just returns whatever it gets passed into it: :class:`~structlog.ReturnLogger`. .. doctest:: >>> from structlog import ReturnLogger >>> ReturnLogger().msg(42) == 42 True >>> obj = ['hi'] >>> ReturnLogger().msg(obj) is obj True >>> ReturnLogger().msg('hello', when='again') (('hello',), {'when': 'again'}) .. [*] Since this is slightly magicy, structlog comes with concrete loggers for the :doc:`standard-library` and :doc:`twisted` that offer you explicit APIs for the supported logging methods but behave identically like the generic BoundLogger otherwise. structlog-15.2.0/docs/logging-best-practices.rst0000644000076500000240000001245212536007657022123 0ustar hynekstaff00000000000000Logging Best Practices ====================== The best practice for you depends very much on your context. To give you some pointers nevertheless, here are a few scenarios that may be applicable to you. Pull requests for further interesting approaches as well as refinements and more complete examples are very welcome. Common Ideas ------------ Logging is not a new concept and in no way special to Python. Logfiles have existed for decades and there's little reason to reinvent the wheel in our little world. There are several concepts that are very well-solved in general and especially in heterogeneous environments, using special tooling for Python applications does more harm than good and makes the operations staff build dart board with your pictures. Therefore let's rely on proven tools as much as possible and do only the absolutely necessary inside of Python\ [*]_. A very nice approach is to simply log to `standard out`_ and let other tools take care of the rest. runit ^^^^^ One runner that makes this very easy is the venerable runit_ project which made it a part of its design: server processes don't detach but log to standard out instead. There it gets processed by other software -- potentially by one of its own tools: svlogd_. We use it extensively and it has proven itself extremely robust and capable; check out `this tutorial`_ if you'd like to try it. If you're not quite convinced and want an overview on running daemons, have a look at cue's `daemon showdown`_ that discusses the most common ones. Local Logging ------------- There are basically two common ways to log to local logfiles: writing yourself into files and syslog. Syslog ^^^^^^ The simplest approach to logging is to forward your entries to the syslogd_. Twisted, uwsgi, and runit support it directly. It will happily add a timestamp and write wherever you tell it in its configuration. You can also log from multiple processes into a single file and use your system's logrotate_ for log rotation. The only downside is that syslog has some quirks that show itself under high load like rate limits (`they can be switched off`_) and lost log entries. runit's svlogd ^^^^^^^^^^^^^^ If you'll choose runit for running your daemons, svlogd_ is a nicer approach. It receives the log entries via a UNIX pipe and acts on them which includes adding of parse-friendly timestamps in tai64n_ as well as filtering and log rotation. Centralized Logging ------------------- Nowadays you usually don't want your logfiles in compressed archives distributed over dozens -- if not thousands -- servers. You want them at a single location; parsed and easy to query. Syslog (Again!) ^^^^^^^^^^^^^^^ The widely deployed syslog implementation rsyslog_ supports remote logging out-of-the-box. Have a look at `this post`_ by Revolution Systems on the how. Since syslog is such a widespread solution, there are also ways to use it with basically any centralized product. Logstash with logstash-forwarder ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Logstash_ is a great way to parse, save, and search your logs. The general modus operandi is that you have log shippers that parse your log files and forward the log entries to your Logstash server and store is in elasticsearch_. If your log entries consist of a JSON dictionary (and perhaps a tai64n_ timestamp), this is pretty easy and efficient. If you can't decide on a log shipper, logstash-forwarder_ (formerly known as Lumberjack) works really well. When Logstash's ``lumberjack`` input is configured to use ``codec => "json"``, having ``structlog`` output JSON is all you need. See the documentation on the :doc:`standard-library` for an example configuration. Graylog2 ^^^^^^^^ Graylog_ goes one step further. It not only supports everything those above do (and then some); you can also log directly JSON entries towards it -- optionally even through an AMQP server (like RabbitMQ_) for better reliability. Additionally, `Graylog's Extended Log Format`_ (GELF) allows for structured data which makes it an obvious choice to use together with structlog. .. [*] This is obviously a privileged UNIX-centric view but even Windows has tools and means for log management although we won't be able to discuss them here. .. _Graylog: http://graylog2.org .. _Logstash: https://www.elastic.co/products/logstash .. _logstash-forwarder: https://github.com/elastic/logstash-forwarder .. _RabbitMQ: http://www.rabbitmq.com .. _`Graylog's Extended Log Format`: https://www.graylog.org/resources/gelf-2/ .. _`daemon showdown`: https://web.archive.org/web/20130907200323/http://tech.cueup.com/blog/2013/03/08/running-daemons/ .. _`standard out`: http://en.wikipedia.org/wiki/Standard_out#Standard_output_.28stdout.29 .. _`they can be switched off`: http://blog.abhijeetr.com/2013/01/disable-rate-limiting-in-rsyslog-v5.html .. _`this post`: http://www.revsys.com/blog/2010/aug/26/centralized-logging-fun-and-profit/ .. _`this tutorial`: https://rubyists.github.io/2011/05/02/runit-for-ruby-and-everything-else.html .. _logrotate: http://manpages.ubuntu.com/manpages/raring/man8/logrotate.8.html .. _rsyslog: http://www.rsyslog.com .. _runit: http://smarden.org/runit/ .. _svlogd: http://smarden.org/runit/svlogd.8.html .. _syslogd: http://en.wikipedia.org/wiki/Syslogd .. _tai64n: http://cr.yp.to/daemontools/tai64n.html .. _elasticsearch: https://www.elastic.co/products/elasticsearch structlog-15.2.0/docs/make.bat0000644000076500000240000001175612204211413016423 0ustar hynekstaff00000000000000@ECHO OFF REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set BUILDDIR=_build set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . set I18NSPHINXOPTS=%SPHINXOPTS% . if NOT "%PAPER%" == "" ( set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% ) if "%1" == "" goto help if "%1" == "help" ( :help echo.Please use `make ^` where ^ is one of echo. html to make standalone HTML files echo. dirhtml to make HTML files named index.html in directories echo. singlehtml to make a single large HTML file echo. pickle to make pickle files echo. json to make JSON files echo. htmlhelp to make HTML files and a HTML help project echo. qthelp to make HTML files and a qthelp project echo. devhelp to make HTML files and a Devhelp project echo. epub to make an epub echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter echo. text to make text files echo. man to make manual pages echo. texinfo to make Texinfo files echo. gettext to make PO message catalogs echo. changes to make an overview over all changed/added/deprecated items echo. linkcheck to check all external links for integrity echo. doctest to run all doctests embedded in the documentation if enabled goto end ) if "%1" == "clean" ( for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i del /q /s %BUILDDIR%\* goto end ) if "%1" == "html" ( %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/html. goto end ) if "%1" == "dirhtml" ( %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. goto end ) if "%1" == "singlehtml" ( %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. goto end ) if "%1" == "pickle" ( %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the pickle files. goto end ) if "%1" == "json" ( %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the JSON files. goto end ) if "%1" == "htmlhelp" ( %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run HTML Help Workshop with the ^ .hhp project file in %BUILDDIR%/htmlhelp. goto end ) if "%1" == "qthelp" ( %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run "qcollectiongenerator" with the ^ .qhcp project file in %BUILDDIR%/qthelp, like this: echo.^> qcollectiongenerator %BUILDDIR%\qthelp\structlog.qhcp echo.To view the help file: echo.^> assistant -collectionFile %BUILDDIR%\qthelp\structlog.ghc goto end ) if "%1" == "devhelp" ( %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp if errorlevel 1 exit /b 1 echo. echo.Build finished. goto end ) if "%1" == "epub" ( %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub if errorlevel 1 exit /b 1 echo. echo.Build finished. The epub file is in %BUILDDIR%/epub. goto end ) if "%1" == "latex" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex if errorlevel 1 exit /b 1 echo. echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. goto end ) if "%1" == "text" ( %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text if errorlevel 1 exit /b 1 echo. echo.Build finished. The text files are in %BUILDDIR%/text. goto end ) if "%1" == "man" ( %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man if errorlevel 1 exit /b 1 echo. echo.Build finished. The manual pages are in %BUILDDIR%/man. goto end ) if "%1" == "texinfo" ( %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo if errorlevel 1 exit /b 1 echo. echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. goto end ) if "%1" == "gettext" ( %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale if errorlevel 1 exit /b 1 echo. echo.Build finished. The message catalogs are in %BUILDDIR%/locale. goto end ) if "%1" == "changes" ( %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes if errorlevel 1 exit /b 1 echo. echo.The overview file is in %BUILDDIR%/changes. goto end ) if "%1" == "linkcheck" ( %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck if errorlevel 1 exit /b 1 echo. echo.Link check complete; look for any errors in the above output ^ or in %BUILDDIR%/linkcheck/output.txt. goto end ) if "%1" == "doctest" ( %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest if errorlevel 1 exit /b 1 echo. echo.Testing of doctests in the sources finished, look at the ^ results in %BUILDDIR%/doctest/output.txt. goto end ) :end structlog-15.2.0/docs/Makefile0000644000076500000240000001271012204211413016445 0ustar hynekstaff00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: -rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/structlog.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/structlog.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/structlog" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/structlog" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." structlog-15.2.0/docs/performance.rst0000644000076500000240000000351712516147413020063 0ustar hynekstaff00000000000000Performance =========== ``structlog``'s default configuration tries to be as unsurprising and not confusing to new developers as possible. Some of the choices made come with an avoidable performance price tag -- although its impact is debatable. Here are a few hints how to get most out of ``structlog`` in production: #. Use plain `dict`\ s as context classes. Python is full of them and they are highly optimized:: configure(context_class=dict) If you don't use automated parsing (you should!) and need predicable order of your keys for some reason, use the `key_order` argument of :class:`~structlog.processors.KeyValueRenderer`. #. Use a specific wrapper class instead of the generic one. ``structlog`` comes with ones for the :doc:`standard-library` and for :doc:`twisted`:: configure(wrapper_class=structlog.stdlib.BoundLogger) :doc:`Writing own wrapper classes ` is straightforward too. #. Avoid (frequently) calling log methods on loggers you get back from :func:`structlog.wrap_logger` and :func:`structlog.get_logger`. Since those functions are usually called in module scope and thus before you are able to configure them, they return a proxy that assembles the correct logger on demand. Create a local logger if you expect to log frequently without binding:: logger = structlog.get_logger() def f(): log = logger.bind() for i in range(1000000000): log.info('iterated', i=i) #. Set the `cache_logger_on_first_use` option to `True` so the aforementioned on-demand loggers will be assembled only once and cached for future uses:: configure(cache_logger_on_first_use=True) This has the only drawback is that later calls on :func:`~structlog.configure` don't have any effect on already cached loggers -- that shouldn't matter outside of testing though. structlog-15.2.0/docs/processors.rst0000644000076500000240000000772412536007344017770 0ustar hynekstaff00000000000000.. _processors: Processors ========== The true power of ``structlog`` lies in its *combinable log processors*. A log processor is a regular callable, i.e. a function or an instance of a class with a ``__call__()`` method. .. _chains: Chains ------ The *processor chain* is a list of processors. Each processors receives three positional arguments: **logger** Your wrapped logger object. For example :class:`logging.Logger`. **method_name** The name of the wrapped method. If you called ``log.warn('foo')``, it will be ``"warn"``. **event_dict** Current context together with the current event. If the context was ``{'a': 42}`` and the event is ``"foo"``, the initial ``event_dict`` will be ``{'a':42, 'event': 'foo'}``. The return value of each processor is passed on to the next one as ``event_dict`` until finally the return value of the last processor gets passed into the wrapped logging method. Examples ^^^^^^^^ If you set up your logger like: .. code:: python from structlog import PrintLogger, wrap_logger wrapped_logger = PrintLogger() logger = wrap_logger(wrapped_logger, processors=[f1, f2, f3, f4]) log = logger.new(x=42) and call ``log.msg('some_event', y=23)``, it results in the following call chain: .. code:: python wrapped_logger.msg( f4(wrapped_logger, 'msg', f3(wrapped_logger, 'msg', f2(wrapped_logger, 'msg', f1(wrapped_logger, 'msg', {'event': 'some_event', 'x': 42, 'y': 23}) ) ) ) ) In this case, ``f4`` has to make sure it returns something ``wrapped_logger.msg`` can handle (see :ref:`adapting`). The simplest modification a processor can make is adding new values to the ``event_dict``. Parsing human-readable timestamps is tedious, not so `UNIX timestamps `_ -- let's add one to each log entry! .. literalinclude:: code_examples/processors/timestamper.py :language: python Easy, isn't it? Please note, that structlog comes with such an processor built in: :class:`~structlog.processors.TimeStamper`. Filtering --------- If a processor raises :exc:`structlog.DropEvent`, the event is silently dropped. Therefore, the following processor drops every entry: .. literalinclude:: code_examples/processors/dropper.py :language: python But we can do better than that! .. _cond_drop: How about dropping only log entries that are marked as coming from a certain peer (e.g. monitoring)? .. literalinclude:: code_examples/processors/conditional_dropper.py :language: python .. _adapting: Adapting and Rendering ---------------------- An important role is played by the *last* processor because its duty is to adapt the ``event_dict`` into something the underlying logging method understands. With that, it's also the *only* processor that needs to know anything about the underlying system. It can return one of three types: - A string that is passed as the first (and only) positional argument to the underlying logger. - A tuple of ``(args, kwargs)`` that are passed as ``log_method(*args, **kwargs)``. - A dictionary which is passed as ``log_method(**kwargs)``. Therefore ``return 'hello world'`` is a shortcut for ``return (('hello world',), {})`` (the example in :ref:`chains` assumes this shortcut has been taken). This should give you enough power to use structlog with any logging system while writing agnostic processors that operate on dictionaries. .. versionchanged:: 14.0.0 Allow final processor to return a `dict`. Examples ^^^^^^^^ The probably most useful formatter for string based loggers is :class:`~structlog.processors.JSONRenderer`. Advanced log aggregation and analysis tools like `logstash `_ offer features like telling them “this is JSON, deal with it” instead of fiddling with regular expressions. More examples can be found in the :ref:`examples ` chapter. For a list of shipped processors, check out the :ref:`API documentation `. structlog-15.2.0/docs/standard-library.rst0000644000076500000240000000732412460460063021021 0ustar hynekstaff00000000000000Python Standard Library ======================= Ideally, structlog should be able to be used as a drop-in replacement for standard library's :mod:`logging` by wrapping it. In other words, you should be able to replace your call to :func:`logging.getLogger` by a call to :func:`structlog.get_logger` and things should keep working as before (if structlog is configured right, see :ref:`stdlib-config` below). If you run into incompatibilities, it is a *bug* so please take the time to `report it `_! If you're a heavy :mod:`logging` user, your `help `_ to ensure a better compatibility would be highly appreciated! Concrete Bound Logger --------------------- To make structlog's behavior less magicy, it ships with a standard library-specific wrapper class that has an explicit API instead of improvising: :class:`structlog.stdlib.BoundLogger`. It behaves exactly like the generic :class:`structlog.BoundLogger` except: - it's slightly faster due to less overhead, - has an explicit API that mirrors the log methods of standard library's :class:`logging.Logger`, - hence causing less cryptic error messages if you get method names wrong. Processors ---------- structlog comes with a few standard library-specific processors: :func:`~structlog.stdlib.filter_by_level`: Checks the log entry's log level against the configuration of standard library's logging. Log entries below the threshold get silently dropped. Put it at the beginning of your processing chain to avoid expensive operations happen in the first place. :func:`~structlog.stdlib.add_logger_name`: Adds the name of the logger to the event dictionary under the key ``logger``. :func:`~structlog.stdlib.add_log_level`: Adds the log level to the event dictionary under the key ``level``. :class:`~structlog.stdlib.PositionalArgumentsFormatter`: This processes and formats positional arguments (if any) passed to log methods in the same way the ``logging`` module would do, e.g. ``logger.info("Hello, %s", name)``. .. _stdlib-config: Suggested Configuration ----------------------- A basic configuration to output structured logs in JSON format looks like this:: import structlog structlog.configure( processors=[ structlog.stdlib.filter_by_level, structlog.stdlib.add_logger_name, structlog.stdlib.add_log_level, structlog.stdlib.PositionalArgumentsFormatter(), structlog.processors.TimeStamper(fmt='iso'), structlog.processors.StackInfoRenderer(), structlog.processors.format_exc_info, structlog.processors.JSONRenderer() ], context_class=dict, logger_factory=structlog.stdlib.LoggerFactory(), wrapper_class=structlog.stdlib.BoundLogger, cache_logger_on_first_use=True, ) To make your program behave like a proper `12 factor app`_ that outputs only JSON to ``stdout``, configure the ``logging`` module like this:: import logging import sys handler = logging.StreamHandler(sys.stdout) root_logger = logging.getLogger() root_logger.addHandler(handler) If you plan to hook up the logging output to `logstash`, as suggested in :doc:`logging-best-practices`, the simplest approach is to configure ``logstash-forwarder`` to pick up the output from your application. To achieve this, configure your process supervisor (such as ``runit`` or ``supervisord``) to store the output in a file, and have ``logstash-forwarder`` monitor that file to ship it to the central log collection server. This approach also applies to other centralized logging solutions. .. _`12 factor app`: http://12factor.net/logs structlog-15.2.0/docs/thread-local.rst0000644000076500000240000001334212460465545020125 0ustar hynekstaff00000000000000.. _threadlocal: Thread Local Context ==================== .. testcleanup:: * import structlog structlog.reset_defaults() Immutability ------------ You should call some functions with some arguments. ---David Reid The behavior of copying itself, adding new values, and returning the result is useful for applications that keep somehow their own context using classes or closures. Twisted is a :ref:`fine example ` for that. Another possible approach is passing wrapped loggers around or log only within your view where you gather errors and events using return codes and exceptions. If you are willing to do that, you should stick to it because `immutable state `_ is a very good thing\ [*]_. Sooner or later, global state and mutable data lead to unpleasant surprises. However, in the case of conventional web development, we realize that passing loggers around seems rather cumbersome, intrusive, and generally against the mainstream culture. And since it's more important that people actually *use* structlog than to be pure and snobby, structlog contains a dirty but convenient trick: thread local context storage which you may already know from `Flask `_: Thread local storage makes your logger's context global but *only within the current thread*\ [*]_. In the case of web frameworks this usually means that your context becomes global to the current request. The following explanations may sound a bit confusing at first but the :ref:`Flask example ` illustrates how simple and elegant this works in practice. Wrapped Dicts ------------- In order to make your context thread local, structlog ships with a function that can wrap any dict-like class to make it usable for thread local storage: :func:`structlog.threadlocal.wrap_dict`. Within one thread, every instance of the returned class will have a *common* instance of the wrapped dict-like class: .. doctest:: >>> from structlog.threadlocal import wrap_dict >>> WrappedDictClass = wrap_dict(dict) >>> d1 = WrappedDictClass({'a': 1}) >>> d2 = WrappedDictClass({'b': 2}) >>> d3 = WrappedDictClass() >>> d3['c'] = 3 >>> d1 is d3 False >>> d1 == d2 == d3 == WrappedDictClass() True >>> d3 # doctest: +ELLIPSIS Then use an instance of the generated class as the context class:: configure(context_class=WrappedDictClass()) .. note:: **Remember**: the instance of the class *doesn't* matter. Only the class *type* matters because *all* instances of one class *share* the *same* data. :func:`structlog.threadlocal.wrap_dict` returns always a completely *new* wrapped class: .. doctest:: >>> from structlog.threadlocal import wrap_dict >>> WrappedDictClass = wrap_dict(dict) >>> AnotherWrappedDictClass = wrap_dict(dict) >>> WrappedDictClass() != AnotherWrappedDictClass() True >>> WrappedDictClass.__name__ # doctest: +SKIP WrappedDict-41e8382d-bee5-430e-ad7d-133c844695cc >>> AnotherWrappedDictClass.__name__ # doctest: +SKIP WrappedDict-e0fc330e-e5eb-42ee-bcec-ffd7bd09ad09 In order to be able to bind values temporarily to a logger, :mod:`structlog.threadlocal` comes with a `context manager `_: :func:`~structlog.threadlocal.tmp_bind`\ : .. testsetup:: ctx from structlog import PrintLogger, wrap_logger from structlog.threadlocal import tmp_bind, wrap_dict WrappedDictClass = wrap_dict(dict) log = wrap_logger(PrintLogger(), context_class=WrappedDictClass) .. doctest:: ctx >>> log.bind(x=42) # doctest: +ELLIPSIS , ...)> >>> log.msg('event!') x=42 event='event!' >>> with tmp_bind(log, x=23, y='foo') as tmp_log: ... tmp_log.msg('another event!') y='foo' x=23 event='another event!' >>> log.msg('one last event!') x=42 event='one last event!' The state before the ``with`` statement is saved and restored once it's left. If you want to detach a logger from thread local data, there's :func:`structlog.threadlocal.as_immutable`. Downsides & Caveats ------------------- The convenience of having a thread local context comes at a price though: .. warning:: - If you can't rule out that your application re-uses threads, you *must* remember to **initialize your thread local context** at the start of each request using :func:`~structlog.BoundLogger.new` (instead of :func:`~structlog.BoundLogger.bind`). Otherwise you may start a new request with the context still filled with data from the request before. - **Don't** stop assigning the results of your ``bind()``\ s and ``new()``\ s! **Do**:: log = log.new(y=23) log = log.bind(x=42) **Don't**:: log.new(y=23) log.bind(x=42) Although the state is saved in a global data structure, you still need the global wrapped logger produce a real bound logger. Otherwise each log call will result in an instantiation of a temporary BoundLogger. See :ref:`configuration` for more details. The general sentiment against thread locals is that they're hard to test. In this case we feel like this is an acceptable trade-off. You can easily write deterministic tests using a call-capturing processor if you use the API properly (cf. warning above). This big red box is also what separates immutable local from mutable global data. .. [*] In the spirit of Python's 'consenting adults', structlog doesn't enforce the immutability with technical means. However, if you don't meddle with undocumented data, the objects can be safely considered immutable. .. [*] Special care has been taken to detect and support greenlets properly. structlog-15.2.0/docs/twisted.rst0000644000076500000240000001036012460465545017246 0ustar hynekstaff00000000000000Twisted ======= .. warning:: Since :func:`sys.exc_clear` has been dropped in Python 3, there is currently no way to avoid multiple tracebacks in your log files if using ``structlog`` together with Twisted on Python 3. Concrete Bound Logger --------------------- To make structlog's behavior less magicy, it ships with a Twisted-specific wrapper class that has an explicit API instead of improvising: :class:`structlog.twisted.BoundLogger`. It behaves exactly like the generic :class:`structlog.BoundLogger` except: - it's slightly faster due to less overhead, - has an explicit API (:func:`~structlog.twisted.BoundLogger.msg` and :func:`~structlog.twisted.BoundLogger.err`), - hence causing less cryptic error messages if you get method names wrong. In order to avoid that structlog disturbs your CamelCase harmony, it comes with an alias for :func:`structlog.get_logger` called :func:`structlog.getLogger`. Processors ---------- structlog comes with two Twisted-specific processors: :class:`~structlog.twisted.EventAdapter` This is useful if you have an existing Twisted application and just want to wrap your loggers for now. It takes care of transforming your event dictionary into something `twisted.python.log.err `_ can digest. For example:: def onError(fail): failure = fail.trap(MoonExploded) log.err(failure, _why='event-that-happend') will still work as expected. Needs to be put at the end of the processing chain. It formats the event using a renderer that needs to be passed into the constructor:: configure(processors=[EventAdapter(KeyValueRenderer()]) The drawback of this approach is that Twisted will format your exceptions as multi-line log entries which is painful to parse. Therefore structlog comes with: :class:`~structlog.twisted.JSONRenderer` Goes a step further and circumvents Twisted logger's Exception/Failure handling and renders it itself as JSON strings. That gives you regular and simple-to-parse single-line JSON log entries no matter what happens. Bending Foreign Logging To Your Will ------------------------------------ structlog comes with a wrapper for Twisted's log observers to ensure the rest of your logs are in JSON too: :func:`~structlog.twisted.JSONLogObserverWrapper`. What it does is determining whether a log entry has been formatted by :class:`~structlog.twisted.JSONRenderer` and if not, converts the log entry to JSON with `event` being the log message and putting Twisted's `system` into a second key. So for example:: 2013-09-15 22:02:18+0200 [-] Log opened. becomes:: 2013-09-15 22:02:18+0200 [-] {"event": "Log opened.", "system": "-"} There is obviously some redundancy here. Also, I'm presuming that if you write out JSON logs, you're going to let something else parse them which makes the human-readable date entries more trouble than they're worth. To get a clean log without timestamps and additional system fields (``[-]``), structlog comes with :class:`~structlog.twisted.PlainFileLogObserver` that writes only the plain message to a file and :func:`~structlog.twisted.plainJSONStdOutLogger` that composes it with the aforementioned :func:`~structlog.twisted.JSONLogObserverWrapper` and gives you a pure JSON log without any timestamps or other noise straight to `standard out`_:: $ twistd -n --logger structlog.twisted.plainJSONStdOutLogger web {"event": "Log opened.", "system": "-"} {"event": "twistd 13.1.0 (python 2.7.3) starting up.", "system": "-"} {"event": "reactor class: twisted...EPollReactor.", "system": "-"} {"event": "Site starting on 8080", "system": "-"} {"event": "Starting factory ", ...} ... Suggested Configuration ----------------------- :: import structlog structlog.configure( processors=[ structlog.processors.StackInfoRenderer(), structlog.twisted.JSONRenderer() ], context_class=dict, logger_factory=structlog.twisted.LoggerFactory(), wrapper_class=structlog.twisted.BoundLogger, cache_logger_on_first_use=True, ) See also :doc:`logging-best-practices`. .. _`standard out`: http://en.wikipedia.org/wiki/Standard_out#Standard_output_.28stdout.29 structlog-15.2.0/docs/why.rst0000644000076500000240000000420712455673423016375 0ustar hynekstaff00000000000000Why… ==== …Structured Logging? --------------------- I believe the widespread use of format strings in logging is based on two presumptions: - The first level consumer of a log message is a human. - The programmer knows what information is needed to debug an issue. I believe these presumptions are **no longer correct** in server side software. ---`Paul Querna `_ Structured logging means that you don't write hard-to-parse and hard-to-keep-consistent prose in your logs but that you log *events* that happen in a *context* instead. …structlog? ------------ Because it's easy and you don't have to replace your underlying logger -- you just add structure to your log entries and format them to strings before they hit your real loggers. structlog supports you with building your context as you go (e.g. if a user logs in, you bind their user name to your current logger) and log events when they happen (i.e. the user does something log-worthy): .. doctest:: >>> from structlog import get_logger >>> log = get_logger() >>> log = log.bind(user='anonymous', some_key=23) >>> log = log.bind(user='hynek', another_key=42) >>> log.info('user.logged_in', happy=True) some_key=23 user='hynek' another_key=42 happy=True event='user.logged_in' This ability to bind key/values pairs to a logger frees you from using conditionals, closures, or boilerplate methods to log out all relevant data. Additionally, structlog offers you a flexible way to *filter* and *modify* your log entries using so called :ref:`processors ` before the entry is passed to your real logger. The possibilities include :class:`logging in JSON `, adding arbitrary meta data like :class:`timestamps `, counting events as metrics, or :ref:`dropping log entries ` caused by your monitoring system. structlog is also flexible enough to allow transparent :ref:`thread local ` storage for your context if you don't like the idea of local bindings as in the example above. structlog-15.2.0/docs-requirements.txt0000644000076500000240000000005612364716307020312 0ustar hynekstaff00000000000000-e . releases sphinx sphinx_rtd_theme twisted structlog-15.2.0/LICENSE0000644000076500000240000000030312432430627015072 0ustar hynekstaff00000000000000This software is made available under the terms of *either* of the licenses found in LICENSE.apache or LICENSE.mit. Contributions to structlog are made under the terms of *both* these licenses. structlog-15.2.0/LICENSE.apache20000644000076500000240000002367612431374731016420 0ustar hynekstaff00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS structlog-15.2.0/LICENSE.mit0000644000076500000240000000207712455675524015710 0ustar hynekstaff00000000000000The MIT License (MIT) Copyright (c) 2013-2015 Hynek Schlawack Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. structlog-15.2.0/MANIFEST.in0000644000076500000240000000053012432432063015621 0ustar hynekstaff00000000000000include LICENSE LICENSE.apache2 LICENSE.mit .coveragerc include *.rst include *.txt include .travis.yml include tox.ini recursive-include docs *.bat recursive-include docs *.keep recursive-include docs *.py recursive-include docs *.rst recursive-include docs *.txt recursive-include docs Makefile recursive-include tests *.py prune docs/_build structlog-15.2.0/PKG-INFO0000644000076500000240000000550312536025152015167 0ustar hynekstaff00000000000000Metadata-Version: 1.1 Name: structlog Version: 15.2.0 Summary: Structured logging for Python. Home-page: http://www.structlog.org/ Author: Hynek Schlawack Author-email: hs@ox.cx License: MIT or Apache License, Version 2.0 Description: ======================================== structlog: Structured Logging for Python ======================================== .. image:: https://pypip.in/version/structlog/badge.svg :target: https://pypi.python.org/pypi/structlog/ :alt: Latest Version .. image:: https://travis-ci.org/hynek/structlog.svg?branch=master :target: https://travis-ci.org/hynek/structlog .. image:: https://coveralls.io/repos/hynek/structlog/badge.svg?branch=master :target: https://coveralls.io/r/hynek/structlog?branch=master ``structlog`` makes structured logging in Python easy by *augmenting* your *existing* logger. It allows you to split your log entries up into key/value pairs and build them incrementally without annoying boilerplate code. .. code-block:: pycon >>> from structlog import get_logger >>> log = get_logger() >>> log = log.bind(user='anonymous', some_key=23) >>> log = log.bind(user='hynek', another_key=42) >>> log.info('user.logged_in', happy=True) some_key=23 user='hynek' another_key=42 happy=True event='user.logged_in' .. begin It's dual-licensed under `Apache License, version 2 `_ and `MIT `_, available from `PyPI `_, the source code can be found on `GitHub `_, the documentation at `http://www.structlog.org/ `_. ``structlog`` targets Python 2.6, 2.7, 3.3, 3.4, and PyPy with no additional dependencies for core functionality. If you need any help, visit us on ``#structlog`` on `Freenode `_! Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: Apache Software License Classifier: License :: OSI Approved :: MIT License Classifier: Natural Language :: English Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python Classifier: Topic :: Software Development :: Libraries :: Python Modules structlog-15.2.0/README.rst0000644000076500000240000000316412536007746015573 0ustar hynekstaff00000000000000======================================== structlog: Structured Logging for Python ======================================== .. image:: https://pypip.in/version/structlog/badge.svg :target: https://pypi.python.org/pypi/structlog/ :alt: Latest Version .. image:: https://travis-ci.org/hynek/structlog.svg?branch=master :target: https://travis-ci.org/hynek/structlog .. image:: https://coveralls.io/repos/hynek/structlog/badge.svg?branch=master :target: https://coveralls.io/r/hynek/structlog?branch=master ``structlog`` makes structured logging in Python easy by *augmenting* your *existing* logger. It allows you to split your log entries up into key/value pairs and build them incrementally without annoying boilerplate code. .. code-block:: pycon >>> from structlog import get_logger >>> log = get_logger() >>> log = log.bind(user='anonymous', some_key=23) >>> log = log.bind(user='hynek', another_key=42) >>> log.info('user.logged_in', happy=True) some_key=23 user='hynek' another_key=42 happy=True event='user.logged_in' .. begin It's dual-licensed under `Apache License, version 2 `_ and `MIT `_, available from `PyPI `_, the source code can be found on `GitHub `_, the documentation at `http://www.structlog.org/ `_. ``structlog`` targets Python 2.6, 2.7, 3.3, 3.4, and PyPy with no additional dependencies for core functionality. If you need any help, visit us on ``#structlog`` on `Freenode `_! structlog-15.2.0/setup.cfg0000644000076500000240000000024412536025152015710 0ustar hynekstaff00000000000000[pytest] minversion = 2.6 strict = true norecursedirs = build dist .* *.egg docs [wheel] universal = 1 [egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 structlog-15.2.0/setup.py0000644000076500000240000000534512522576074015621 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. from setuptools import setup, find_packages from setuptools.command.test import test as TestCommand import codecs import os import re import sys here = os.path.abspath(os.path.dirname(__file__)) def read(*parts): with codecs.open(os.path.join(here, *parts), 'r') as f: return f.read() def find_version(*file_paths): version_file = read(*file_paths) version_match = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]", version_file, re.M) if version_match: return version_match.group(1) raise RuntimeError("Unable to find version string.") class PyTest(TestCommand): user_options = [('pytest-args=', 'a', "Arguments to pass to py.test")] def initialize_options(self): TestCommand.initialize_options(self) self.pytest_args = None def finalize_options(self): TestCommand.finalize_options(self) self.test_args = [] self.test_suite = True def run_tests(self): # import here, cause outside the eggs aren't loaded import pytest errno = pytest.main(self.pytest_args or [] + ["tests"]) sys.exit(errno) if __name__ == "__main__": setup( name='structlog', version=find_version('structlog', '__init__.py'), description='Structured logging for Python.', long_description=read('README.rst'), url='http://www.structlog.org/', license='MIT or Apache License, Version 2.0', author='Hynek Schlawack', author_email='hs@ox.cx', packages=find_packages(exclude=['tests*']), classifiers=[ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', 'License :: OSI Approved :: Apache Software License', 'License :: OSI Approved :: MIT License', 'Natural Language :: English', 'Operating System :: OS Independent', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.6', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python', 'Topic :: Software Development :: Libraries :: Python Modules', ], tests_require=[ "freezegun>=0.2.8", "pretend", "pytest", "twisted", ], cmdclass={ "test": PyTest, }, ) structlog-15.2.0/structlog/0000755000076500000240000000000012536025152016115 5ustar hynekstaff00000000000000structlog-15.2.0/structlog/__init__.py0000644000076500000240000000276112536006171020234 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Structured logging for Python. """ from __future__ import absolute_import, division, print_function from structlog import ( processors, stdlib, threadlocal, ) from structlog._base import ( BoundLoggerBase, ) from structlog._config import ( configure, configure_once, getLogger, get_logger, reset_defaults, wrap_logger, ) from structlog._exc import ( DropEvent, ) from structlog._generic import ( BoundLogger ) from structlog._loggers import ( PrintLogger, PrintLoggerFactory, ReturnLogger, ReturnLoggerFactory, ) try: from structlog import twisted except ImportError: # pragma: nocover twisted = None __version__ = "15.2.0" __title__ = "structlog" __description__ = "Structured Logging in Python" __uri__ = "http://www.structlog.org/" __author__ = "Hynek Schlawack" __email__ = "hs@ox.cx" __license__ = "MIT or Apache License, Version 2.0" __copyright__ = "Copyright (c) 2013-2015 {0}".format(__author__) __all__ = [ 'BoundLogger', 'BoundLoggerBase', 'DropEvent', 'PrintLogger', 'PrintLoggerFactory', 'ReturnLogger', 'ReturnLoggerFactory', 'configure', 'configure_once', 'getLogger', 'get_logger', 'processors', 'reset_defaults', 'stdlib', 'threadlocal', 'twisted', 'wrap_logger', ] structlog-15.2.0/structlog/_base.py0000644000076500000240000001337512516165577017567 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Logger wrapper and helper class. """ from __future__ import absolute_import, division, print_function from structlog._compat import string_types from structlog._exc import DropEvent class BoundLoggerBase(object): """ Immutable context carrier. Doesn't do any actual logging; examples for useful subclasses are: - the generic :class:`BoundLogger` that can wrap anything, - :class:`structlog.twisted.BoundLogger`, - and :class:`structlog.stdlib.BoundLogger`. See also :doc:`custom-wrappers`. """ _logger = None """ Wrapped logger. .. note:: Despite underscore available **read-only** to custom wrapper classes. See also :doc:`custom-wrappers`. """ def __init__(self, logger, processors, context): self._logger = logger self._processors = processors self._context = context def __repr__(self): return '<{0}(context={1!r}, processors={2!r})>'.format( self.__class__.__name__, self._context, self._processors, ) def __eq__(self, other): try: if self._context == other._context: return True else: return False except AttributeError: return False def __ne__(self, other): return not self.__eq__(other) def bind(self, **new_values): """ Return a new logger with *new_values* added to the existing ones. :rtype: `self.__class__` """ return self.__class__( self._logger, self._processors, self._context.__class__(self._context, **new_values) ) def unbind(self, *keys): """ Return a new logger with *keys* removed from the context. :raises KeyError: If the key is not part of the context. :rtype: `self.__class__` """ bl = self.bind() for key in keys: del bl._context[key] return bl def new(self, **new_values): """ Clear context and binds *initial_values* using :func:`bind`. Only necessary with dict implementations that keep global state like those wrapped by :func:`structlog.threadlocal.wrap_dict` when threads are re-used. :rtype: `self.__class__` """ self._context.clear() return self.bind(**new_values) # Helper methods for sub-classing concrete BoundLoggers. def _process_event(self, method_name, event, event_kw): """ Combines creates an `event_dict` and runs the chain. Call it to combine your *event* and *context* into an event_dict and process using the processor chain. :param str method_name: The name of the logger method. Is passed into the processors. :param event: The event -- usually the first positional argument to a logger. :param event_kw: Additional event keywords. For example if someone calls ``log.msg('foo', bar=42)``, *event* would to be ``'foo'`` and *event_kw* ``{'bar': 42}``. :raises: :class:`structlog.DropEvent` if log entry should be dropped. :raises: :class:`ValueError` if the final processor doesn't return a string, tuple, or a dict. :rtype: `tuple` of `(*args, **kw)` .. note:: Despite underscore available to custom wrapper classes. See also :doc:`custom-wrappers`. .. versionchanged:: 14.0.0 Allow final processor to return a `dict`. """ event_dict = self._context.copy() event_dict.update(**event_kw) if event: event_dict['event'] = event for proc in self._processors: event_dict = proc(self._logger, method_name, event_dict) if isinstance(event_dict, string_types): return (event_dict,), {} elif isinstance(event_dict, tuple): # In this case we assume that the last processor returned a tuple # of ``(args, kwargs)`` and pass it right through. return event_dict elif isinstance(event_dict, dict): return (), event_dict else: raise ValueError( "Last processor didn't return an approriate value. Allowed " "return values are a dict, a tuple of (args, kwargs), or a " "string." ) def _proxy_to_logger(self, method_name, event=None, **event_kw): """ Run processor chain on event & call *method_name* on wrapped logger. DRY convenience method that runs :func:`_process_event`, takes care of handling :exc:`structlog.DropEvent`, and finally calls *method_name* on :attr:`_logger` with the result. :param str method_name: The name of the method that's going to get called. Technically it should be identical to the method the user called because it also get passed into processors. :param event: The event -- usually the first positional argument to a logger. :param event_kw: Additional event keywords. For example if someone calls ``log.msg('foo', bar=42)``, *event* would to be ``'foo'`` and *event_kw* ``{'bar': 42}``. .. note:: Despite underscore available to custom wrapper classes. See also :doc:`custom-wrappers`. """ try: args, kw = self._process_event(method_name, event, event_kw) return getattr(self._logger, method_name)(*args, **kw) except DropEvent: return structlog-15.2.0/structlog/_compat.py0000644000076500000240000000302212522604512020104 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Python 2 + 3 compatibility utilities. Derived from MIT-licensed https://bitbucket.org/gutworth/six/ which is Copyright 2010-2013 by Benjamin Peterson. """ from __future__ import absolute_import, division, print_function import abc import sys import types PY2 = sys.version_info[0] == 2 PY3 = sys.version_info[0] == 3 try: from cStringIO import StringIO except ImportError: from io import StringIO # flake8: noqa if sys.version_info[:2] == (2, 6): try: from ordereddict import OrderedDict except ImportError: class OrderedDict(object): def __init__(self, *args, **kw): raise NotImplementedError( 'The ordereddict package is needed on Python 2.6. ' 'See .' ) else: from collections import OrderedDict if PY3: string_types = str, integer_types = int, class_types = type, text_type = str binary_type = bytes unicode_type = str else: string_types = basestring, integer_types = (int, long) class_types = (type, types.ClassType) text_type = unicode binary_type = str unicode_type = unicode def with_metaclass(meta, *bases): """ Create a base class with a metaclass. """ return meta("NewBase", bases, {}) structlog-15.2.0/structlog/_config.py0000644000076500000240000002413012516144575020104 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Global state department. Don't reload this module or everything breaks. """ from __future__ import absolute_import, division, print_function import warnings from structlog._compat import OrderedDict from structlog._generic import BoundLogger from structlog._loggers import ( PrintLoggerFactory, ) from structlog.processors import ( KeyValueRenderer, StackInfoRenderer, format_exc_info, ) _BUILTIN_DEFAULT_PROCESSORS = [ StackInfoRenderer(), format_exc_info, KeyValueRenderer() ] _BUILTIN_DEFAULT_CONTEXT_CLASS = OrderedDict _BUILTIN_DEFAULT_WRAPPER_CLASS = BoundLogger _BUILTIN_DEFAULT_LOGGER_FACTORY = PrintLoggerFactory() _BUILTIN_CACHE_LOGGER_ON_FIRST_USE = False class _Configuration(object): """ Global defaults. """ is_configured = False default_processors = _BUILTIN_DEFAULT_PROCESSORS[:] default_context_class = _BUILTIN_DEFAULT_CONTEXT_CLASS default_wrapper_class = _BUILTIN_DEFAULT_WRAPPER_CLASS logger_factory = _BUILTIN_DEFAULT_LOGGER_FACTORY cache_logger_on_first_use = _BUILTIN_CACHE_LOGGER_ON_FIRST_USE _CONFIG = _Configuration() """ Global defaults used when arguments to :func:`wrap_logger` are omitted. """ def get_logger(*args, **initial_values): """ Convenience function that returns a logger according to configuration. >>> from structlog import get_logger >>> log = get_logger(y=23) >>> log.msg('hello', x=42) y=23 x=42 event='hello' :param args: *Optional* positional arguments that are passed unmodified to the logger factory. Therefore it depends on the factory what they mean. :param initial_values: Values that are used to pre-populate your contexts. :rtype: A proxy that creates a correctly configured bound logger when necessary. See :ref:`configuration` for details. If you prefer CamelCase, there's an alias for your reading pleasure: :func:`structlog.getLogger`. .. versionadded:: 0.4.0 `args` """ return wrap_logger(None, logger_factory_args=args, **initial_values) getLogger = get_logger """ CamelCase alias for :func:`structlog.get_logger`. This function is supposed to be in every source file -- we don't want it to stick out like a sore thumb in frameworks like Twisted or Zope. """ def wrap_logger(logger, processors=None, wrapper_class=None, context_class=None, cache_logger_on_first_use=None, logger_factory_args=None, **initial_values): """ Create a new bound logger for an arbitrary *logger*. Default values for *processors*, *wrapper_class*, and *context_class* can be set using :func:`configure`. If you set an attribute here, :func:`configure` calls have *no* effect for the *respective* attribute. In other words: selective overwriting of the defaults while keeping some *is* possible. :param initial_values: Values that are used to pre-populate your contexts. :param tuple logger_factory_args: Values that are passed unmodified as ``*logger_factory_args`` to the logger factory if not `None`. :rtype: A proxy that creates a correctly configured bound logger when necessary. See :func:`configure` for the meaning of the rest of the arguments. .. versionadded:: 0.4.0 `logger_factory_args` """ return BoundLoggerLazyProxy( logger, wrapper_class=wrapper_class, processors=processors, context_class=context_class, cache_logger_on_first_use=cache_logger_on_first_use, initial_values=initial_values, logger_factory_args=logger_factory_args, ) def configure(processors=None, wrapper_class=None, context_class=None, logger_factory=None, cache_logger_on_first_use=None): """ Configures the **global** defaults. They are used if :func:`wrap_logger` has been called without arguments. Also sets the global class attribute :attr:`is_configured` to `True` on first call. Can be called several times, keeping an argument at `None` leaves is unchanged from the current setting. Use :func:`reset_defaults` to undo your changes. :param list processors: List of processors. :param type wrapper_class: Class to use for wrapping loggers instead of :class:`structlog.BoundLogger`. See :doc:`standard-library`, :doc:`twisted`, and :doc:`custom-wrappers`. :param type context_class: Class to be used for internal context keeping. :param callable logger_factory: Factory to be called to create a new logger that shall be wrapped. :param bool cache_logger_on_first_use: `wrap_logger` doesn't return an actual wrapped logger but a proxy that assembles one when it's first used. If this option is set to `True`, this assembled logger is cached. See :doc:`performance`. .. versionadded:: 0.3.0 `cache_logger_on_first_use` """ _CONFIG.is_configured = True if processors is not None: _CONFIG.default_processors = processors if wrapper_class: _CONFIG.default_wrapper_class = wrapper_class if context_class: _CONFIG.default_context_class = context_class if logger_factory: _CONFIG.logger_factory = logger_factory if cache_logger_on_first_use is not None: _CONFIG.cache_logger_on_first_use = cache_logger_on_first_use def configure_once(*args, **kw): """ Configures iff structlog isn't configured yet. It does *not* matter whether is was configured using :func:`configure` or :func:`configure_once` before. Raises a RuntimeWarning if repeated configuration is attempted. """ if not _CONFIG.is_configured: configure(*args, **kw) else: warnings.warn('Repeated configuration attempted.', RuntimeWarning) def reset_defaults(): """ Resets global default values to builtins. That means [:class:`~structlog.processors.StackInfoRenderer`, :func:`~structlog.processors.format_exc_info`, :class:`~structlog.processors.KeyValueRenderer`] for *processors*, :class:`~structlog.BoundLogger` for *wrapper_class*, ``OrderedDict`` for *context_class*, :class:`~structlog.PrintLoggerFactory` for *logger_factory*, and `False` for *cache_logger_on_first_use*. Also sets the global class attribute :attr:`is_configured` to `False`. """ _CONFIG.is_configured = False _CONFIG.default_processors = _BUILTIN_DEFAULT_PROCESSORS[:] _CONFIG.default_wrapper_class = _BUILTIN_DEFAULT_WRAPPER_CLASS _CONFIG.default_context_class = _BUILTIN_DEFAULT_CONTEXT_CLASS _CONFIG.logger_factory = _BUILTIN_DEFAULT_LOGGER_FACTORY _CONFIG.cache_logger_on_first_use = _BUILTIN_CACHE_LOGGER_ON_FIRST_USE class BoundLoggerLazyProxy(object): """ Instantiates a BoundLogger on first usage. Takes both configuration and instantiation parameters into account. The only points where a BoundLogger changes state are bind(), unbind(), and new() and that return the actual BoundLogger. If and only if configuration says so, that actual BoundLogger is cached on first usage. .. versionchanged:: 0.4.0 Added support for `logger_factory_args`. """ def __init__(self, logger, wrapper_class=None, processors=None, context_class=None, cache_logger_on_first_use=None, initial_values=None, logger_factory_args=None): self._logger = logger self._wrapper_class = wrapper_class self._processors = processors self._context_class = context_class self._cache_logger_on_first_use = cache_logger_on_first_use self._initial_values = initial_values or {} self._logger_factory_args = logger_factory_args or () def __repr__(self): return ( ''.format(self) ) def bind(self, **new_values): """ Assemble a new BoundLogger from arguments and configuration. """ if self._context_class: ctx = self._context_class(self._initial_values) else: ctx = _CONFIG.default_context_class(self._initial_values) cls = self._wrapper_class or _CONFIG.default_wrapper_class if not self._logger: self._logger = _CONFIG.logger_factory(*self._logger_factory_args) if self._processors is None: procs = _CONFIG.default_processors else: procs = self._processors logger = cls( self._logger, processors=procs, context=ctx, ) def finalized_bind(**new_values): """ Use cached assembled logger to bind potentially new values. """ if new_values: return logger.bind(**new_values) else: return logger if ( self._cache_logger_on_first_use is True or (self._cache_logger_on_first_use is None and _CONFIG.cache_logger_on_first_use is True) ): self.bind = finalized_bind return finalized_bind(**new_values) def unbind(self, *keys): """ Same as bind, except unbind *keys* first. In our case that could be only initial values. """ return self.bind().unbind(*keys) def new(self, **new_values): """ Clear context, then bind. """ if self._context_class: self._context_class().clear() else: _CONFIG.default_context_class().clear() bl = self.bind(**new_values) return bl def __getattr__(self, name): """ If a logging method if called on a lazy proxy, we have to create an ephemeral BoundLogger first. """ bl = self.bind() return getattr(bl, name) structlog-15.2.0/structlog/_exc.py0000644000076500000240000000064612432431173017412 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Exceptions factored out to avoid import loops. """ class DropEvent(BaseException): """ If raised by an processor, the event gets silently dropped. Derives from BaseException because it's technically not an error. """ structlog-15.2.0/structlog/_frames.py0000644000076500000240000000306612522570240020106 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. from __future__ import absolute_import, division, print_function import sys import traceback from structlog._compat import StringIO def _format_exception(exc_info): """ Prettyprint an `exc_info` tuple. Shamelessly stolen from stdlib's logging module. """ sio = StringIO() traceback.print_exception(exc_info[0], exc_info[1], exc_info[2], None, sio) s = sio.getvalue() sio.close() if s[-1:] == "\n": s = s[:-1] return s def _find_first_app_frame_and_name(additional_ignores=None): """ Remove all intra-structlog calls and return the relevant app frame. :param additional_ignores: Additional names with which the first frame must not start. :type additional_ignores: `list` of `str` or `None` :rtype: tuple of (frame, name) """ ignores = ["structlog"] + (additional_ignores or []) f = sys._getframe() name = f.f_globals.get("__name__", "?") while any(name.startswith(i) for i in ignores): f = f.f_back name = f.f_globals.get("__name__", "?") return f, name def _format_stack(frame): """ Pretty-print the stack of `frame` like logging would. """ sio = StringIO() sio.write('Stack (most recent call last):\n') traceback.print_stack(frame, file=sio) sinfo = sio.getvalue() if sinfo[-1] == '\n': sinfo = sinfo[:-1] sio.close() return sinfo structlog-15.2.0/structlog/_generic.py0000644000076500000240000000223112432431215020234 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Generic bound logger that can wrap anything. """ from __future__ import absolute_import, division, print_function from functools import partial from structlog._base import BoundLoggerBase class BoundLogger(BoundLoggerBase): """ A generic BoundLogger that can wrap anything. Every unknown method will be passed to the wrapped logger. If that's too much magic for you, try :class:`structlog.twisted.BoundLogger` or `:class:`structlog.twisted.BoundLogger` which also take advantage of knowing the wrapped class which generally results in better performance. Not intended to be instantiated by yourself. See :func:`~structlog.wrap_logger` and :func:`~structlog.get_logger`. """ def __getattr__(self, method_name): """ If not done so yet, wrap the desired logger method & cache the result. """ wrapped = partial(self._proxy_to_logger, method_name) setattr(self, method_name, wrapped) return wrapped structlog-15.2.0/structlog/_loggers.py0000644000076500000240000000624712501601331020270 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Logger wrapper and helper class. """ from __future__ import absolute_import, division, print_function import sys import threading from structlog._utils import until_not_interrupted class PrintLoggerFactory(object): """ Produces :class:`PrintLogger`\ s. To be used with :func:`structlog.configure`\ 's `logger_factory`. :param file file: File to print to. (default: stdout) Positional arguments are silently ignored. .. versionadded:: 0.4.0 """ def __init__(self, file=None): self._file = file def __call__(self, *args): return PrintLogger(self._file) WRITE_LOCKS = {} class PrintLogger(object): """ Prints events into a file. :param file file: File to print to. (default: stdout) >>> from structlog import PrintLogger >>> PrintLogger().msg('hello') hello Useful if you just capture your stdout with tools like `runit `_ or if you `forward your stderr to syslog `_. Also very useful for testing and examples since logging is finicky in doctests. """ def __init__(self, file=None): self._file = file or sys.stdout self._write = self._file.write self._flush = self._file.flush lock = WRITE_LOCKS.get(self._file) if lock is None: lock = threading.Lock() WRITE_LOCKS[self._file] = lock self._lock = lock def __repr__(self): return ''.format(self._file) def msg(self, message): """ Print *message*. """ with self._lock: until_not_interrupted(self._write, message + '\n') until_not_interrupted(self._flush) log = debug = info = warn = warning = msg err = error = critical = exception = msg class ReturnLoggerFactory(object): """ Produces and caches :class:`ReturnLogger`\ s. To be used with :func:`structlog.configure`\ 's `logger_factory`. Positional arguments are silently ignored. .. versionadded:: 0.4.0 """ def __init__(self): self._logger = ReturnLogger() def __call__(self, *args): return self._logger class ReturnLogger(object): """ Returns the string that it's called with. >>> from structlog import ReturnLogger >>> ReturnLogger().msg('hello') 'hello' >>> ReturnLogger().msg('hello', when='again') (('hello',), {'when': 'again'}) Useful for unit tests. .. versionchanged:: 0.3.0 Allow for arbitrary arguments and keyword arguments to be passed in. """ def msg(self, *args, **kw): """ Return tuple of ``args, kw`` or just ``args[0]`` if only one arg passed """ # Slightly convoluted for backwards compatibility. if len(args) == 1 and not kw: return args[0] else: return args, kw log = debug = info = warn = warning = msg err = error = critical = exception = msg structlog-15.2.0/structlog/_utils.py0000644000076500000240000000137112432431240017762 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Generic utilities. """ from __future__ import absolute_import, division, print_function import errno def until_not_interrupted(f, *args, **kw): """ Retry until *f* succeeds or an exception that isn't caused by EINTR occurs. :param callable f: A callable like a function. :param *args: Positional arguments for *f*. :param **kw: Keyword arguments for *f*. """ while True: try: return f(*args, **kw) except (IOError, OSError) as e: if e.args[0] == errno.EINTR: continue raise structlog-15.2.0/structlog/processors.py0000644000076500000240000002430612522602744020701 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Processors useful regardless of the logging framework. """ from __future__ import absolute_import, division, print_function import calendar import datetime import json import operator import sys import time from structlog._compat import unicode_type from structlog._frames import ( _find_first_app_frame_and_name, _format_exception, _format_stack, ) class KeyValueRenderer(object): """ Render `event_dict` as a list of ``Key=repr(Value)`` pairs. :param bool sort_keys: Whether to sort keys when formatting. :param list key_order: List of keys that should be rendered in this exact order. Missing keys will be rendered as `None`, extra keys depending on *sort_keys* and the dict class. >>> from structlog.processors import KeyValueRenderer >>> KeyValueRenderer(sort_keys=True)(None, None, {'a': 42, 'b': [1, 2, 3]}) 'a=42 b=[1, 2, 3]' >>> KeyValueRenderer(key_order=['b', 'a'])(None, None, ... {'a': 42, 'b': [1, 2, 3]}) 'b=[1, 2, 3] a=42' .. versionadded:: 0.2.0 `key_order` """ def __init__(self, sort_keys=False, key_order=None): # Use an optimized version for each case. if key_order and sort_keys: def ordered_items(event_dict): items = [] for key in key_order: value = event_dict.pop(key, None) items.append((key, value)) items += sorted(event_dict.items()) return items elif key_order: def ordered_items(event_dict): items = [] for key in key_order: value = event_dict.pop(key, None) items.append((key, value)) items += event_dict.items() return items elif sort_keys: def ordered_items(event_dict): return sorted(event_dict.items()) else: ordered_items = operator.methodcaller('items') self._ordered_items = ordered_items def __call__(self, _, __, event_dict): return ' '.join(k + '=' + repr(v) for k, v in self._ordered_items(event_dict)) class UnicodeEncoder(object): """ Encode unicode values in `event_dict`. :param str encoding: Encoding to encode to (default: ``'utf-8'``. :param str errors: How to cope with encoding errors (default ``'backslashreplace'``). Useful for :class:`KeyValueRenderer` if you don't want to see u-prefixes: >>> from structlog.processors import KeyValueRenderer, UnicodeEncoder >>> KeyValueRenderer()(None, None, {'foo': u'bar'}) "foo=u'bar'" >>> KeyValueRenderer()(None, None, ... UnicodeEncoder()(None, None, {'foo': u'bar'})) "foo='bar'" or :class:`JSONRenderer` and :class:`structlog.twisted.JSONRenderer` to make sure user-supplied strings don't break the renderer. Just put it in the processor chain before the renderer. """ def __init__(self, encoding='utf-8', errors='backslashreplace'): self._encoding = encoding self._errors = errors def __call__(self, logger, name, event_dict): for key, value in event_dict.items(): if isinstance(value, unicode_type): event_dict[key] = value.encode(self._encoding, self._errors) return event_dict class JSONRenderer(object): """ Render the `event_dict` using `json.dumps(event_dict, **json_kw)`. :param json_kw: Are passed unmodified to `json.dumps()`. >>> from structlog.processors import JSONRenderer >>> JSONRenderer(sort_keys=True)(None, None, {'a': 42, 'b': [1, 2, 3]}) '{"a": 42, "b": [1, 2, 3]}' Bound objects are attempted to be serialize using a ``__structlog__`` method. If none is defined, ``repr()`` is used: >>> class C1(object): ... def __structlog__(self): ... return ['C1!'] ... def __repr__(self): ... return '__structlog__ took precedence' >>> class C2(object): ... def __repr__(self): ... return 'No __structlog__, so this is used.' >>> from structlog.processors import JSONRenderer >>> JSONRenderer(sort_keys=True)(None, None, {'c1': C1(), 'c2': C2()}) '{"c1": ["C1!"], "c2": "No __structlog__, so this is used."}' Please note that additionally to strings, you can also return any type the standard library JSON module knows about -- like in this example a list. .. versionchanged:: 0.2.0 Added support for ``__structlog__`` serialization method. """ def __init__(self, **dumps_kw): self._dumps_kw = dumps_kw def __call__(self, logger, name, event_dict): return json.dumps(event_dict, cls=_JSONFallbackEncoder, **self._dumps_kw) class _JSONFallbackEncoder(json.JSONEncoder): """ Serialize custom datatypes and pass the rest to __structlog__ & repr(). """ def default(self, obj): """ Serialize obj with repr(obj) as fallback. """ # circular imports :( from structlog.threadlocal import _ThreadLocalDictWrapper if isinstance(obj, _ThreadLocalDictWrapper): return obj._dict else: try: return obj.__structlog__() except AttributeError: return repr(obj) def format_exc_info(logger, name, event_dict): """ Replace an `exc_info` field by an `exception` string field: If *event_dict* contains the key ``exc_info``, there are two possible behaviors: - If the value is a tuple, render it into the key ``exception``. - If the value true but no tuple, obtain exc_info ourselves and render that. If there is no ``exc_info`` key, the *event_dict* is not touched. This behavior is analogue to the one of the stdlib's logging. >>> from structlog.processors import format_exc_info >>> try: ... raise ValueError ... except ValueError: ... format_exc_info(None, None, {'exc_info': True})# doctest: +ELLIPSIS {'exception': 'Traceback (most recent call last):... """ exc_info = event_dict.pop('exc_info', None) if exc_info: if not isinstance(exc_info, tuple): exc_info = sys.exc_info() event_dict['exception'] = _format_exception(exc_info) return event_dict class TimeStamper(object): """ Add a timestamp to `event_dict`. .. note:: You probably want to let OS tools take care of timestamping. See also :doc:`logging-best-practices`. :param str format: strftime format string, or ``"iso"`` for `ISO 8601 `_, or `None` for a `UNIX timestamp `_. :param bool utc: Whether timestamp should be in UTC or local time. :param str key: Target key in `event_dict` for added timestamps. >>> from structlog.processors import TimeStamper >>> TimeStamper()(None, None, {}) # doctest: +SKIP {'timestamp': 1378994017} >>> TimeStamper(fmt='iso')(None, None, {}) # doctest: +SKIP {'timestamp': '2013-09-12T13:54:26.996778Z'} >>> TimeStamper(fmt='%Y', key='year')(None, None, {}) # doctest: +SKIP {'year': '2013'} """ def __new__(cls, fmt=None, utc=True, key='timestamp'): if fmt is None and not utc: raise ValueError('UNIX timestamps are always UTC.') now_method = getattr(datetime.datetime, 'utcnow' if utc else 'now') if fmt is None: def stamper(self, _, __, event_dict): event_dict[key] = calendar.timegm(time.gmtime()) return event_dict elif fmt.upper() == 'ISO': if utc: def stamper(self, _, __, event_dict): event_dict[key] = now_method().isoformat() + 'Z' return event_dict else: def stamper(self, _, __, event_dict): event_dict[key] = now_method().isoformat() return event_dict else: def stamper(self, _, __, event_dict): event_dict[key] = now_method().strftime(fmt) return event_dict return type('TimeStamper', (object,), {'__call__': stamper})() class ExceptionPrettyPrinter(object): """ Pretty print exceptions and remove them from the `event_dict`. :param file file: Target file for output (default: `sys.stdout`). This processor is mostly for development and testing so you can read exceptions properly formatted. It behaves like :func:`format_exc_info` except it removes the exception data from the event dictionary after printing it. It's tolerant to having `format_exc_info` in front of itself in the processor chain but doesn't require it. In other words, it handles both `exception` as well as `exc_info` keys. .. versionadded:: 0.4.0 """ def __init__(self, file=None): if file is not None: self._file = file else: self._file = sys.stdout def __call__(self, logger, name, event_dict): exc = event_dict.pop('exception', None) if exc is None: exc_info = event_dict.pop('exc_info', None) if exc_info: if not isinstance(exc_info, tuple): exc_info = sys.exc_info() exc = _format_exception(exc_info) if exc: print(exc, file=self._file) return event_dict class StackInfoRenderer(object): """ Add stack information with key `stack` if `stack_info` is true. Useful when you want to attach a stack dump to a log entry without involving an exception. It works analogously to the `stack_info` argument of the Python 3 standard library logging but works on both 2 and 3. .. versionadded:: 0.4.0 """ def __call__(self, logger, name, event_dict): if event_dict.pop('stack_info', None): event_dict['stack'] = _format_stack( _find_first_app_frame_and_name()[0] ) return event_dict structlog-15.2.0/structlog/stdlib.py0000644000076500000240000002626612501224114017753 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Processors and helpers specific to the :mod:`logging` module from the `Python standard library `_. See also :doc:`structlog's standard library support `. """ from __future__ import absolute_import, division, print_function import logging from structlog._base import BoundLoggerBase from structlog._compat import PY3 from structlog._exc import DropEvent from structlog._frames import _find_first_app_frame_and_name, _format_stack class _FixedFindCallerLogger(logging.Logger): """ Change the behavior of findCaller to cope with structlog's extra frames. """ def findCaller(self, stack_info=False): """ Finds the first caller frame outside of structlog so that the caller info is populated for wrapping stdlib. This logger gets set as the default one when using LoggerFactory. """ f, name = _find_first_app_frame_and_name(['logging']) if PY3: # pragma: nocover if stack_info: sinfo = _format_stack(f) else: sinfo = None return f.f_code.co_filename, f.f_lineno, f.f_code.co_name, sinfo else: return f.f_code.co_filename, f.f_lineno, f.f_code.co_name class BoundLogger(BoundLoggerBase): """ Python Standard Library version of :class:`structlog.BoundLogger`. Works exactly like the generic one except that it takes advantage of knowing the logging methods in advance. Use it like:: structlog.configure( wrapper_class=structlog.stdlib.BoundLogger, ) """ def debug(self, event=None, *args, **kw): """ Process event and call :meth:`logging.Logger.debug` with the result. """ return self._proxy_to_logger('debug', event, *args, **kw) def info(self, event=None, *args, **kw): """ Process event and call :meth:`logging.Logger.info` with the result. """ return self._proxy_to_logger('info', event, *args, **kw) def warning(self, event=None, *args, **kw): """ Process event and call :meth:`logging.Logger.warning` with the result. """ return self._proxy_to_logger('warning', event, *args, **kw) warn = warning def error(self, event=None, *args, **kw): """ Process event and call :meth:`logging.Logger.error` with the result. """ return self._proxy_to_logger('error', event, *args, **kw) def critical(self, event=None, *args, **kw): """ Process event and call :meth:`logging.Logger.critical` with the result. """ return self._proxy_to_logger('critical', event, *args, **kw) def exception(self, event=None, *args, **kw): """ Process event and call :meth:`logging.Logger.error` with the result, after setting ``exc_info`` to `True`. """ kw['exc_info'] = True return self.error(event, *args, **kw) def log(self, level, event, *args, **kw): """ Process event and call the appropriate logging method depending on `level`. """ return self._proxy_to_logger(_LEVEL_TO_NAME[level], event, *args, **kw) fatal = critical def _proxy_to_logger(self, method_name, event, *event_args, **event_kw): """ Propagate a method call to the wrapped logger. This is the same as the superclass implementation, except that it also preserves positional arguments in the `event_dict` so that the stdblib's support for format strings can be used. """ if event_args: event_kw['positional_args'] = event_args return super(BoundLogger, self)._proxy_to_logger(method_name, event=event, **event_kw) # # Pass-through methods to mimick the stdlib's logger interface. # def setLevel(self, level): """ Calls :meth:`logging.Logger.setLevel` with unmodified arguments. """ self._logger.setLevel(level) def findCaller(self, stack_info=False): """ Calls :meth:`logging.Logger.findCaller` with unmodified arguments. """ return self._logger.findCaller(stack_info=stack_info) def makeRecord(self, name, level, fn, lno, msg, args, exc_info, func=None, extra=None): """ Calls :meth:`logging.Logger.makeRecord` with unmodified arguments. """ return self._logger.makeRecord(name, level, fn, lno, msg, args, exc_info, func=func, extra=extra) def handle(self, record): """ Calls :meth:`logging.Logger.handle` with unmodified arguments. """ self._logger.handle(record) def addHandler(self, hdlr): """ Calls :meth:`logging.Logger.addHandler` with unmodified arguments. """ self._logger.addHandler(hdlr) def removeHandler(self, hdlr): """ Calls :meth:`logging.Logger.removeHandler` with unmodified arguments. """ self._logger.removeHandler(hdlr) def hasHandlers(self): """ Calls :meth:`logging.Logger.hasHandlers` with unmodified arguments. Exists only in Python 3. """ return self._logger.hasHandlers() # pragma: nocover def callHandlers(self, record): """ Calls :meth:`logging.Logger.callHandlers` with unmodified arguments. """ self._logger.callHandlers(record) def getEffectiveLevel(self): """ Calls :meth:`logging.Logger.getEffectiveLevel` with unmodified arguments. """ return self._logger.getEffectiveLevel() def isEnabledFor(self, level): """ Calls :meth:`logging.Logger.isEnabledFor` with unmodified arguments. """ return self._logger.isEnabledFor(level) def getChild(self, suffix): """ Calls :meth:`logging.Logger.getChild` with unmodified arguments. """ return self._logger.getChild(suffix) class LoggerFactory(object): """ Build a standard library logger when an *instance* is called. Sets a custom logger using :func:`logging.setLoggerClass` so variables in log format are expanded properly. >>> from structlog import configure >>> from structlog.stdlib import LoggerFactory >>> configure(logger_factory=LoggerFactory()) :param ignore_frame_names: When guessing the name of a logger, skip frames whose names *start* with one of these. For example, in pyramid applications you'll want to set it to ``['venusian', 'pyramid.config']``. :type ignore_frame_names: `list` of `str` """ def __init__(self, ignore_frame_names=None): self._ignore = ignore_frame_names logging.setLoggerClass(_FixedFindCallerLogger) def __call__(self, *args): """ Deduce the caller's module name and create a stdlib logger. If an optional argument is passed, it will be used as the logger name instead of guesswork. This optional argument would be passed from the :func:`structlog.get_logger` call. For example ``structlog.get_logger('foo')`` would cause this method to be called with ``'foo'`` as its first positional argument. :rtype: `logging.Logger` .. versionchanged:: 0.4.0 Added support for optional positional arguments. Using the first one for naming the constructed logger. """ if args: return logging.getLogger(args[0]) # We skip all frames that originate from within structlog or one of the # configured names. _, name = _find_first_app_frame_and_name(self._ignore) return logging.getLogger(name) class PositionalArgumentsFormatter(object): """ Apply stdlib-like string formatting to the `event` key. If the `positional_args` key in the event dict is set, it must contain a tuple that is used for formatting (using the `%s` string formatting operator) of the value from the `event` key. This works in the same way as the stdlib handles arguments to the various log methods: if the tuple contains only a single `dict` argument it is used for keyword placeholders in the `event` string, otherwise it will be used for positional placeholders. `positional_args` is populated by `structlog.stdlib.BoundLogger` or can be set manually. The `remove_positional_args` flag can be set to `False` to keep the `positional_args` key in the event dict; by default it will be removed from the event dict after formatting a message. """ def __init__(self, remove_positional_args=True): self.remove_positional_args = remove_positional_args def __call__(self, _, __, event_dict): args = event_dict.get('positional_args') # Mimick the formatting behaviour of the stdlib's logging # module, which accepts both positional arguments and a single # dict argument. The "single dict" check is the same one as the # stdlib's logging module performs in LogRecord.__init__(). if args: if len(args) == 1 and isinstance(args[0], dict) and args[0]: args = args[0] event_dict['event'] = event_dict['event'] % args if self.remove_positional_args: del event_dict['positional_args'] return event_dict # Adapted from the stdlib CRITICAL = 50 FATAL = CRITICAL ERROR = 40 WARNING = 30 WARN = WARNING INFO = 20 DEBUG = 10 NOTSET = 0 _NAME_TO_LEVEL = { 'critical': CRITICAL, 'exception': ERROR, 'error': ERROR, 'warn': WARNING, 'warning': WARNING, 'info': INFO, 'debug': DEBUG, 'notset': NOTSET, } _LEVEL_TO_NAME = dict( (v, k) for k, v in _NAME_TO_LEVEL.items() if k not in ("warn", "notset") ) def filter_by_level(logger, name, event_dict): """ Check whether logging is configured to accept messages from this log level. Should be the first processor if stdlib's filtering by level is used so possibly expensive processors like exception formatters are avoided in the first place. >>> import logging >>> from structlog.stdlib import filter_by_level >>> logging.basicConfig(level=logging.WARN) >>> logger = logging.getLogger() >>> filter_by_level(logger, 'warn', {}) {} >>> filter_by_level(logger, 'debug', {}) Traceback (most recent call last): ... DropEvent """ if logger.isEnabledFor(_NAME_TO_LEVEL[name]): return event_dict else: raise DropEvent def add_log_level(logger, method_name, event_dict): """ Add the log level to the event dict. """ if method_name == 'warn': # The stdlib has an alias method_name = 'warning' event_dict['level'] = method_name return event_dict def add_logger_name(logger, method_name, event_dict): """ Add the logger name to the event dict. """ event_dict['logger'] = logger.name return event_dict structlog-15.2.0/structlog/threadlocal.py0000644000076500000240000001161112536007610020750 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Primitives to keep context global but thread (and greenlet) local. """ from __future__ import absolute_import, division, print_function import contextlib import uuid from structlog._config import BoundLoggerLazyProxy try: from greenlet import getcurrent except ImportError: # pragma: nocover from threading import local as ThreadLocal else: # pragma: nocover from weakref import WeakKeyDictionary class ThreadLocal(object): """ threading.local() replacement for greenlets. """ def __init__(self): self.__dict__["_weakdict"] = WeakKeyDictionary() def __getattr__(self, name): key = getcurrent() try: return self._weakdict[key][name] except KeyError: raise AttributeError(name) def __setattr__(self, name, val): key = getcurrent() self._weakdict.setdefault(key, {})[name] = val def __delattr__(self, name): key = getcurrent() try: del self._weakdict[key][name] except KeyError: raise AttributeError(name) def wrap_dict(dict_class): """ Wrap a dict-like class and return the resulting class. The wrapped class and used to keep global in the current thread. :param type dict_class: Class used for keeping context. :rtype: `type` """ Wrapped = type('WrappedDict-' + str(uuid.uuid4()), (_ThreadLocalDictWrapper,), {}) Wrapped._tl = ThreadLocal() Wrapped._dict_class = dict_class return Wrapped def as_immutable(logger): """ Extract the context from a thread local logger into an immutable logger. :param structlog.BoundLogger logger: A logger with *possibly* thread local state. :rtype: :class:`~structlog.BoundLogger` with an immutable context. """ if isinstance(logger, BoundLoggerLazyProxy): logger = logger.bind() try: ctx = logger._context._tl.dict_.__class__(logger._context._dict) bl = logger.__class__( logger._logger, processors=logger._processors, context={}, ) bl._context = ctx return bl except AttributeError: return logger @contextlib.contextmanager def tmp_bind(logger, **tmp_values): """ Bind *tmp_values* to *logger* & memorize current state. Rewind afterwards. >>> from structlog import wrap_logger, PrintLogger >>> from structlog.threadlocal import tmp_bind, wrap_dict >>> logger = wrap_logger(PrintLogger(), context_class=wrap_dict(dict)) >>> with tmp_bind(logger, x=5) as tmp_logger: ... logger = logger.bind(y=3) ... tmp_logger.msg('event') y=3 x=5 event='event' >>> logger.msg('event') event='event' """ saved = as_immutable(logger)._context yield logger.bind(**tmp_values) logger._context.clear() logger._context.update(saved) class _ThreadLocalDictWrapper(object): """ Wrap a dict-like class and keep the state *global* but *thread-local*. Attempts to re-initialize only updates the wrapped dictionary. Useful for short-lived threaded applications like requests in web app. Use :func:`wrap` to instantiate and use :func:`structlog._loggers.BoundLogger.new` to clear the context. """ def __init__(self, *args, **kw): """ We cheat. A context dict gets never recreated. """ if args and isinstance(args[0], self.__class__): # our state is global, no need to look at args[0] if it's of our # class self._dict.update(**kw) else: self._dict.update(*args, **kw) @property def _dict(self): """ Return or create and return the current context. """ try: return self.__class__._tl.dict_ except AttributeError: self.__class__._tl.dict_ = self.__class__._dict_class() return self.__class__._tl.dict_ def __repr__(self): return '<{0}({1!r})>'.format(self.__class__.__name__, self._dict) def __eq__(self, other): # Same class == same dictionary return self.__class__ == other.__class__ def __ne__(self, other): return not self.__eq__(other) # Proxy methods necessary for structlog. # Dunder methods don't trigger __getattr__ so we need to proxy by hand. def __iter__(self): return self._dict.__iter__() def __setitem__(self, key, value): self._dict[key] = value def __delitem__(self, key): self._dict.__delitem__(key) def __len__(self): return self._dict.__len__() def __getattr__(self, name): method = getattr(self._dict, name) return method structlog-15.2.0/structlog/twisted.py0000644000076500000240000002222312522630354020154 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Processors and tools specific to the `Twisted `_ networking engine. See also :doc:`structlog's Twisted support `. """ from __future__ import absolute_import, division, print_function import json import sys from twisted.python import log from twisted.python.failure import Failure from twisted.python.log import ILogObserver, textFromEventDict from zope.interface import implementer from structlog._base import BoundLoggerBase from structlog._compat import PY2, string_types from structlog._utils import until_not_interrupted from structlog.processors import ( # can't import processors module without risking circular imports JSONRenderer as GenericJSONRenderer, KeyValueRenderer, ) class BoundLogger(BoundLoggerBase): """ Twisted-specific version of :class:`structlog.BoundLogger`. Works exactly like the generic one except that it takes advantage of knowing the logging methods in advance. Use it like:: configure( wrapper_class=structlog.twisted.BoundLogger, ) """ def msg(self, event=None, **kw): """ Process event and call ``log.msg()`` with the result. """ return self._proxy_to_logger('msg', event, **kw) def err(self, event=None, **kw): """ Process event and call ``log.err()`` with the result. """ return self._proxy_to_logger('err', event, **kw) class LoggerFactory(object): """ Build a Twisted logger when an *instance* is called. >>> from structlog import configure >>> from structlog.twisted import LoggerFactory >>> configure(logger_factory=LoggerFactory()) """ def __call__(self, *args): """ Positional arguments are silently ignored. :rvalue: A new Twisted logger. .. versionchanged:: 0.4.0 Added support for optional positional arguments. """ return log _FAIL_TYPES = (BaseException, Failure) def _extractStuffAndWhy(eventDict): """ Removes all possible *_why*s and *_stuff*s, analyzes exc_info and returns a tuple of `(_stuff, _why, eventDict)`. **Modifies** *eventDict*! """ _stuff = eventDict.pop('_stuff', None) _why = eventDict.pop('_why', None) event = eventDict.pop('event', None) if ( isinstance(_stuff, _FAIL_TYPES) and isinstance(event, _FAIL_TYPES) ): raise ValueError('Both _stuff and event contain an Exception/Failure.') # `log.err('event', _why='alsoEvent')` is ambiguous. if _why and isinstance(event, string_types): raise ValueError('Both `_why` and `event` supplied.') # Two failures are ambiguous too. if not isinstance(_stuff, _FAIL_TYPES) and isinstance(event, _FAIL_TYPES): _why = _why or 'error' _stuff = event if isinstance(event, string_types): _why = event if not _stuff and sys.exc_info() != (None, None, None): _stuff = Failure() # Either we used the error ourselves or the user supplied one for # formatting. Avoid log.err() to dump another traceback into the log. if isinstance(_stuff, BaseException): _stuff = Failure(_stuff) if PY2: # pragma: nocover we don't care about the implicit else sys.exc_clear() return _stuff, _why, eventDict class ReprWrapper(object): """ Wrap a string and return it as the __repr__. This is needed for log.err() that calls repr() on _stuff: >>> repr("foo") "'foo'" >>> repr(ReprWrapper("foo")) 'foo' Note the extra quotes in the unwrapped example. """ def __init__(self, string): self.string = string def __eq__(self, other): """ Check for equality, actually just for tests. """ return isinstance(other, self.__class__) \ and self.string == other.string def __repr__(self): return self.string class JSONRenderer(GenericJSONRenderer): """ Behaves like :class:`structlog.processors.JSONRenderer` except that it formats tracebacks and failures itself if called with `err()`. .. note:: This ultimately means that the messages get logged out using `msg()`, and *not* `err()` which renders failures in separate lines. Therefore it will break your tests that contain assertions using `flushLoggedErrors `_. *Not* an adapter like :class:`EventAdapter` but a real formatter. Nor does it require to be adapted using it. Use together with a :class:`JSONLogObserverWrapper`-wrapped Twisted logger like :func:`plainJSONStdOutLogger` for pure-JSON logs. """ def __call__(self, logger, name, eventDict): _stuff, _why, eventDict = _extractStuffAndWhy(eventDict) if name == 'err': eventDict['event'] = _why if isinstance(_stuff, Failure): eventDict['exception'] = _stuff.getTraceback(detail='verbose') _stuff.cleanFailure() else: eventDict['event'] = _why return ((ReprWrapper( GenericJSONRenderer.__call__(self, logger, name, eventDict) ),), {'_structlog': True}) @implementer(ILogObserver) class PlainFileLogObserver(object): """ Write only the the plain message without timestamps or anything else. Great to just print JSON to stdout where you catch it with something like runit. :param file file: File to print to. .. versionadded:: 0.2.0 """ def __init__(self, file): self._write = file.write self._flush = file.flush def __call__(self, eventDict): until_not_interrupted(self._write, textFromEventDict(eventDict) + '\n') until_not_interrupted(self._flush) @implementer(ILogObserver) class JSONLogObserverWrapper(object): """ Wrap a log *observer* and render non-:class:`JSONRenderer` entries to JSON. :param ILogObserver observer: Twisted log observer to wrap. For example :class:`PlainFileObserver` or Twisted's stock `FileLogObserver `_ .. versionadded:: 0.2.0 """ def __init__(self, observer): self._observer = observer def __call__(self, eventDict): if '_structlog' not in eventDict: eventDict['message'] = (json.dumps({ 'event': textFromEventDict(eventDict), 'system': eventDict.get('system'), }),) eventDict['_structlog'] = True return self._observer(eventDict) def plainJSONStdOutLogger(): """ Return a logger that writes only the message to stdout. Transforms non-:class:`~structlog.twisted.JSONRenderer` messages to JSON. Ideal for JSONifying log entries from Twisted plugins and libraries that are outside of your control:: $ twistd -n --logger structlog.twisted.plainJSONStdOutLogger web {"event": "Log opened.", "system": "-"} {"event": "twistd 13.1.0 (python 2.7.3) starting up.", "system": "-"} {"event": "reactor class: twisted...EPollReactor.", "system": "-"} {"event": "Site starting on 8080", "system": "-"} {"event": "Starting factory ", ...} ... Composes :class:`PlainFileLogObserver` and :class:`JSONLogObserverWrapper` to a usable logger. .. versionadded:: 0.2.0 """ return JSONLogObserverWrapper(PlainFileLogObserver(sys.stdout)) class EventAdapter(object): """ Adapt an ``event_dict`` to Twisted logging system. Particularly, make a wrapped `twisted.python.log.err `_ behave as expected. :param callable dictRenderer: Renderer that is used for the actual log message. Please note that structlog comes with a dedicated :class:`JSONRenderer`. **Must** be the last processor in the chain and requires a `dictRenderer` for the actual formatting as an constructor argument in order to be able to fully support the original behaviors of ``log.msg()`` and ``log.err()``. """ def __init__(self, dictRenderer=None): """ :param dictRenderer: A processor used to format the log message. """ self._dictRenderer = dictRenderer or KeyValueRenderer() def __call__(self, logger, name, eventDict): if name == 'err': # This aspires to handle the following cases correctly: # - log.err(failure, _why='event', **kw) # - log.err('event', **kw) # - log.err(_stuff=failure, _why='event', **kw) _stuff, _why, eventDict = _extractStuffAndWhy(eventDict) eventDict['event'] = _why return ((), { '_stuff': _stuff, '_why': self._dictRenderer(logger, name, eventDict), }) else: return self._dictRenderer(logger, name, eventDict) structlog-15.2.0/structlog.egg-info/0000755000076500000240000000000012536025152017607 5ustar hynekstaff00000000000000structlog-15.2.0/structlog.egg-info/dependency_links.txt0000644000076500000240000000000112536025152023655 0ustar hynekstaff00000000000000 structlog-15.2.0/structlog.egg-info/PKG-INFO0000644000076500000240000000550312536025152020707 0ustar hynekstaff00000000000000Metadata-Version: 1.1 Name: structlog Version: 15.2.0 Summary: Structured logging for Python. Home-page: http://www.structlog.org/ Author: Hynek Schlawack Author-email: hs@ox.cx License: MIT or Apache License, Version 2.0 Description: ======================================== structlog: Structured Logging for Python ======================================== .. image:: https://pypip.in/version/structlog/badge.svg :target: https://pypi.python.org/pypi/structlog/ :alt: Latest Version .. image:: https://travis-ci.org/hynek/structlog.svg?branch=master :target: https://travis-ci.org/hynek/structlog .. image:: https://coveralls.io/repos/hynek/structlog/badge.svg?branch=master :target: https://coveralls.io/r/hynek/structlog?branch=master ``structlog`` makes structured logging in Python easy by *augmenting* your *existing* logger. It allows you to split your log entries up into key/value pairs and build them incrementally without annoying boilerplate code. .. code-block:: pycon >>> from structlog import get_logger >>> log = get_logger() >>> log = log.bind(user='anonymous', some_key=23) >>> log = log.bind(user='hynek', another_key=42) >>> log.info('user.logged_in', happy=True) some_key=23 user='hynek' another_key=42 happy=True event='user.logged_in' .. begin It's dual-licensed under `Apache License, version 2 `_ and `MIT `_, available from `PyPI `_, the source code can be found on `GitHub `_, the documentation at `http://www.structlog.org/ `_. ``structlog`` targets Python 2.6, 2.7, 3.3, 3.4, and PyPy with no additional dependencies for core functionality. If you need any help, visit us on ``#structlog`` on `Freenode `_! Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: Apache Software License Classifier: License :: OSI Approved :: MIT License Classifier: Natural Language :: English Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python Classifier: Topic :: Software Development :: Libraries :: Python Modules structlog-15.2.0/structlog.egg-info/SOURCES.txt0000644000076500000240000000317112536025152021475 0ustar hynekstaff00000000000000.coveragerc .travis.yml AUTHORS.rst CONTRIBUTING.rst LICENSE LICENSE.apache2 LICENSE.mit MANIFEST.in README.rst dev-requirements.txt docs-requirements.txt setup.cfg setup.py tox.ini docs/Makefile docs/api.rst docs/changelog.rst docs/conf.py docs/configuration.rst docs/contributing.rst docs/custom-wrappers.rst docs/examples.rst docs/getting-started.rst docs/index.rst docs/license.rst docs/loggers.rst docs/logging-best-practices.rst docs/make.bat docs/performance.rst docs/processors.rst docs/standard-library.rst docs/thread-local.rst docs/twisted.rst docs/why.rst docs/_static/.keep docs/code_examples/twisted_echo.py docs/code_examples/flask_/__init__.py docs/code_examples/flask_/some_module.py docs/code_examples/flask_/webapp.py docs/code_examples/getting-started/imaginary_web.py docs/code_examples/getting-started/imaginary_web_better.py docs/code_examples/processors/conditional_dropper.py docs/code_examples/processors/dropper.py docs/code_examples/processors/timestamper.py structlog/__init__.py structlog/_base.py structlog/_compat.py structlog/_config.py structlog/_exc.py structlog/_frames.py structlog/_generic.py structlog/_loggers.py structlog/_utils.py structlog/processors.py structlog/stdlib.py structlog/threadlocal.py structlog/twisted.py structlog.egg-info/PKG-INFO structlog.egg-info/SOURCES.txt structlog.egg-info/dependency_links.txt structlog.egg-info/top_level.txt tests/__init__.py tests/additional_frame.py tests/test_base.py tests/test_config.py tests/test_frames.py tests/test_generic.py tests/test_loggers.py tests/test_processors.py tests/test_stdlib.py tests/test_threadlocal.py tests/test_twisted.py tests/test_utils.pystructlog-15.2.0/structlog.egg-info/top_level.txt0000644000076500000240000000001212536025152022332 0ustar hynekstaff00000000000000structlog structlog-15.2.0/tests/0000755000076500000240000000000012536025152015231 5ustar hynekstaff00000000000000structlog-15.2.0/tests/__init__.py0000644000076500000240000000026512432431321017337 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. structlog-15.2.0/tests/additional_frame.py0000644000076500000240000000077512432431376021102 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. """ Helper function for testing the deduction of stdlib logger names. Since the logger factories are called from within structlog._config, they have to skip a frame. Calling them here emulates that. """ from __future__ import absolute_import, division, print_function def additional_frame(callable): return callable() structlog-15.2.0/tests/test_base.py0000644000076500000240000001150212471544331017556 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. from __future__ import absolute_import, division, print_function import pytest from pretend import raiser, stub from structlog._base import BoundLoggerBase from structlog._config import _CONFIG from structlog._exc import DropEvent from structlog._loggers import ReturnLogger from structlog.processors import KeyValueRenderer def build_bl(logger=None, processors=None, context=None): """ Convenience function to build BoundLoggerBases with sane defaults. """ return BoundLoggerBase( logger or ReturnLogger(), processors or _CONFIG.default_processors, context if context is not None else _CONFIG.default_context_class(), ) class TestBinding(object): def test_repr(self): l = build_bl(processors=[1, 2, 3], context={}) assert '' == repr(l) def test_binds_independently(self): """ Ensure BoundLogger is immutable by default. """ b = build_bl(processors=[KeyValueRenderer(sort_keys=True)]) b = b.bind(x=42, y=23) b1 = b.bind(foo='bar') b2 = b.bind(foo='qux') assert b._context != b1._context != b2._context def test_new_clears_state(self): b = build_bl() b = b.bind(x=42) assert 42 == b._context['x'] b = b.bind() assert 42 == b._context['x'] b = b.new() assert 'x' not in b._context def test_comparison(self): b = build_bl() assert b == b.bind() assert b is not b.bind() assert b != b.bind(x=5) assert b != 'test' def test_bind_keeps_class(self): class Wrapper(BoundLoggerBase): pass b = Wrapper(None, [], {}) assert isinstance(b.bind(), Wrapper) def test_new_keeps_class(self): class Wrapper(BoundLoggerBase): pass b = Wrapper(None, [], {}) assert isinstance(b.new(), Wrapper) def test_unbind(self): b = build_bl().bind(x=42, y=23).unbind('x', 'y') assert {} == b._context class TestProcessing(object): def test_copies_context_before_processing(self): """ BoundLoggerBase._process_event() gets called before relaying events to wrapped loggers. """ def chk(_, __, event_dict): assert b._context is not event_dict return '' b = build_bl(processors=[chk]) assert (('',), {}) == b._process_event('', 'event', {}) assert 'event' not in b._context def test_chain_does_not_swallow_all_exceptions(self): b = build_bl(processors=[raiser(ValueError)]) with pytest.raises(ValueError): b._process_event('', 'boom', {}) def test_last_processor_returns_string(self): """ If the final processor returns a string, ``(the_string,), {}`` is returned. """ logger = stub(msg=lambda *args, **kw: (args, kw)) b = build_bl(logger, processors=[lambda *_: 'foo']) assert ( (('foo',), {}) == b._process_event('', 'foo', {}) ) def test_last_processor_returns_tuple(self): """ If the final processor returns a tuple, it is just passed through. """ logger = stub(msg=lambda *args, **kw: (args, kw)) b = build_bl(logger, processors=[lambda *_: (('foo',), {'key': 'value'})]) assert ( (('foo',), {'key': 'value'}) == b._process_event('', 'foo', {}) ) def test_last_processor_returns_dict(self): """ If the final processor returns a dict, ``(), the_dict`` is returnend. """ logger = stub(msg=lambda *args, **kw: (args, kw)) b = build_bl(logger, processors=[lambda *_: {'event': 'foo'}]) assert ( ((), {'event': 'foo'}) == b._process_event('', 'foo', {}) ) def test_last_processor_returns_unknown_value(self): """ If the final processor returns something unexpected, raise ValueError with a helpful error message. """ logger = stub(msg=lambda *args, **kw: (args, kw)) b = build_bl(logger, processors=[lambda *_: object()]) with pytest.raises(ValueError) as exc: b._process_event('', 'foo', {}) assert ( exc.value.args[0].startswith("Last processor didn't return") ) class TestProxying(object): def test_processor_raising_DropEvent_silently_aborts_chain(self, capsys): b = build_bl(processors=[raiser(DropEvent), raiser(ValueError)]) b._proxy_to_logger('', None, x=5) assert (('', '') == capsys.readouterr()) structlog-15.2.0/tests/test_config.py0000644000076500000240000002214012516144575020117 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. from __future__ import absolute_import, division, print_function import warnings import pytest from pretend import call_recorder, call, stub from structlog._base import BoundLoggerBase from structlog._compat import PY3 from structlog._config import ( BoundLoggerLazyProxy, _CONFIG, _BUILTIN_DEFAULT_CONTEXT_CLASS, _BUILTIN_DEFAULT_PROCESSORS, _BUILTIN_DEFAULT_LOGGER_FACTORY, _BUILTIN_DEFAULT_WRAPPER_CLASS, configure, configure_once, get_logger, reset_defaults, wrap_logger, ) @pytest.fixture def proxy(): """ Returns a BoundLoggerLazyProxy constructed w/o paramaters & None as logger. """ return BoundLoggerLazyProxy(None) class Wrapper(BoundLoggerBase): """ Custom wrapper class for testing. """ class TestConfigure(object): def teardown_method(self, method): reset_defaults() def test_configure_all(self, proxy): x = stub() configure(processors=[x], context_class=dict) b = proxy.bind() assert [x] == b._processors assert dict is b._context.__class__ def test_reset(self, proxy): x = stub() configure(processors=[x], context_class=dict, wrapper_class=Wrapper) reset_defaults() b = proxy.bind() assert [x] != b._processors assert _BUILTIN_DEFAULT_PROCESSORS == b._processors assert isinstance(b, _BUILTIN_DEFAULT_WRAPPER_CLASS) assert _BUILTIN_DEFAULT_CONTEXT_CLASS == b._context.__class__ assert _BUILTIN_DEFAULT_LOGGER_FACTORY is _CONFIG.logger_factory def test_just_processors(self, proxy): x = stub() configure(processors=[x]) b = proxy.bind() assert [x] == b._processors assert _BUILTIN_DEFAULT_PROCESSORS != b._processors assert _BUILTIN_DEFAULT_CONTEXT_CLASS == b._context.__class__ def test_just_context_class(self, proxy): configure(context_class=dict) b = proxy.bind() assert dict is b._context.__class__ assert _BUILTIN_DEFAULT_PROCESSORS == b._processors def test_configure_sets_is_configured(self): assert False is _CONFIG.is_configured configure() assert True is _CONFIG.is_configured def test_rest_resets_is_configured(self): configure() reset_defaults() assert False is _CONFIG.is_configured def test_configures_logger_factory(self): def f(): pass configure(logger_factory=f) assert f is _CONFIG.logger_factory class TestBoundLoggerLazyProxy(object): def teardown_method(self, method): reset_defaults() def test_repr(self): p = BoundLoggerLazyProxy( None, processors=[1, 2, 3], context_class=dict, initial_values={'foo': 42}, logger_factory_args=(4, 5), ) assert ( ", " "initial_values={'foo': 42}, " "logger_factory_args=(4, 5))>" % ('class' if PY3 else 'type',) ) == repr(p) def test_returns_bound_logger_on_bind(self, proxy): assert isinstance(proxy.bind(), BoundLoggerBase) def test_returns_bound_logger_on_new(self, proxy): assert isinstance(proxy.new(), BoundLoggerBase) def test_prefers_args_over_config(self): p = BoundLoggerLazyProxy(None, processors=[1, 2, 3], context_class=dict) b = p.bind() assert isinstance(b._context, dict) assert [1, 2, 3] == b._processors class Class(object): def __init__(self, *args, **kw): pass def update(self, *args, **kw): pass configure(processors=[4, 5, 6], context_class=Class) b = p.bind() assert not isinstance(b._context, Class) assert [1, 2, 3] == b._processors def test_falls_back_to_config(self, proxy): b = proxy.bind() assert isinstance(b._context, _CONFIG.default_context_class) assert _CONFIG.default_processors == b._processors def test_bind_honors_initial_values(self): p = BoundLoggerLazyProxy(None, initial_values={'a': 1, 'b': 2}) b = p.bind() assert {'a': 1, 'b': 2} == b._context b = p.bind(c=3) assert {'a': 1, 'b': 2, 'c': 3} == b._context def test_bind_binds_new_values(self, proxy): b = proxy.bind(c=3) assert {'c': 3} == b._context def test_unbind_unbinds_from_initial_values(self): p = BoundLoggerLazyProxy(None, initial_values={'a': 1, 'b': 2}) b = p.unbind('a') assert {'b': 2} == b._context def test_honors_wrapper_class(self): p = BoundLoggerLazyProxy(None, wrapper_class=Wrapper) b = p.bind() assert isinstance(b, Wrapper) def test_honors_wrapper_from_config(self, proxy): configure(wrapper_class=Wrapper) b = proxy.bind() assert isinstance(b, Wrapper) def test_new_binds_only_initial_values_impolicit_ctx_class(self, proxy): proxy = BoundLoggerLazyProxy(None, initial_values={'a': 1, 'b': 2}) b = proxy.new(foo=42) assert {'a': 1, 'b': 2, 'foo': 42} == b._context def test_new_binds_only_initial_values_explicit_ctx_class(self, proxy): proxy = BoundLoggerLazyProxy(None, initial_values={'a': 1, 'b': 2}, context_class=dict) b = proxy.new(foo=42) assert {'a': 1, 'b': 2, 'foo': 42} == b._context def test_rebinds_bind_method(self, proxy): """ To save time, be rebind the bind method once the logger has been cached. """ configure(cache_logger_on_first_use=True) bind = proxy.bind proxy.bind() assert bind != proxy.bind def test_does_not_cache_by_default(self, proxy): """ Proxy's bind method doesn't change by default. """ bind = proxy.bind proxy.bind() assert bind == proxy.bind def test_argument_takes_precedence_over_configuration(self): configure(cache_logger_on_first_use=True) proxy = BoundLoggerLazyProxy(None, cache_logger_on_first_use=False) bind = proxy.bind proxy.bind() assert bind == proxy.bind def test_argument_takes_precedence_over_configuration2(self): configure(cache_logger_on_first_use=False) proxy = BoundLoggerLazyProxy(None, cache_logger_on_first_use=True) bind = proxy.bind proxy.bind() assert bind != proxy.bind def test_emphemeral(self): """ Calling an unknown method proxy creates a new wrapped bound logger first. """ class Foo(BoundLoggerBase): def foo(self): return 42 proxy = BoundLoggerLazyProxy( None, wrapper_class=Foo, cache_logger_on_first_use=False, ) assert 42 == proxy.foo() class TestFunctions(object): def teardown_method(self, method): reset_defaults() def test_wrap_passes_args(self): logger = object() p = wrap_logger(logger, processors=[1, 2, 3], context_class=dict) assert logger is p._logger assert [1, 2, 3] == p._processors assert dict is p._context_class def test_empty_processors(self): """ An empty list is a valid value for processors so it must be preserved. """ # We need to do a bind such that we get an actual logger and not just # a lazy proxy. l = wrap_logger(object(), processors=[]).new() assert [] == l._processors def test_wrap_returns_proxy(self): assert isinstance(wrap_logger(None), BoundLoggerLazyProxy) def test_configure_once_issues_warning_on_repeated_call(self): with warnings.catch_warnings(record=True) as warns: configure_once() assert 0 == len(warns) with warnings.catch_warnings(record=True) as warns: configure_once() assert 1 == len(warns) assert RuntimeWarning == warns[0].category assert 'Repeated configuration attempted.' == warns[0].message.args[0] def test_get_logger_configures_according_to_config(self): b = get_logger().bind() assert isinstance(b._logger, _BUILTIN_DEFAULT_LOGGER_FACTORY().__class__) assert _BUILTIN_DEFAULT_PROCESSORS == b._processors assert isinstance(b, _BUILTIN_DEFAULT_WRAPPER_CLASS) assert _BUILTIN_DEFAULT_CONTEXT_CLASS == b._context.__class__ def test_get_logger_passes_positional_arguments_to_logger_factory(self): """ Ensure `get_logger` passes optional positional arguments through to the logger factory. """ factory = call_recorder(lambda *args: object()) configure(logger_factory=factory) get_logger('test').bind(x=42) assert [call('test')] == factory.calls structlog-15.2.0/tests/test_frames.py0000644000076500000240000000704212524701723020124 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. from __future__ import absolute_import, division, print_function import sys import pytest from pretend import stub import structlog._frames from structlog._frames import ( _find_first_app_frame_and_name, _format_exception, _format_stack, ) class TestFindFirstAppFrameAndName(object): def test_ignores_structlog_by_default(self, monkeypatch): """ No matter what you pass in, structlog frames get always ignored. """ f1 = stub(f_globals={'__name__': 'test'}, f_back=None) f2 = stub(f_globals={'__name__': 'structlog.blubb'}, f_back=f1) monkeypatch.setattr(structlog._frames.sys, '_getframe', lambda: f2) f, n = _find_first_app_frame_and_name() assert ((f1, 'test') == f, n) def test_ignoring_of_additional_frame_names_works(self, monkeypatch): """ Additional names are properly ignored too. """ f1 = stub(f_globals={'__name__': 'test'}, f_back=None) f2 = stub(f_globals={'__name__': 'ignored.bar'}, f_back=f1) f3 = stub(f_globals={'__name__': 'structlog.blubb'}, f_back=f2) monkeypatch.setattr(structlog._frames.sys, '_getframe', lambda: f3) f, n = _find_first_app_frame_and_name() assert ((f1, 'test') == f, n) def test_tolerates_missing_name(self, monkeypatch): """ Use ``?`` if `f_globals` lacks a `__name__` key """ f1 = stub(f_globals={}, f_back=None) f, n = _find_first_app_frame_and_name() monkeypatch.setattr(structlog._frames.sys, "_getframe", lambda: f1) assert ((f1, "?") == f, n) @pytest.fixture def exc_info(): """ Fake a valid exc_info. """ try: raise ValueError except ValueError: return sys.exc_info() class TestFormatException(object): def test_returns_str(self, exc_info): """ Always returns a native string. """ assert isinstance(_format_exception(exc_info), str) def test_formats(self, exc_info): """ The passed exc_info is formatted. """ assert _format_exception(exc_info).startswith( "Traceback (most recent call last):\n" ) def test_no_trailing_nl(self, exc_info, monkeypatch): """ Trailing newlines are snipped off but if the string does not contain one nothing is removed. """ from structlog._frames import traceback monkeypatch.setattr( traceback, "print_exception", lambda *a: a[-1].write("foo") ) assert "foo" == _format_exception(exc_info) class TestFormatStack(object): def test_returns_str(self): """ Always returns a native string. """ assert isinstance(_format_stack(sys._getframe()), str) def test_formats(self): """ The passed stack is formatted. """ assert _format_stack(sys._getframe()).startswith( "Stack (most recent call last):\n" ) def test_no_trailing_nl(self, monkeypatch): """ Trailing newlines are snipped off but if the string does not contain one nothing is removed. """ from structlog._frames import traceback monkeypatch.setattr( traceback, "print_stack", lambda frame, file: file.write("foo") ) assert _format_stack(sys._getframe()).endswith("foo") structlog-15.2.0/tests/test_generic.py0000644000076500000240000000243512432431440020256 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. from __future__ import absolute_import, division, print_function from structlog._config import _CONFIG from structlog._generic import BoundLogger from structlog._loggers import ReturnLogger class TestLogger(object): def log(self, msg): return 'log', msg def gol(self, msg): return 'gol', msg class TestGenericBoundLogger(object): def test_caches(self): """ __getattr__() gets called only once per logger method. """ b = BoundLogger( ReturnLogger(), _CONFIG.default_processors, _CONFIG.default_context_class(), ) assert 'msg' not in b.__dict__ b.msg('foo') assert 'msg' in b.__dict__ def test_proxies_anything(self): """ Anything that isn't part of BoundLoggerBase gets proxied to the correct wrapped logger methods. """ b = BoundLogger( ReturnLogger(), _CONFIG.default_processors, _CONFIG.default_context_class(), ) assert 'log', 'foo' == b.log('foo') assert 'gol', 'bar' == b.gol('bar') structlog-15.2.0/tests/test_loggers.py0000644000076500000240000000704712501601331020303 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. from __future__ import absolute_import, division, print_function import sys import pytest from structlog._compat import StringIO from structlog._loggers import ( PrintLogger, PrintLoggerFactory, ReturnLogger, ReturnLoggerFactory, WRITE_LOCKS, ) from structlog.stdlib import _NAME_TO_LEVEL def test_return_logger(): obj = ['hello'] assert obj is ReturnLogger().msg(obj) STDLIB_MSG_METHODS = [m for m in _NAME_TO_LEVEL if m != 'notset'] class TestPrintLogger(object): def test_prints_to_stdout_by_default(self, capsys): """ Instantiating without arguments gives conveniently a logger to standard out. """ PrintLogger().msg('hello') out, err = capsys.readouterr() assert 'hello\n' == out assert '' == err def test_prints_to_correct_file(self, tmpdir, capsys): """ Supplied files are respected. """ f = tmpdir.join('test.log') fo = f.open('w') PrintLogger(fo).msg('hello') out, err = capsys.readouterr() assert '' == out == err fo.close() assert 'hello\n' == f.read() def test_repr(self): """ __repr__ makes sense. """ assert repr(PrintLogger()).startswith( "' return {'a': A(), 'b': [3, 4], 'x': 7, 'y': 'test', 'z': (1, 2)} class TestKeyValueRenderer(object): def test_sort_keys(self, event_dict): assert ( r"a= b=[3, 4] x=7 y='test' z=(1, 2)" == KeyValueRenderer(sort_keys=True)(None, None, event_dict) ) def test_order_complete(self, event_dict): assert ( r"y='test' b=[3, 4] a= z=(1, 2) x=7" == KeyValueRenderer(key_order=['y', 'b', 'a', 'z', 'x']) (None, None, event_dict) ) def test_order_missing(self, event_dict): """ Missing keys get rendered as None. """ assert ( r"c=None y='test' b=[3, 4] a= z=(1, 2) x=7" == KeyValueRenderer(key_order=['c', 'y', 'b', 'a', 'z', 'x']) (None, None, event_dict) ) def test_order_extra(self, event_dict): """ Extra keys get sorted if sort_keys=True. """ event_dict['B'] = 'B' event_dict['A'] = 'A' assert ( r"c=None y='test' b=[3, 4] a= z=(1, 2) x=7 A='A' B='B'" == KeyValueRenderer(key_order=['c', 'y', 'b', 'a', 'z', 'x'], sort_keys=True) (None, None, event_dict) ) def test_random_order(self, event_dict): rv = KeyValueRenderer()(None, None, event_dict) assert isinstance(rv, str) class TestJSONRenderer(object): def test_renders_json(self, event_dict): assert ( r'{"a": "", "b": [3, 4], "x": 7, "y": "test", "z": ' r'[1, 2]}' == JSONRenderer(sort_keys=True)(None, None, event_dict) ) def test_FallbackEncoder_handles_ThreadLocalDictWrapped_dicts(self): s = json.dumps(wrap_dict(dict)({'a': 42}), cls=_JSONFallbackEncoder) assert '{"a": 42}' == s def test_FallbackEncoder_falls_back(self): s = json.dumps({'date': datetime.date(1980, 3, 25)}, cls=_JSONFallbackEncoder,) assert '{"date": "datetime.date(1980, 3, 25)"}' == s class TestTimeStamper(object): def test_disallowsNonUTCUNIXTimestamps(self): with pytest.raises(ValueError) as e: TimeStamper(utc=False) assert 'UNIX timestamps are always UTC.' == e.value.args[0] def test_insertsUTCUNIXTimestampByDefault(self): ts = TimeStamper() d = ts(None, None, {}) # freezegun doesn't work with time.gmtime :( assert isinstance(d['timestamp'], int) @freeze_time('1980-03-25 16:00:00') def test_local(self): ts = TimeStamper(fmt='iso', utc=False) d = ts(None, None, {}) assert '1980-03-25T16:00:00' == d['timestamp'] @freeze_time('1980-03-25 16:00:00') def test_formats(self): ts = TimeStamper(fmt='%Y') d = ts(None, None, {}) assert '1980' == d['timestamp'] @freeze_time('1980-03-25 16:00:00') def test_adds_Z_to_iso(self): ts = TimeStamper(fmt='iso', utc=True) d = ts(None, None, {}) assert '1980-03-25T16:00:00Z' == d['timestamp'] @freeze_time('1980-03-25 16:00:00') def test_key_can_be_specified(self): """ Timestamp is stored with the specified key. """ ts = TimeStamper(fmt='%m', key='month') d = ts(None, None, {}) assert '03' == d['month'] class TestFormatExcInfo(object): def test_formats_tuple(self, monkeypatch): monkeypatch.setattr(structlog.processors, '_format_exception', lambda exc_info: exc_info) d = format_exc_info(None, None, {'exc_info': (None, None, 42)}) assert {'exception': (None, None, 42)} == d def test_gets_exc_info_on_bool(self): # monkeypatching sys.exc_info makes currently py.test return 1 on # success. try: raise ValueError('test') except ValueError: d = format_exc_info(None, None, {'exc_info': True}) assert 'exc_info' not in d assert 'raise ValueError(\'test\')\nValueError: test' in d['exception'] class TestUnicodeEncoder(object): def test_encodes(self): """ Unicode strings get encoded (as UTF-8 by default). """ ue = UnicodeEncoder() assert {"foo": b"b\xc3\xa4r"} == ue(None, None, {"foo": u"b\xe4r"}) def test_passes_arguments(self): """ Encoding options are passed into the encoding call. """ ue = UnicodeEncoder("latin1", "xmlcharrefreplace") assert {"foo": b"–"} == ue(None, None, {"foo": u"\u2013"}) def test_bytes_nop(self): """ If the string is already bytes, don't do anything. """ ue = UnicodeEncoder() assert {"foo": b"b\xc3\xa4r"} == ue(None, None, {"foo": b"b\xc3\xa4r"}) class TestExceptionPrettyPrinter(object): def test_stdout_by_default(self): """ If no file is supplied, use stdout. """ epp = ExceptionPrettyPrinter() assert sys.stdout is epp._file def test_prints_exception(self, sio): """ If there's an `exception` key in the event_dict, just print it out. This happens if `format_exc_info` was run before us in the chain. """ epp = ExceptionPrettyPrinter(file=sio) try: raise ValueError except ValueError: ed = format_exc_info(None, None, {'exc_info': True}) epp(None, None, ed) out = sio.getvalue() assert 'test_prints_exception' in out assert 'raise ValueError' in out def test_removes_exception_after_printing(self, sio): """ After pretty printing `exception` is removed from the event_dict. """ epp = ExceptionPrettyPrinter(sio) try: raise ValueError except ValueError: ed = format_exc_info(None, None, {'exc_info': True}) assert 'exception' in ed new_ed = epp(None, None, ed) assert 'exception' not in new_ed def test_handles_exc_info(self, sio): """ If `exc_info` is passed in, it behaves like `format_exc_info`. """ epp = ExceptionPrettyPrinter(sio) try: raise ValueError except ValueError: epp(None, None, {'exc_info': True}) out = sio.getvalue() assert 'test_handles_exc_info' in out assert 'raise ValueError' in out def test_removes_exc_info_after_printing(self, sio): """ After pretty printing `exception` is removed from the event_dict. """ epp = ExceptionPrettyPrinter(sio) try: raise ValueError except ValueError: ed = epp(None, None, {'exc_info': True}) assert 'exc_info' not in ed def test_nop_if_no_exception(self, sio): """ If there is no exception, don't print anything. """ epp = ExceptionPrettyPrinter(sio) epp(None, None, {}) assert '' == sio.getvalue() def test_own_exc_info(self, sio): """ If exc_info is a tuple, use it. """ epp = ExceptionPrettyPrinter(sio) try: raise ValueError("XXX") except ValueError: ei = sys.exc_info() epp(None, None, {"exc_info": ei}) assert "XXX" in sio.getvalue() @pytest.fixture def sir(): return StackInfoRenderer() class TestStackInfoRenderer(object): def test_removes_stack_info(self, sir): """ The `stack_info` key is removed from `event_dict`. """ ed = sir(None, None, {'stack_info': True}) assert 'stack_info' not in ed def test_adds_stack_if_asked(self, sir): """ If `stack_info` is true, `stack` is added. """ ed = sir(None, None, {'stack_info': True}) assert 'stack' in ed def test_renders_correct_stack(self, sir): ed = sir(None, None, {'stack_info': True}) assert "ed = sir(None, None, {'stack_info': True})" in ed['stack'] structlog-15.2.0/tests/test_stdlib.py0000644000076500000240000002330412522607406020130 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. from __future__ import absolute_import, division, print_function import os import logging import pytest from pretend import call_recorder from structlog._exc import DropEvent from structlog._loggers import ReturnLogger from structlog.stdlib import ( BoundLogger, CRITICAL, LoggerFactory, PositionalArgumentsFormatter, WARN, filter_by_level, add_log_level, add_logger_name, _FixedFindCallerLogger, ) from structlog._compat import PY2 from .additional_frame import additional_frame def build_bl(logger=None, processors=None, context=None): """ Convenience function to build BoundLogger with sane defaults. """ return BoundLogger( logger or ReturnLogger(), processors, {} ) def return_method_name(_, method_name, __): """ A final renderer that returns the name of the logging method. """ return method_name class TestLoggerFactory(object): def setup_method(self, method): """ The stdlib logger factory modifies global state to fix caller identification. """ self.original_logger = logging.getLoggerClass() def teardown_method(self, method): logging.setLoggerClass(self.original_logger) def test_deduces_correct_name(self): """ The factory isn't called directly but from structlog._config so deducing has to be slightly smarter. """ assert 'tests.additional_frame' == ( additional_frame(LoggerFactory()).name ) assert 'tests.test_stdlib' == LoggerFactory()().name def test_ignores_frames(self): """ The name guesser walks up the frames until it reaches a frame whose name is not from structlog or one of the configurable other names. """ assert '__main__' == additional_frame(LoggerFactory( ignore_frame_names=['tests.', '_pytest.']) ).name def test_deduces_correct_caller(self): logger = _FixedFindCallerLogger('test') file_name, line_number, func_name = logger.findCaller()[:3] assert file_name == os.path.realpath(__file__) assert func_name == 'test_deduces_correct_caller' @pytest.mark.skipif(PY2, reason="Py3-only") def test_stack_info(self): logger = _FixedFindCallerLogger('test') testing, is_, fun, stack_info = logger.findCaller(stack_info=True) assert 'testing, is_, fun' in stack_info @pytest.mark.skipif(PY2, reason="Py3-only") def test_no_stack_info_by_default(self): logger = _FixedFindCallerLogger('test') testing, is_, fun, stack_info = logger.findCaller() assert None is stack_info def test_find_caller(self, monkeypatch): logger = LoggerFactory()() log_handle = call_recorder(lambda x: None) monkeypatch.setattr(logger, 'handle', log_handle) logger.error('Test') log_record = log_handle.calls[0].args[0] assert log_record.funcName == 'test_find_caller' assert log_record.name == __name__ assert log_record.filename == os.path.basename(__file__) def test_sets_correct_logger(self): assert logging.getLoggerClass() is logging.Logger LoggerFactory() assert logging.getLoggerClass() is _FixedFindCallerLogger def test_positional_argument_avoids_guessing(self): """ If a positional argument is passed to the factory, it's used as the name instead of guessing. """ l = LoggerFactory()('foo') assert 'foo' == l.name class TestFilterByLevel(object): def test_filters_lower_levels(self): logger = logging.Logger(__name__) logger.setLevel(CRITICAL) with pytest.raises(DropEvent): filter_by_level(logger, 'warn', {}) def test_passes_higher_levels(self): logger = logging.Logger(__name__) logger.setLevel(WARN) event_dict = {'event': 'test'} assert event_dict is filter_by_level(logger, 'warn', event_dict) assert event_dict is filter_by_level(logger, 'error', event_dict) assert event_dict is filter_by_level(logger, 'exception', event_dict) class TestBoundLogger(object): @pytest.mark.parametrize(('method_name'), [ 'debug', 'info', 'warning', 'error', 'critical', ]) def test_proxies_to_correct_method(self, method_name): """ The basic proxied methods are proxied to the correct counterparts. """ bl = BoundLogger(ReturnLogger(), [return_method_name], {}) assert method_name == getattr(bl, method_name)('event') def test_proxies_exception(self): """ BoundLogger.exception is proxied to Logger.error. """ bl = BoundLogger(ReturnLogger(), [return_method_name], {}) assert "error" == bl.exception("event") def test_proxies_log(self): """ BoundLogger.exception.log() is proxied to the apropriate method. """ bl = BoundLogger(ReturnLogger(), [return_method_name], {}) assert "critical" == bl.log(50, "event") assert "debug" == bl.log(10, "event") def test_positional_args_proxied(self): """ Positional arguments supplied are proxied as kwarg. """ bl = BoundLogger(ReturnLogger(), [], {}) args, kwargs = bl.debug('event', 'foo', bar='baz') assert 'baz' == kwargs.get('bar') assert ('foo',) == kwargs.get('positional_args') @pytest.mark.parametrize('method_name,method_args', [ ('addHandler', [None]), ('removeHandler', [None]), ('hasHandlers', None), ('callHandlers', [None]), ('handle', [None]), ('setLevel', [None]), ('getEffectiveLevel', None), ('isEnabledFor', [None]), ('findCaller', None), ('makeRecord', ['name', 'debug', 'test_func', '1', 'test msg', ['foo'], False]), ('getChild', [None]), ]) def test_stdlib_passthrough_methods(self, method_name, method_args): """ stdlib logger methods are also available in stdlib BoundLogger. """ called_stdlib_method = [False] def validate(*args, **kw): called_stdlib_method[0] = True stdlib_logger = logging.getLogger('Test') stdlib_logger_method = getattr(stdlib_logger, method_name, None) if stdlib_logger_method: setattr(stdlib_logger, method_name, validate) bl = BoundLogger(stdlib_logger, [], {}) bound_logger_method = getattr(bl, method_name) assert bound_logger_method is not None if method_args: bound_logger_method(*method_args) else: bound_logger_method() assert called_stdlib_method[0] is True def test_exception_exc_info(self): """ BoundLogger.exception sets exc_info=True. """ bl = BoundLogger(ReturnLogger(), [], {}) assert ((), {"exc_info": True, "event": "event"}) == bl.exception('event') def test_exception_maps_to_error(self): bl = BoundLogger(ReturnLogger(), [return_method_name], {}) assert "error" == bl.exception("event") class TestPositionalArgumentsFormatter(object): def test_formats_tuple(self): """ Positional arguments as simple types are rendered. """ formatter = PositionalArgumentsFormatter() event_dict = formatter(None, None, {'event': '%d %d %s', 'positional_args': (1, 2, 'test')}) assert '1 2 test' == event_dict['event'] assert 'positional_args' not in event_dict def test_formats_dict(self): """ Positional arguments as dict are rendered. """ formatter = PositionalArgumentsFormatter() event_dict = formatter(None, None, {'event': '%(foo)s bar', 'positional_args': ( {'foo': 'bar'},)}) assert 'bar bar' == event_dict['event'] assert 'positional_args' not in event_dict def test_positional_args_retained(self): """ Positional arguments are retained if remove_positional_args argument is set to False. """ formatter = PositionalArgumentsFormatter(remove_positional_args=False) positional_args = (1, 2, 'test') event_dict = formatter( None, None, {'event': '%d %d %s', 'positional_args': positional_args}) assert 'positional_args' in event_dict assert positional_args == event_dict['positional_args'] def test_nop_no_args(self): """ If no positional args are passed, nothing happens. """ formatter = PositionalArgumentsFormatter() assert {} == formatter(None, None, {}) class TestAddLogLevel(object): def test_log_level_added(self): """ The log level is added to the event dict. """ event_dict = add_log_level(None, 'error', {}) assert 'error' == event_dict['level'] def test_log_level_alias_normalized(self): """ The normalized name of the log level is added to the event dict. """ event_dict = add_log_level(None, 'warn', {}) assert 'warning' == event_dict['level'] class TestAddLoggerName(object): def test_logger_name_added(self): """ The logger name is added to the event dict. """ name = 'sample-name' logger = logging.getLogger(name) event_dict = add_logger_name(logger, None, {}) assert name == event_dict['logger'] structlog-15.2.0/tests/test_threadlocal.py0000644000076500000240000001275312432431504021131 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. from __future__ import absolute_import, division, print_function import threading import pytest from structlog._base import BoundLoggerBase from structlog._config import wrap_logger from structlog._compat import OrderedDict from structlog._loggers import ReturnLogger from structlog.threadlocal import as_immutable, wrap_dict, tmp_bind try: import greenlet except ImportError: greenlet = None @pytest.fixture def D(): """ Returns a dict wrapped in _ThreadLocalDictWrapper. """ return wrap_dict(dict) @pytest.fixture def log(): """ Returns a ReturnLogger with a freshly wrapped OrderedDict. """ return wrap_logger(logger(), context_class=wrap_dict(OrderedDict)) @pytest.fixture def logger(): """ Returns a simple logger stub with a *msg* method that takes one argument which gets returned. """ return ReturnLogger() class TestTmpBind(object): def test_bind(self, log): """ tmp_bind does not modify the thread-local state. """ log = log.bind(y=23) with tmp_bind(log, x=42, y='foo') as tmp_log: assert { 'y': 'foo', 'x': 42 } == tmp_log._context._dict == log._context._dict assert {'y': 23} == log._context._dict class TestAsImmutable(object): def test_does_not_affect_global(self, log): """ A logger from as_mutable is independent from thread local state. """ log = log.new(x=42) il = as_immutable(log) assert isinstance(il._context, dict) il = il.bind(y=23) assert {'x': 42, 'y': 23} == il._context assert {'x': 42} == log._context._dict def test_converts_proxy(self, log): """ as_immutable converts a BoundLoggerLazyProxy into a concrete bound logger. """ il = as_immutable(log) assert isinstance(il._context, dict) assert isinstance(il, BoundLoggerBase) def test_idempotency(self, log): """ as_immutable on an as_immutable logger works. """ il = as_immutable(log) assert isinstance(as_immutable(il), BoundLoggerBase) class TestThreadLocalDict(object): def test_wrap_returns_distinct_classes(self): """ Each call to wrap_dict returns a distinct new class whose context is independent from others. """ D1 = wrap_dict(dict) D2 = wrap_dict(dict) assert D1 != D2 assert D1 is not D2 D1.x = 42 D2.x = 23 assert D1.x != D2.x @pytest.mark.skipif(greenlet is not None, reason="Don't mix threads and greenlets.") def test_is_thread_local(self, D): """ The context is *not* shared between threads. """ class TestThread(threading.Thread): def __init__(self, d): self._d = d threading.Thread.__init__(self) def run(self): assert 'tl' not in self._d._dict self._d['tl'] = 23 d = wrap_dict(dict)() d['tl'] = 42 t = TestThread(d) t.start() t.join() assert 42 == d._dict['tl'] def test_context_is_global_to_thread(self, D): """ The context is shared between all instances of a wrapped class. """ d1 = D({'a': 42}) d2 = D({'b': 23}) d3 = D() assert {'a': 42, 'b': 23} == d1._dict == d2._dict == d3._dict assert d1 == d2 == d3 D_ = wrap_dict(dict) d_ = D_({'a': 42, 'b': 23}) assert d1 != d_ def test_init_with_itself_works(self, D): """ Initializing with an instance of the wrapped class will use its values. """ d = D({'a': 42}) assert {'a': 42, 'b': 23} == D(d, b=23)._dict def test_iter_works(self, D): """ ___iter__ is proxied to the wrapped class. """ d = D({'a': 42}) assert ['a'] == list(iter(d)) def test_non_dunder_proxy_works(self, D): """ Calls to a non-dunder method get proxied to the wrapped class. """ d = D({'a': 42}) d.clear() assert 0 == len(d) def test_repr(self, D): """ ___repr__ takes the repr of the wrapped class into account. """ r = repr(D({'a': 42})) assert r.startswith('") @pytest.mark.skipif(greenlet is None, reason="Needs greenlet.") def test_is_greenlet_local(self, D): """ Context is shared between greenlets. """ d = wrap_dict(dict)() d['switch'] = 42 def run(): assert 'x' not in d._dict d['switch'] = 23 greenlet.greenlet(run).switch() assert 42 == d._dict["switch"] def test_delattr(self, D): """ ___delattr__ is proxied to the wrapped class. """ d = D() d['delattr'] = 42 assert 42 == d._dict["delattr"] del d.__class__._tl.dict_ def test_del(self, D): """ ___del__ is proxied to the wrapped class. """ d = D() d['del'] = 13 del d['del'] assert 'del' not in d._dict def test_new_class(self, D): """ The context of a new wrapped class is empty. """ assert 0 == len(D()) structlog-15.2.0/tests/test_twisted.py0000644000076500000240000002414312522630221020323 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. from __future__ import absolute_import, division, print_function import pytest pytest.importorskip('twisted') # noqa import json from pretend import call_recorder from twisted.python.failure import Failure, NoCurrentExceptionError from twisted.python.log import ILogObserver from structlog._config import _CONFIG from structlog._compat import OrderedDict, StringIO, PY3 from structlog._loggers import ReturnLogger from structlog.twisted import ( BoundLogger, EventAdapter, JSONLogObserverWrapper, JSONRenderer, LoggerFactory, PlainFileLogObserver, ReprWrapper, _extractStuffAndWhy, plainJSONStdOutLogger, ) def test_LoggerFactory(): from twisted.python import log assert log is LoggerFactory()() def _render_repr(_, __, event_dict): return repr(event_dict) def build_bl(logger=None, processors=None, context=None): """ Convenience function to build BoundLoggerses with sane defaults. """ return BoundLogger( logger or ReturnLogger(), processors or _CONFIG.default_processors, context if context is not None else _CONFIG.default_context_class(), ) class TestBoundLogger(object): def test_msg(self): bl = build_bl() assert "foo=42 event='event'" == bl.msg('event', foo=42) def test_errVanilla(self): bl = build_bl() assert "foo=42 event='event'" == bl.err('event', foo=42) def test_errWithFailure(self): bl = build_bl(processors=[EventAdapter()]) try: raise ValueError except ValueError: # Use str() for comparison to avoid tricky # deep-compares of Failures. assert ( str(((), {'_stuff': Failure(ValueError()), '_why': "foo=42 event='event'"})) == str(bl.err('event', foo=42)) ) class TestExtractStuffAndWhy(object): def test_extractFailsOnTwoFailures(self): """ Raise ValueError if both _stuff and event contain exceptions. """ with pytest.raises(ValueError) as e: _extractStuffAndWhy({'_stuff': Failure(ValueError()), 'event': Failure(TypeError())}) assert ( "Both _stuff and event contain an Exception/Failure." == e.value.args[0] ) def test_failsOnConflictingEventAnd_why(self): """ Raise ValueError if both _why and event are in the event_dict. """ with pytest.raises(ValueError) as e: _extractStuffAndWhy({'_why': 'foo', 'event': 'bar'}) assert ( "Both `_why` and `event` supplied." == e.value.args[0] ) def test_handlesFailures(self): """ Extracts failures and events. """ assert ( Failure(ValueError()), "foo", {} == _extractStuffAndWhy({"_why": "foo", "_stuff": Failure(ValueError())}) ) assert ( Failure(ValueError()), "error", {} == _extractStuffAndWhy({"_stuff": Failure(ValueError())}) ) def test_handlesMissingFailure(self): """ Missing failures extract a None. """ assert ( (None, "foo", {}) == _extractStuffAndWhy({"event": "foo"}) ) @pytest.mark.xfail(PY3, reason="Py3 does not allow for cleaning exc_info") def test_recognizesErrorsAndCleansThem(self): """ If no error is supplied, the environment is checked for one. If one is found, it's used and cleared afterwards so log.err doesn't add it as well. """ try: raise ValueError except ValueError: f = Failure() _stuff, _why, ed = _extractStuffAndWhy({'event': 'foo'}) assert _stuff.value is f.value with pytest.raises(NoCurrentExceptionError): Failure() class TestEventAdapter(object): """ Some tests here are redundant because they predate _extractStuffAndWhy. """ def test_EventAdapterFormatsLog(self): la = EventAdapter(_render_repr) assert "{'foo': 'bar'}" == la(None, 'msg', {'foo': 'bar'}) def test_transforms_whyIntoEvent(self): """ log.err(_stuff=exc, _why='foo') makes the output 'event="foo"' """ la = EventAdapter(_render_repr) error = ValueError('test') rv = la(None, 'err', { '_stuff': error, '_why': 'foo', 'event': None, }) assert () == rv[0] assert isinstance(rv[1]['_stuff'], Failure) assert error == rv[1]['_stuff'].value assert "{'event': 'foo'}" == rv[1]['_why'] def test_worksUsualCase(self): """ log.err(exc, _why='foo') makes the output 'event="foo"' """ la = EventAdapter(_render_repr) error = ValueError('test') rv = la(None, 'err', {'event': error, '_why': 'foo'}) assert () == rv[0] assert isinstance(rv[1]['_stuff'], Failure) assert error == rv[1]['_stuff'].value assert "{'event': 'foo'}" == rv[1]['_why'] def test_allKeywords(self): """ log.err(_stuff=exc, _why='event') """ la = EventAdapter(_render_repr) error = ValueError('test') rv = la(None, 'err', {'_stuff': error, '_why': 'foo'}) assert () == rv[0] assert isinstance(rv[1]['_stuff'], Failure) assert error == rv[1]['_stuff'].value assert "{'event': 'foo'}" == rv[1]['_why'] def test_noFailure(self): """ log.err('event') """ la = EventAdapter(_render_repr) assert ((), { '_stuff': None, '_why': "{'event': 'someEvent'}", }) == la(None, 'err', { 'event': 'someEvent' }) def test_noFailureWithKeyword(self): """ log.err(_why='event') """ la = EventAdapter(_render_repr) assert ((), { '_stuff': None, '_why': "{'event': 'someEvent'}", }) == la(None, 'err', { '_why': 'someEvent' }) def test_catchesConflictingEventAnd_why(self): la = EventAdapter(_render_repr) with pytest.raises(ValueError) as e: la(None, 'err', { 'event': 'someEvent', '_why': 'someReason', }) assert 'Both `_why` and `event` supplied.' == e.value.args[0] @pytest.fixture def jr(): """ A plain Twisted JSONRenderer. """ return JSONRenderer() class TestJSONRenderer(object): def test_dumpsKWsAreHandedThrough(self, jr): """ JSONRenderer allows for setting arguments that are passed to json.dumps(). Make sure they are passed. """ d = OrderedDict(x='foo') d.update(a='bar') jr_sorted = JSONRenderer(sort_keys=True) assert jr_sorted(None, 'err', d) != jr(None, 'err', d) def test_handlesMissingFailure(self, jr): """ Calling err without an actual failure works and returns the event as a string wrapped in ReprWrapper. """ assert ReprWrapper( '{"event": "foo"}' ) == jr(None, "err", {"event": "foo"})[0][0] assert ReprWrapper( '{"event": "foo"}' ) == jr(None, "err", {"_why": "foo"})[0][0] def test_msgWorksToo(self, jr): """ msg renders the event as a string and wraps it using ReprWrapper. """ assert ReprWrapper( '{"event": "foo"}' ) == jr(None, 'msg', {'_why': 'foo'})[0][0] def test_handlesFailure(self, jr): rv = jr(None, 'err', {'event': Failure(ValueError())})[0][0].string assert 'Failure: {0}.ValueError'.format("builtins" if PY3 else "exceptions") in rv assert '"event": "error"' in rv def test_setsStructLogField(self, jr): """ Formatted entries are marked so they can be identified without guessing for example in JSONLogObserverWrapper. """ assert {'_structlog': True} == jr(None, 'msg', {'_why': 'foo'})[1] class TestReprWrapper(object): def test_repr(self): """ The repr of the wrapped string is the vanilla string without quotes. """ assert "foo" == repr(ReprWrapper("foo")) class TestPlainFileLogObserver(object): def test_isLogObserver(self): assert ILogObserver.providedBy(PlainFileLogObserver(StringIO())) def test_writesOnlyMessageWithLF(self): sio = StringIO() PlainFileLogObserver(sio)({'system': 'some system', 'message': ('hello',)}) assert 'hello\n' == sio.getvalue() class TestJSONObserverWrapper(object): def test_IsAnObserver(self): assert ILogObserver.implementedBy(JSONLogObserverWrapper) def test_callsWrappedObserver(self): """ The wrapper always runs the wrapped observer in the end. """ o = call_recorder(lambda *a, **kw: None) JSONLogObserverWrapper(o)({'message': ('hello',)}) assert 1 == len(o.calls) def test_jsonifiesPlainLogEntries(self): """ Entries that aren't formatted by JSONRenderer are rendered as JSON now. """ o = call_recorder(lambda *a, **kw: None) JSONLogObserverWrapper(o)({'message': ('hello',), 'system': '-'}) msg = json.loads(o.calls[0].args[0]['message'][0]) assert msg == {'event': 'hello', 'system': '-'} def test_leavesStructLogAlone(self): """ Entries that are formatted by JSONRenderer are left alone. """ d = {'message': ('hello',), '_structlog': True} def verify(eventDict): assert d == eventDict JSONLogObserverWrapper(verify)(d) class TestPlainJSONStdOutLogger(object): def test_isLogObserver(self): assert ILogObserver.providedBy(plainJSONStdOutLogger()) structlog-15.2.0/tests/test_utils.py0000644000076500000240000000203512432431527020004 0ustar hynekstaff00000000000000# This file is dual licensed under the terms of the Apache License, Version # 2.0, and the MIT License. See the LICENSE file in the root of this # repository for complete details. from __future__ import absolute_import, division, print_function import errno import pytest from pretend import raiser from structlog._utils import until_not_interrupted class TestUntilNotInterrupted(object): def test_passes_arguments_and_returns_return_value(self): def returner(*args, **kw): return args, kw assert ((42,), {'x': 23}) == until_not_interrupted(returner, 42, x=23) def test_leaves_unrelated_exceptions_through(self): exc = IOError with pytest.raises(exc): until_not_interrupted(raiser(exc('not EINTR'))) def test_retries_on_EINTR(self): calls = [0] def raise_on_first_three(): if calls[0] < 3: calls[0] += 1 raise IOError(errno.EINTR) until_not_interrupted(raise_on_first_three) assert 3 == calls[0] structlog-15.2.0/tox.ini0000644000076500000240000000153512522576014015411 0ustar hynekstaff00000000000000[tox] envlist = {py26,py27,py33,py34,pypy}-{threads,greenlets}, flake8-{py2,py3}, docs, manifest [testenv] deps = setuptools>=7.0 coverage py26: ordereddict greenlets: greenlet setenv = PYTHONHASHSEED = 0 threads: TRICKING_TOX_INTO_GENERATING_AN_ENVIRONMENT = 1 commands = coverage run setup.py test coverage report [testenv:flake8-py2] deps = flake8 commands = flake8 structlog tests [testenv:flake8-py3] basepython = py3: python3.4 deps = flake8 commands = flake8 structlog tests [testenv:docs] setenv = PYTHONHASHSEED = 0 deps = releases sphinx==1.2.3 twisted commands = sphinx-build -W -b html -d {envtmpdir}/doctrees docs docs/_build/html sphinx-build -W -b doctest -d {envtmpdir}/doctrees docs docs/_build/html [testenv:manifest] deps = check-manifest commands = check-manifest