pax_global_header00006660000000000000000000000064136556473560014536gustar00rootroot0000000000000052 comment=743430d0d57043854c3a0eb82be9b1041dfbaba5 PyStaticConfiguration-0.10.5/000077500000000000000000000000001365564735600161115ustar00rootroot00000000000000PyStaticConfiguration-0.10.5/.gitignore000066400000000000000000000004201365564735600200750ustar00rootroot00000000000000*.py[co] # Packages *.egg *.egg-info dist build eggs parts bin var sdist develop-eggs .installed.cfg # Installer logs pip-log.txt # Unit test / coverage reports .coverage .tox htmlcov/ #Translations *.mo #Mr Developer .mr.developer.cfg .idea .*.sw[po] .dobi .cache PyStaticConfiguration-0.10.5/.landscape.yaml000066400000000000000000000000761365564735600210100ustar00rootroot00000000000000 strictness: high ignore-patterns: - docs/source/conf.py PyStaticConfiguration-0.10.5/.pyautotest000066400000000000000000000001531365564735600203320ustar00rootroot00000000000000test_runner_name: full_suite command: - 'py.test' - '-v' - '-s' - '--tb=short' - tests PyStaticConfiguration-0.10.5/.travis.yml000066400000000000000000000005101365564735600202160ustar00rootroot00000000000000language: python matrix: include: - env: TOXENV=py27 - env: TOXENV=py34 - env: TOXENV=py35 python: 3.5 - env: TOXENV=py36 python: 3.6 - env: TOXENV=pypy python: pypy - env: TOXENV=docs - env: TOXENV=coverage install: pip install tox coveralls script: tox after_success: coveralls PyStaticConfiguration-0.10.5/LICENSE000066400000000000000000000261361365564735600171260ustar00rootroot00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. PyStaticConfiguration-0.10.5/NOTICE000066400000000000000000000011051365564735600170120ustar00rootroot00000000000000Copyright 2012 Daniel Nephin Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. PyStaticConfiguration-0.10.5/README.rst000066400000000000000000000071441365564735600176060ustar00rootroot00000000000000PyStaticConfiguration ===================== A python library for loading, validating and reading configuration from many heterogeneous formats. Configuration is split into two phases. Configuration Loading --------------------- Configuration is read from files or python objects, flattened, and merged into a container called a `namespace`. Namespaces are used to separate unrelated configuration groups. If configuration is changed frequently, it can also be reloaded easily with very little change to the existing code. Configuration Reading --------------------- A configuration value is looked up in the `namespace`. It is validating and converted to the requested type. .. contents:: Contents :local: :depth: 1 :backlinks: none Build Status ------------ .. image:: https://travis-ci.org/dnephin/PyStaticConfiguration.svg?branch=master :target: https://travis-ci.org/dnephin/PyStaticConfiguration :alt: Travis CI build status .. image:: https://img.shields.io/pypi/v/PyStaticConfiguration.svg :target: https://pypi.python.org/pypi/PyStaticConfiguration :alt: Latest PyPI version .. image:: https://coveralls.io/repos/github/dnephin/PyStaticConfiguration/badge.svg?branch=master :target: https://coveralls.io/github/dnephin/PyStaticConfiguration?branch=master :alt: Code Test Coverage Install ------- * PyStaticConfiguration is available on pypi: https://pypi.python.org/pypi/PyStaticConfiguration * The source is hosted on github: https://github.com/dnephin/PyStaticConfiguration .. code-block:: bash $ pip install PyStaticConfiguration Also see the `release notes `_. Documentation ------------- http://pythonhosted.org/PyStaticConfiguration/ Examples -------- A common minimal use of staticconf would be to use a single yaml configuration file and read some values from it. .. code-block:: python import staticconf filename = 'hosts.yaml' namespace = 'hosts' # Load configuration from the file into namespace `hosts` staticconf.YamlConfiguration(filename, namespace=namespace) ... # Some time later on, read values from that namespace print staticconf.read('database.slave', namespace=namespace) print staticconf.read('database.master', namespace=namespace) `hosts.yaml` might look something like this: .. code-block:: yaml database: slave: dbslave_1 master: dbmaster_1 A more involved example would load configuration from multiple files, create a watcher for reloading, and read some config values. .. code-block:: python from functools import partial import os import staticconf def load_config(config_path, defaults='~/.myapp.yaml`) # First load global defaults if the file exists staticconf.INIConfiguration('/etc/myapp.ini', optional=True) # Next load user defaults staticconf.YamlConfiguration(defaults, optional=True) # Next load the specified configuration file staticconf.YamlConfiguration(config_path) # Now let's override it with some environment settings staticconf.DictConfiguration( (k[6:].lower(), v) for k, v in os.environ.items() if k.startswith('MYAPP_')) def build_watcher(filename): return staticconf.ConfigFacade.load( filenames, 'DEFAULT', partial(load_config, filename)) def run(config_path): watcher = build_watcher(config_path) while is_work(): watcher.reload_if_changed() current_threshold = staticconf.read_float('current_threshold') do_some_work(current_thresold) PyStaticConfiguration-0.10.5/dobi.yaml000066400000000000000000000015621365564735600177160ustar00rootroot00000000000000 meta: project: pystaticconf default: test mount=source: bind: . path: /work mount=pypiconf: bind: ~/.pypirc path: /root/.pypirc file: true image=builder: image: staticconf-dev context: dockerfiles/ dockerfile: Dockerfile job=test: use: builder mounts: [source] interactive: true command: tox env: ['TOXENV={env.TOXENV:}'] job=shell: use: builder mounts: [source, pypiconf] interactive: true command: bash job=clean: use: builder mounts: [source] command: "find -name *.pyc -o -name __pycache__ -exec rm -rf {} \\;" job=docs: use: builder mounts: [source] command: "tox -e docs" artifact: docs/build job=release: use: builder mounts: [source, pypiconf] interactive: true depends: [docs] command: "python3 setup.py sdist bdist_wheel upload upload_docs" PyStaticConfiguration-0.10.5/dockerfiles/000077500000000000000000000000001365564735600204035ustar00rootroot00000000000000PyStaticConfiguration-0.10.5/dockerfiles/Dockerfile000066400000000000000000000012151365564735600223740ustar00rootroot00000000000000 FROM ubuntu:xenial ENV LANG C.UTF-8 RUN export DEBIAN_FRONTEND=noninteractive; \ apt-get update && apt-get install -y --no-install-recommends \ software-properties-common RUN export DEBIAN_FRONTEND=noninteractive; \ add-apt-repository ppa:deadsnakes/ppa && \ apt-get update && apt-get install -y --no-install-recommends \ python2.6 \ python2.7 \ python3.4 \ python3.5 \ pypy \ curl RUN curl -Ls https://bootstrap.pypa.io/get-pip.py | python3 RUN pip install \ tox \ yapyautotest WORKDIR /work PyStaticConfiguration-0.10.5/docs/000077500000000000000000000000001365564735600170415ustar00rootroot00000000000000PyStaticConfiguration-0.10.5/docs/source/000077500000000000000000000000001365564735600203415ustar00rootroot00000000000000PyStaticConfiguration-0.10.5/docs/source/conf.py000066400000000000000000000017641365564735600216500ustar00rootroot00000000000000# -*- coding: utf-8 -*- import staticconf import sphinx_rtd_theme # -- General configuration ----------------------------------------------------- extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.viewcode', 'sphinx.ext.intersphinx', ] templates_path = ['_templates'] source_suffix = '.rst' master_doc = 'index' # General information about the project. project = u'PyStaticConfiguration' copyright = u'2013, Daniel Nephin' version = staticconf.version release = version exclude_patterns = [] pygments_style = 'sphinx' # -- Options for HTML output --------------------------------------------------- html_theme = 'sphinx_rtd_theme' html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] html_static_path = [] htmlhelp_basename = 'PyStaticConfigurationdoc' # -- Extensions ---------------------------------------------------------------- autodoc_member_order = 'groupwise' intersphinx_mapping = { 'http://docs.python.org/': None, } PyStaticConfiguration-0.10.5/docs/source/config.rst000066400000000000000000000006141365564735600223410ustar00rootroot00000000000000 Config ====== .. automodule:: staticconf.config :members: ConfigNamespace, get_namespace, validate, view_help, ConfigurationWatcher, IComparator, InodeComparator, MTimeComparator, MD5Comparator, ReloadCallbackChain, ConfigFacade, reload Errors ------ .. automodule:: staticconf.errors :members: PyStaticConfiguration-0.10.5/docs/source/getters.rst000066400000000000000000000002021365564735600225420ustar00rootroot00000000000000 Getters ======= .. automodule:: staticconf.getters :members: Proxy ----- .. automodule:: staticconf.proxy :members: PyStaticConfiguration-0.10.5/docs/source/index.rst000066400000000000000000000022451365564735600222050ustar00rootroot00000000000000 PyStaticConfiguration Documentation =================================== Build Status ------------ .. image:: https://travis-ci.org/dnephin/PyStaticConfiguration.svg?branch=master :target: https://travis-ci.org/dnephin/PyStaticConfiguration :alt: Travis CI build status .. image:: https://img.shields.io/pypi/v/PyStaticConfiguration.svg :target: https://pypi.python.org/pypi/PyStaticConfiguration :alt: Latest PyPI version .. image:: https://coveralls.io/repos/github/dnephin/PyStaticConfiguration/badge.svg?branch=master :target: https://coveralls.io/github/dnephin/PyStaticConfiguration?branch=master :alt: Code Test Coverage Install ------- * PyStaticConfiguration is available on pypi: https://pypi.python.org/pypi/PyStaticConfiguration * The source is hosted on github: https://github.com/dnephin/PyStaticConfiguration .. code-block:: bash $ pip install PyStaticConfiguration Also see the :doc:`release_notes` Contents -------- .. toctree:: :maxdepth: 2 overview config loader validation readers schema getters testing Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` PyStaticConfiguration-0.10.5/docs/source/loader.rst000066400000000000000000000001641365564735600223420ustar00rootroot00000000000000 Configuration Loading ===================== .. automodule:: staticconf.loader :members: :undoc-members: PyStaticConfiguration-0.10.5/docs/source/overview.rst000066400000000000000000000162541365564735600227510ustar00rootroot00000000000000Overview ======== :mod:`staticconf` is a library for loading configuration values from many heterogeneous formats and reading those values. This process is split into two phases. Configuration Loading --------------------- Configuration is read from files or python objects (:class:`dict`, :class:`list`, or any :class:`object` with attributes), flattened, and merged into a container called a :class:`staticconf.config.ConfigNamespace`. Namespaces are used to group related configuration together. If configuration is changed frequently, it can also be reloaded easily with very little change to the existing code. See :mod:`staticconf.loader` for a list of supported file formats and config loading examples. See :func:`staticconf.config.ConfigFacade.load` for examples of building a reloader. See :class:`staticconf.config.ConfigNamespace` for more details about namespaces. Reading configuration values ---------------------------- Once configuration data is loaded into a :class:`staticconf.config.ConfigNamespace` there are three options for retrieving the configuration values. All of them have a similar set of methods which use validators to ensure you're getting the type you expect. When a value is missing they will raise :class:`staticconf.errors.ConfigurationError` unless a default was given. They will raises :class:`staticconf.errors.ValidationError` if the value in the config fails to validate. The list of provided validators is :mod:`staticconf.validation`, but you can create custom validators for any type. Functions for reading configuration values are named using a convention based on the validator name. For example the methods for getting a date using :func:`staticconf.validation.validate_date` would be: * ``staticconf.read_date()`` * ``schema.date()`` * ``staticconf.get_date()`` Readers ~~~~~~~ :mod:`staticconf.readers` Readers are the most straightforward option. They simply lookup the value in the namespace and return it. There is no caching of the type-coercion. Schema ~~~~~~ :mod:`staticconf.schema` Schemas are classes where each property is an accessor for a configuration value. The type-coercion is cached on the class. This can be useful if you're performing more involved coercion (such as converting a large list to a mapping, or building complex types). Getters ~~~~~~~ :mod:`staticconf.getters` Getters have the same properties as schema accessors, but do not use a class. See the module documentation for some limitations with this option. Example ------- For this example we'll use yaml configuration files. Given two files, a `application.yaml`: .. code-block:: yaml pid: /var/run/app1.pid storage_paths: - /mnt/storage - /mnt/nfs min_date: 2014-12-12 groups: users: - userone - usertwo admins: - admin And an `overrides.yaml` .. code-block:: yaml max_files: 10 groups: users: - customuser First load some configuration from a file. This is often done during the "startup" phase of an application, such as after :mod:`argparse` has completed (potentially where one of the command line args is a config filename). For a web application, this might happen during the initialization of the webapp. .. code-block:: python import staticconf app_config = 'application.yaml' app_custom = 'overrides.yaml' YamlConfiguration(app_config) YamlConfiguration(app_custom, optional=True) Now we've loaded up our application config, and overridden it with the data from `overrides.yaml`. `overrides.yaml` was optional, so if the file was missing there would be no error. Next we'll want to read these values at some point. .. code-block:: python import staticconf pid = staticconf.read_string('pid') storage_paths = staticconf.read_list_of_string('storage_paths') # This is the just the list of one user `customuser` since we loaded our # `overrides.yaml` over the original list # Also notice the key was flattened, so we use a dotted notation users = staticconf.read_list_of_string('groups.users') # Using doted notation allows us to preserve any part of the mapping # structure, so in this case, the admins from `application.yaml` are # still there admins = staticconf.read_list_of_string('groups.admins') # We can also read other types. In our config this was a string, but we're # reading a date, so we receive a datetime.date object min_date = staticconf.read_date('min_date') See :class:`staticconf.config.ConfigFacade` for examples of how to reload configuration on changes. Logging ------- :mod:`staticconf` logs a message at `INFO` level when configuration is loaded with the message "Unexpected value in configuration: ...", with the list of unexpected values. Unexpected values are those that haven't been registered by a :mod:`staticconf.schema` or :mod:`staticconf.getter`. This message is used to debug configuration errors. If you'd like to disable it you can set the logging level for `staticconf.config` to `WARN`. Reading dicts ------------- By default :mod:`staticconf` flattens all the values it receives from the loaders. There are two ways to get dicts from a loader. Disable Flatten ~~~~~~~~~~~~~~~ You can call loaders with the kwargs ``flatten=False``. Example: .. code-block:: python YamlConfiguration(filename, flatten=False) The disadvantage with this approach is that the entire config file will preserve its nested structure, so you lose out of the ability to easily merge and override configuration files. Custom Reader ~~~~~~~~~~~~~ The second option is to represent a dict structures using lists of values (either a list of pairs or a list of dicts). This list can then be converted into a dict mapping using a custom getter/reader. Below are some examples on how this is done. The :mod:`staticconf.readers` interface is used as an example, but the same can be done for the :mod:`staticconf.getters` and :mod:`staticconf.schema` interfaces by replacing :func:`staticconf.readers.build_reader` with :func:`staticconf.getters.build_getter` or :func:`staticconf.schema.build_value_type`. Create a reader which translates a list of dicts into a mapping .. code-block:: python from staticconf import validation, readers def build_map_from_key_value(item): return item['key'], item['value'] read_mapping = readers.build_reader( validation.build_map_type_validator(build_map_from_key_value)) my_mapping = read_mapping('config_key_of_a_list_of_dicts') Create a reader which translates a list of pairs into a mapping .. code-block:: python from staticconf import validation, readers read_mapping = readers.build_reader( validation.build_map_type_validator(tuple)) my_mapping = read_mapping('config_key_of_a_list_of_pairs') Create a reader from translates a list of complex dicts into a mapping .. code-block:: python from staticconf import validation, readers def build_map_from_dicts(item): return item.pop('name'), item read_mapping = readers.build_reader( validation.build_map_type_validator(build_map_from_dicts)) my_mapping = read_mapping('config_key_of_a_list_of_dicts') PyStaticConfiguration-0.10.5/docs/source/readers.rst000066400000000000000000000001431365564735600225160ustar00rootroot00000000000000 Readers ======= .. automodule:: staticconf.readers :members: NamespaceReaders, build_reader PyStaticConfiguration-0.10.5/docs/source/release_notes.rst000066400000000000000000000013201365564735600237170ustar00rootroot00000000000000 Release Notes ============= v0.10.2 ------- * add ``log_keys_only`` to loaders to prevent the printing of unknown values. v0.10.1 ------- * add ``compare_func`` support to ``MTimeComparator`` and adds ``build_compare_func`` helper * fixes an error in ``_validate_iterable`` v0.10.0 ------- * add ``get_config_dict()`` to retrieve a mapping * remove ``InodeComparator`` from the list of default comparators * add ``PatchConfiguration`` for testing v0.9.0 ------ * support different file comparison strategies for the ConfigurationWatcher v0.7.1 ------ * bug fixes * test failure fixes * python3 support v0.7.0 ------ * `flatten` kwarg added to config loaders to disable dict flattening when the config is loaded PyStaticConfiguration-0.10.5/docs/source/schema.rst000066400000000000000000000001461365564735600223340ustar00rootroot00000000000000 Schema ====== .. automodule:: staticconf.schema :members: build_value_type, SchemaMeta, Schema PyStaticConfiguration-0.10.5/docs/source/testing.rst000066400000000000000000000001041365564735600225430ustar00rootroot00000000000000 Testing ======= .. automodule:: staticconf.testing :members: PyStaticConfiguration-0.10.5/docs/source/validation.rst000066400000000000000000000001411365564735600232210ustar00rootroot00000000000000 Validation ========== .. automodule:: staticconf.validation :members: :undoc-members: PyStaticConfiguration-0.10.5/setup.cfg000066400000000000000000000003771365564735600177410ustar00rootroot00000000000000[build_docs] source-dir = docs/ build-dir = docs/build all_files = 1 [upload_docs] upload-dir = docs/build/html [wheel] universal = True [metadata] license_file = LICENSE [flake8] max-line-length = 85 max-complexity = 8 ignore = E126,E241,E221,E128 PyStaticConfiguration-0.10.5/setup.py000066400000000000000000000022551365564735600176270ustar00rootroot00000000000000import os.path from setuptools import setup about = {} version_path = os.path.join(os.path.dirname(__file__), 'staticconf', 'version.py') with open(version_path) as f: exec(f.read(), about) setup( name="PyStaticConfiguration", version=about['version'], provides=["staticconf"], author="Daniel Nephin", author_email="dnephin@gmail.com", url="https://github.com/dnephin/PyStaticConfiguration", description='A python library for loading static configuration', classifiers=[ "Programming Language :: Python", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Operating System :: OS Independent", "License :: OSI Approved :: Apache Software License", "Intended Audience :: Developers", "Development Status :: 5 - Production/Stable", ], extras_require={ 'yaml': ['pyyaml'], }, packages=['staticconf'], install_requires=['six'], license='APACHE20', ) PyStaticConfiguration-0.10.5/staticconf/000077500000000000000000000000001365564735600202465ustar00rootroot00000000000000PyStaticConfiguration-0.10.5/staticconf/__init__.py000066400000000000000000000006131365564735600223570ustar00rootroot00000000000000from . import config # noqa: F401 from .version import version # noqa: F401 from .loader import * # noqa: F401,F403 from .getters import * # noqa: F401,F403 from .readers import * # noqa: F401,F403 view_help = config.view_help reload = config.reload validate = config.validate ConfigurationWatcher = config.ConfigurationWatcher ConfigFacade = config.ConfigFacade PyStaticConfiguration-0.10.5/staticconf/config.py000066400000000000000000000506321365564735600220730ustar00rootroot00000000000000""" Store configuration in :class:`ConfigNamespace` objects and provide tools for reloading, and displaying help messages. Configuration Reloading ----------------------- Configuration reloading is supported using a :class:`ConfigFacade`, which composes a :class:`ConfigurationWatcher` and a :class:`ReloadCallbackChain`. These classes provide a way of reloading configuration when the file is modified. """ from collections import namedtuple import hashlib import logging import os import time import weakref import six from staticconf import errors log = logging.getLogger(__name__) # Name for the default namespace DEFAULT = 'DEFAULT' def remove_by_keys(dictionary, keys): keys = set(keys) def filter_by_keys(item): k, _ = item return k not in keys return list(filter(filter_by_keys, six.iteritems(dictionary))) class ConfigMap(object): """A ConfigMap can be used to wrap a dictionary in your configuration. It will allow you to retain your mapping structure (and prevent it from being flattened). """ def __init__(self, *args, **kwargs): self.data = dict(*args, **kwargs) def __getitem__(self, item): return self.data[item] def get(self, item, default=None): return self.data.get(item, default) def __contains__(self, item): return item in self.data def __len__(self): return len(self.data) class ConfigNamespace(object): """A container for related configuration values. Values are stored using flattened keys which map to values. Values are added to this container using :mod:`staticconf.loader`. When a :class:`ConfigNamespace` is created, it persists for the entire life of the process. Values will stay in the namespace until :func:`clear` is called to remove them. To retrieve a namespace, use :func:`get_namespace`. To access values stored in this namespace use :mod:`staticconf.readers` or :mod:`staticconf.schema`. """ def __init__(self, name): self.name = name self.configuration_values = {} self.value_proxies = weakref.WeakValueDictionary() def get_name(self): return self.name def get_value_proxies(self): return list(self.value_proxies.values()) def register_proxy(self, proxy): self.value_proxies[id(proxy)] = proxy def apply_config_data( self, config_data, error_on_unknown, error_on_dupe, log_keys_only=False, ): self.validate_keys( config_data, error_on_unknown, log_keys_only=log_keys_only, ) self.has_duplicate_keys(config_data, error_on_dupe) self.update_values(config_data) def update_values(self, *args, **kwargs): self.configuration_values.update(*args, **kwargs) def get_config_values(self): """Return all configuration stored in this object as a dict. """ return self.configuration_values def get_config_dict(self): """Reconstruct the nested structure of this object's configuration and return it as a dict. """ config_dict = {} for dotted_key, value in self.get_config_values().items(): subkeys = dotted_key.split('.') d = config_dict for key in subkeys: d = d.setdefault(key, value if key == subkeys[-1] else {}) return config_dict def get_known_keys(self): return set(vproxy.config_key for vproxy in self.get_value_proxies()) def validate_keys( self, config_data, error_on_unknown, log_keys_only=False, ): unknown = remove_by_keys(config_data, self.get_known_keys()) if not unknown: return if log_keys_only: unknown = [k for k, _ in unknown] msg = "Unexpected value in %s configuration: %s" % (self.name, unknown) if error_on_unknown: raise errors.ConfigurationError(msg) log.info(msg) def has_duplicate_keys(self, config_data, error_on_duplicate): args = config_data, self.configuration_values, error_on_duplicate return has_duplicate_keys(*args) def get(self, item, default=None): return self.configuration_values.get(item, default) def __getitem__(self, item): return self.configuration_values[item] def __setitem__(self, key, value): self.configuration_values[key] = value def __contains__(self, item): return item in self.configuration_values def clear(self): """Remove all values from the namespace.""" self.configuration_values.clear() def _reset(self): self.clear() self.value_proxies.clear() def __str__(self): return "%s(%s)" % (type(self).__name__, self.name) configuration_namespaces = {DEFAULT: ConfigNamespace(DEFAULT)} KeyDescription = namedtuple('KeyDescription', 'name validator default help') def get_namespaces_from_names(name, all_names): """Return a generator which yields namespace objects.""" names = configuration_namespaces.keys() if all_names else [name] for name in names: yield get_namespace(name) def get_namespace(name): """Return a :class:`ConfigNamespace` by name, creating the namespace if it does not exist. """ if name not in configuration_namespaces: configuration_namespaces[name] = ConfigNamespace(name) return configuration_namespaces[name] def reload(name=DEFAULT, all_names=False): """Reload one or all :class:`ConfigNamespace`. Reload clears the cache of :mod:`staticconf.schema` and :mod:`staticconf.getters`, allowing them to pickup the latest values in the namespace. Defaults to reloading just the DEFAULT namespace. :param name: the name of the :class:`ConfigNamespace` to reload :param all_names: If True, reload all namespaces, and ignore `name` """ for namespace in get_namespaces_from_names(name, all_names): for value_proxy in namespace.get_value_proxies(): value_proxy.reset() def validate(name=DEFAULT, all_names=False): """Validate all registered keys after loading configuration. Missing values or values which do not pass validation raise :class:`staticconf.errors.ConfigurationError`. By default only validates the `DEFAULT` namespace. :param name: the namespace to validate :type name: string :param all_names: if True validates all namespaces and ignores `name` :type all_names: boolean """ for namespace in get_namespaces_from_names(name, all_names): all(value_proxy.get_value() for value_proxy in namespace.get_value_proxies()) class ConfigHelp(object): """Register and display help messages about config keys.""" def __init__(self): self.descriptions = {} def add(self, name, validator, default, namespace, help): desc = KeyDescription(name, validator, default, help) self.descriptions.setdefault(namespace, []).append(desc) def view_help(self): """Return a help message describing all the statically configured keys. """ def format_desc(desc): return "%s (Type: %s, Default: %s)\n%s" % ( desc.name, desc.validator.__name__.replace('validate_', ''), desc.default, desc.help or '') def format_namespace(key, desc_list): return "\nNamespace: %s\n%s" % ( key, '\n'.join(sorted(format_desc(desc) for desc in desc_list))) def namespace_cmp(item): name, _ = item return chr(0) if name == DEFAULT else name return '\n'.join(format_namespace(*desc) for desc in sorted(six.iteritems(self.descriptions), key=namespace_cmp)) def clear(self): self.descriptions.clear() config_help = ConfigHelp() view_help = config_help.view_help def _reset(): """Used for internal testing.""" for namespace in configuration_namespaces.values(): namespace._reset() config_help.clear() def has_duplicate_keys(config_data, base_conf, raise_error): """Compare two dictionaries for duplicate keys. if raise_error is True then raise on exception, otherwise log return True.""" duplicate_keys = set(base_conf) & set(config_data) if not duplicate_keys: return msg = "Duplicate keys in config: %s" % duplicate_keys if raise_error: raise errors.ConfigurationError(msg) log.info(msg) return True class ConfigurationWatcher(object): """Watches a file for modification and reloads the configuration when it's modified. Accepts a min_interval to throttle checks. The default :func:`reload()` operation is to reload all namespaces. To only reload a specific namespace use a :class:`ReloadCallbackChain` for the `reloader`. .. seealso:: :func:`ConfigFacade.load` which provides a more concise interface for the common case. Usage: .. code-block:: python import staticconf from staticconf import config def build_configuration(filename, namespace): config_loader = partial(staticconf.YamlConfiguration, filename, namespace=namespace) reloader = config.ReloadCallbackChain(namespace) return config.ConfigurationWatcher( config_loader, filename, min_interval=2, reloader=reloader) config_watcher = build_configuration('config.yaml', 'my_namespace') # Load the initial configuration config_watcher.config_loader() # Do some work for item in work: config_watcher.reload_if_changed() ... :param config_loader: a function which takes no arguments. It is called by :func:`reload_if_changed` if the file has been modified :param filenames: a filename or list of filenames to watch for modifications :param min_interval: minimum number of seconds to wait between calls to :func:`os.path.getmtime` to check if a file has been modified. :param reloader: a function which is called after `config_loader` when a file has been modified. Defaults to an empty :class:`ReloadCallbackChain` :param comparators: a list of classes which support the :class:`IComparator` interface which are used to determine if a config file has been modified. Defaults to :class:`MTimeComparator`. """ def __init__( self, config_loader, filenames, min_interval=0, reloader=None, comparators=None): self.config_loader = config_loader self.filenames = self.get_filename_list(filenames) self.min_interval = min_interval self.last_check = time.time() self.reloader = reloader or ReloadCallbackChain(all_names=True) comparators = comparators or [MTimeComparator] self.comparators = [comp(self.filenames) for comp in comparators] def get_filename_list(self, filenames): if isinstance(filenames, six.string_types): filenames = [filenames] filenames = sorted(os.path.abspath(name) for name in filenames) if not filenames: raise ValueError( "ConfigurationWatcher requires at least one filename to watch") return filenames @property def should_check(self): return self.last_check + self.min_interval <= time.time() def reload_if_changed(self, force=False): """If the file(s) being watched by this object have changed, their configuration will be loaded again using `config_loader`. Otherwise this is a noop. :param force: If True ignore the `min_interval` and proceed to file modified comparisons. To force a reload use :func:`reload` directly. """ if (force or self.should_check) and self.file_modified(): return self.reload() def file_modified(self): self.last_check = time.time() return any(comp.has_changed() for comp in self.comparators) def reload(self): config_dict = self.config_loader() self.reloader() return config_dict def get_reloader(self): return self.reloader def load_config(self): return self.config_loader() class IComparator(object): """Interface for a comparator which is used by :class:`ConfigurationWatcher` to determine if a file has been modified since the last check. A comparator is used to reduce the work required to reload configuration. Comparators should implement a mechanism that is relatively efficient (and scalable), so it can be performed frequently. :param filenames: A list of absolute paths to configuration files. """ def __init__(self, filenames): pass def has_changed(self): """Returns True if any of the files have been modified since the last call to :func:`has_changed`. Returns False otherwise. """ pass class InodeComparator(object): """Compare files by inode and device number. This is a good comparator to use when your files can change multiple times per second. """ def __init__(self, filenames): self.filenames = filenames self.inodes = self.get_inodes() def get_inodes(self): def get_inode(stbuf): return stbuf.st_dev, stbuf.st_ino return [get_inode(os.stat(filename)) for filename in self.filenames] def has_changed(self): last_inodes, self.inodes = self.inodes, self.get_inodes() return last_inodes != self.inodes def build_compare_func(err_logger=None): """Returns a compare_func that can be passed to MTimeComparator. The returned compare_func first tries os.path.getmtime(filename), then calls err_logger(filename) if that fails. If err_logger is None, then it does nothing. err_logger is always called within the context of an OSError raised by os.path.getmtime(filename). Information on this error can be retrieved by calling sys.exc_info inside of err_logger.""" def compare_func(filename): try: return os.path.getmtime(filename) except OSError: if err_logger is not None: err_logger(filename) return -1 return compare_func class MTimeComparator(object): """Compare files by modified time, or using compare_func, if it is not None. .. note:: Most filesystems only store modified time with second grangularity so multiple changes within the same second can be ignored. """ def __init__(self, filenames, compare_func=None): self.compare_func = (os.path.getmtime if compare_func is None else compare_func) self.filenames_mtimes = { filename: self.compare_func(filename) for filename in filenames } def has_changed(self): for filename, compare_val in self.filenames_mtimes.items(): current_compare_val = self.compare_func(filename) if compare_val != current_compare_val: self.filenames_mtimes[filename] = current_compare_val return True return False class MD5Comparator(object): """Compare files by md5 hash of their contents. This comparator will be slower for larger files, but is more resilient to modifications which only change mtime, but not the files contents. """ def __init__(self, filenames): self.filenames = filenames self.hashes = self.get_hashes() def get_hashes(self): def build_hash(filename): hasher = hashlib.md5() with open(filename, 'rb') as fh: hasher.update(fh.read()) return hasher.digest() return [build_hash(filename) for filename in self.filenames] def has_changed(self): last_hashes, self.hashes = self.hashes, self.get_hashes() return last_hashes != self.hashes class ReloadCallbackChain(object): """A chain of callbacks which will be triggered after configuration is reloaded. Designed to work with :class:`ConfigurationWatcher`. When this class is called it performs two operations: * calls :func:`reload` on the `namespace` * calls all attached callbacks Usage: .. code-block:: python chain = ReloadCallbackChain() chain.add('some_id', callback_foo) chain.add('other_id', other_callback) ... # some time later chain.remove('some_id') :param namespace: the name of the namespace to :func:`reload` :param all_names: if True :func:`reload` all namespaces and ignore the `namespace` param. Defaults to False :param callbacks: initial list of tuples to add to the callback chain """ def __init__(self, namespace=DEFAULT, all_names=False, callbacks=None): self.namespace = namespace self.all_names = all_names self.callbacks = dict(callbacks or ()) def add(self, identifier, callback): self.callbacks[identifier] = callback def remove(self, identifier): del self.callbacks[identifier] def __call__(self): reload(name=self.namespace, all_names=self.all_names) for callback in six.itervalues(self.callbacks): callback() def build_loader_callable(load_func, filename, namespace): def load_configuration(): get_namespace(namespace).clear() return load_func(filename, namespace=namespace) return load_configuration class ConfigFacade(object): """A facade around a :class:`ConfigurationWatcher` and a :class:`ReloadCallbackChain`. See :func:`ConfigFacade.load`. When a :class:`ConfigFacade` is loaded it will clear the namespace of all configuration and load the file into the namespace. If this is not the behaviour you want, use a :class:`ConfigurationWatcher` instead. Usage: .. code-block:: python import staticconf watcher = staticconf.ConfigFacade.load( 'config.yaml', # Filename or list of filenames to watch 'my_namespace', staticconf.YamlConfiguration, # Callable which takes the filename min_interval=3 # Wait at least 3 seconds before checking modified time ) watcher.add_callback('identifier', do_this_after_reload) watcher.reload_if_changed() """ def __init__(self, watcher): self.watcher = watcher self.callback_chain = watcher.get_reloader() @classmethod def load( cls, filename, namespace, loader_func, min_interval=0, comparators=None, ): """Create a new :class:`ConfigurationWatcher` and load the initial configuration by calling `loader_func`. :param filename: a filename or list of filenames to monitor for changes :param namespace: the name of a namespace to use when loading configuration. All config data from `filename` will end up in a :class:`ConfigNamespace` with this name :param loader_func: a function which accepts two arguments and uses loader functions from :mod:`staticconf.loader` to load configuration data into a namespace. The arguments are `filename` and `namespace` :param min_interval: minimum number of seconds to wait between calls to :func:`os.path.getmtime` to check if a file has been modified. :param comparators: a list of classes which support the :class:`IComparator` interface which are used to determine if a config file has been modified. See ConfigurationWatcher::__init__. :returns: a :class:`ConfigFacade` """ watcher = ConfigurationWatcher( build_loader_callable(loader_func, filename, namespace=namespace), filename, min_interval=min_interval, reloader=ReloadCallbackChain(namespace=namespace), comparators=comparators, ) watcher.load_config() return cls(watcher) def add_callback(self, identifier, callback): self.callback_chain.add(identifier, callback) def reload_if_changed(self, force=False): """See :func:`ConfigurationWatcher.reload_if_changed` """ self.watcher.reload_if_changed(force=force) PyStaticConfiguration-0.10.5/staticconf/errors.py000066400000000000000000000004511365564735600221340ustar00rootroot00000000000000 class ValidationError(Exception): """Thrown when a configuration value can not be coerced into the expected type by a validator. """ pass class ConfigurationError(Exception): """Thrown when there is an error loading configuration from a file or object. """ pass PyStaticConfiguration-0.10.5/staticconf/getters.py000066400000000000000000000076311365564735600223040ustar00rootroot00000000000000""" Functions used to retrieve proxies around values in a :class:`staticconf.config.ConfigNamespace`. All of the getter methods return a :class:`ValueProxy`. These proxies are wrappers around a configuration value. They don't access the configuration until some attribute of the object is accessed. .. warning:: This module should be considered deprecated. There are edge cases which make these getters non-obvious to use (such as passing a :class:`ValueProxy` to a cmodule. Please use :class:`staticconf.readers` if you don't need static definitions, or :class:`staticconf.schema` if you do. Example ------- .. code-block:: python import staticconf # Returns a ValueProxy which can be used just like an int max_cycles = staticconf.get_int('max_cycles') print "Half of max_cycles", max_cycles / 2 # Using a NamespaceGetters object to retrieve from a namespace config = staticconf.NamespaceGetters('special') ratio = config.get_float('ratio') To retrieve values from a namespace, you can create a ``NamespaceGetters`` object. .. code-block:: python my_package_conf = staticconf.NamespaceGetters('my_package_namespace') max_size = my_package_conf.get_int('max_size') error_msg = my_package_conf.get_string('error_msg') Arguments --------- Getters accept the following kwargs: config_key string configuration key default if no ``default`` is given, the key must be present in the configuration. Raises ConfigurationError on missing key. help a help string describing the purpose of the config value. See :func:`staticconf.config.view_help()`. namespace get the value from this namespace instead of DEFAULT. """ from staticconf import config, proxy, readers from staticconf.proxy import UndefToken def register_value_proxy(namespace, value_proxy, help_text): """Register a value proxy with the namespace, and add the help_text.""" namespace.register_proxy(value_proxy) config.config_help.add( value_proxy.config_key, value_proxy.validator, value_proxy.default, namespace.get_name(), help_text) class ProxyFactory(object): """Create ProxyValue objects so that there is never a duplicate proxy for any (namespace, validator, config_key, default) group. """ def __init__(self): self.proxies = {} def build(self, validator, namespace, config_key, default, help): """Build or retrieve a ValueProxy from the attributes. Proxies are keyed using a repr because default values can be mutable types. """ proxy_attrs = validator, namespace, config_key, default proxy_key = repr(proxy_attrs) if proxy_key in self.proxies: return self.proxies[proxy_key] value_proxy = proxy.ValueProxy(*proxy_attrs) register_value_proxy(namespace, value_proxy, help) return self.proxies.setdefault(proxy_key, value_proxy) proxy_factory = ProxyFactory() def build_getter(validator, getter_namespace=None): """Create a getter function for retrieving values from the config cache. Getters will default to the DEFAULT namespace. """ def proxy_register(key_name, default=UndefToken, help=None, namespace=None): name = namespace or getter_namespace or config.DEFAULT namespace = config.get_namespace(name) return proxy_factory.build(validator, namespace, key_name, default, help) return proxy_register class GetterNameFactory(object): @staticmethod def get_name(validator_name): return 'get_%s' % validator_name if validator_name else 'get' @staticmethod def get_list_of_name(validator_name): return 'get_list_of_%s' % validator_name NamespaceGetters = readers.build_accessor_type(GetterNameFactory, build_getter) default_getters = NamespaceGetters(config.DEFAULT) globals().update(default_getters.get_methods()) __all__ = ['NamespaceGetters'] + list(default_getters.get_methods()) PyStaticConfiguration-0.10.5/staticconf/loader.py000066400000000000000000000213021365564735600220640ustar00rootroot00000000000000""" Load configuration values from different file formats and python structures. Nested dictionaries are flattened using dotted notation. These flattened keys and values are merged into a :class:`staticconf.config.ConfigNamespace`. Examples -------- Basic example: .. code-block:: python staticconf.YamlConfiguration('config.yaml') Multiple loaders can be used to override values from previous loaders. .. code-block:: python import staticconf # Start by loading some values from a defaults file staticconf.YamlConfiguration('defaults.yaml') # Override with some user specified options staticconf.YamlConfiguration('user.yaml', optional=True) # Further override with some command line options staticconf.ListConfiguration(opts.config_values) For configuration reloading see :class:`staticconf.config.ConfigFacade`. Arguments --------- Configuration loaders accept the following kwargs: error_on_unknown raises a :class:`staticconf.errors.ConfigurationError` if there are keys in the config that have not been defined by a getter or a schema. optional if True only warns on failure to load configuration (Default False) namespace load the configuration values into a namespace. Defaults to the `DEFAULT` namespace. flatten flatten nested structures into a mapping with depth of 1 (Default True) .. versionadded:: 0.7.0 `flatten` was added as a kwarg to all loaders Custom Loader ------------- You can create your own loaders for other formats by using :func:`build_loader()`. It takes a single argument, a function, which can accept any arguments, but must return a dictionary of configuration values. .. code-block:: python from staticconf import loader def load_from_db(table_name, conn): ... return dict((row.field, row.value) for row in cursor.fetchall()) DBConfiguration = loader.build_loader(load_from_db) # Now lets use it DBConfiguration('config_table', conn, namespace='special') """ import logging import os import re import six from six.moves import ( configparser, filter, reload_module, ) from staticconf import config, errors __all__ = [ 'YamlConfiguration', 'JSONConfiguration', 'ListConfiguration', 'DictConfiguration', 'AutoConfiguration', 'PythonConfiguration', 'INIConfiguration', 'XMLConfiguration', 'PropertiesConfiguration', 'CompositeConfiguration', 'ObjectConfiguration', ] log = logging.getLogger(__name__) def flatten_dict(config_data): for key, value in six.iteritems(config_data): if hasattr(value, 'items') or hasattr(value, 'iteritems'): for k, v in flatten_dict(value): yield '%s.%s' % (key, k), v continue yield key, value def load_config_data(loader_func, *args, **kwargs): optional = kwargs.pop('optional', False) try: return loader_func(*args, **kwargs) except Exception as e: log.info("Optional configuration failed: %s" % e) if not optional: raise return {} def build_loader(loader_func): def loader(*args, **kwargs): err_on_unknown = kwargs.pop('error_on_unknown', False) log_keys_only = kwargs.pop('log_keys_only', True) err_on_dupe = kwargs.pop('error_on_duplicate', False) flatten = kwargs.pop('flatten', True) name = kwargs.pop('namespace', config.DEFAULT) config_data = load_config_data(loader_func, *args, **kwargs) if flatten: config_data = dict(flatten_dict(config_data)) namespace = config.get_namespace(name) namespace.apply_config_data( config_data, err_on_unknown, err_on_dupe, log_keys_only=log_keys_only, ) return config_data return loader def yaml_loader(filename): import yaml try: from yaml import CSafeLoader as SafeLoader except ImportError: from yaml import SafeLoader with open(filename) as fh: return yaml.load(fh, Loader=SafeLoader) or {} def json_loader(filename): try: import simplejson as json assert json except ImportError: import json with open(filename) as fh: return json.load(fh) def list_loader(seq): return dict(pair.split('=', 1) for pair in seq) def auto_loader(base_dir='.', auto_configurations=None): auto_configurations = auto_configurations or [ (yaml_loader, 'config.yaml'), (json_loader, 'config.json'), (ini_file_loader, 'config.ini'), (xml_loader, 'config.xml'), (properties_loader, 'config.properties') ] for config_loader, config_arg in auto_configurations: path = os.path.join(base_dir, config_arg) if os.path.isfile(path): return config_loader(path) msg = "Failed to auto-load configuration. No configuration files found." raise errors.ConfigurationError(msg) def python_loader(module_name): module = __import__(module_name, fromlist=['*']) reload_module(module) return object_loader(module) def object_loader(obj): return dict((name, getattr(obj, name)) for name in dir(obj) if not name.startswith('_')) def ini_file_loader(filename): parser = configparser.SafeConfigParser() parser.read([filename]) config_dict = {} for section in parser.sections(): for key, value in parser.items(section, True): config_dict['%s.%s' % (section, key)] = value return config_dict def xml_loader(filename, safe=False): from xml.etree import ElementTree def build_from_element(element): items = dict(element.items()) child_items = dict( (child.tag, build_from_element(child)) for child in element) config.has_duplicate_keys(child_items, items, safe) items.update(child_items) if element.text: if 'value' in items and safe: msg = "%s has tag with child or attribute named value." raise errors.ConfigurationError(msg % filename) items['value'] = element.text return items tree = ElementTree.parse(filename) return build_from_element(tree.getroot()) def properties_loader(filename): split_pattern = re.compile(r'[=:]') def parse_line(line): line = line.strip() if not line or line.startswith('#'): return try: key, value = split_pattern.split(line, 1) except ValueError: msg = "Invalid properties line: %s" % line raise errors.ConfigurationError(msg) return key.strip(), value.strip() with open(filename) as fh: return dict(filter(None, (parse_line(line) for line in fh))) class CompositeConfiguration(object): """Store a list of configuration loaders and their params, so they can be reloaded in the correct order. """ def __init__(self, loaders=None): self.loaders = loaders or [] def append(self, loader): self.loaders.append(loader) def load(self): output = {} for loader_with_args in self.loaders: output.update(loader_with_args[0](*loader_with_args[1:])) return output def __call__(self, *args): return self.load() YamlConfiguration = build_loader(yaml_loader) """Load configuration from a yaml file. :param filename: path to a yaml file """ JSONConfiguration = build_loader(json_loader) """Load configuration from a json file. :param filename: path to a json file """ ListConfiguration = build_loader(list_loader) """Load configuration from a list of strings in the form `key=value`. :param seq: a sequence of strings """ DictConfiguration = build_loader(lambda d: d) """Load configuration from a :class:`dict`. :param dict: a dictionary """ ObjectConfiguration = build_loader(object_loader) """Load configuration from any object. Attributes are keys and the attribute value are values. :param object: an object """ AutoConfiguration = build_loader(auto_loader) """ .. deprecated:: v0.7.0 Do not use. It will be removed in future versions. """ PythonConfiguration = build_loader(python_loader) """Load configuration from a python module. :param module: python path to a module as you would pass it to :func:`__import__` """ INIConfiguration = build_loader(ini_file_loader) """Load configuration from a .ini file :param filename: path to the ini file """ XMLConfiguration = build_loader(xml_loader) """Load configuration from an XML file. :param filename: path to the XML file """ PropertiesConfiguration = build_loader(properties_loader) """Load configuration from a properties file :param filename: path to the properties file """ PyStaticConfiguration-0.10.5/staticconf/proxy.py000066400000000000000000000110751365564735600220050ustar00rootroot00000000000000""" Proxy a configuration value. Defers the lookup until the value is used, so that values can be read statically at import time. """ import functools import operator from staticconf import errors import six class UndefToken(object): """A token to represent an undefined value, so that None can be used as a default value. """ def __repr__(self): return "" UndefToken = UndefToken() _special_names = [ '__abs__', '__add__', '__and__', '__bool__', '__call__', '__cmp__', '__coerce__', '__contains__', '__delitem__', '__delslice__', '__div__', '__divmod__', '__eq__', '__float__', '__floordiv__', '__ge__', '__getitem__', '__getslice__', '__gt__', '__hash__', '__hex__', '__iadd__', '__iand__', '__idiv__', '__idivmod__', '__ifloordiv__', '__ilshift__', '__imod__', '__imul__', '__int__', '__invert__', '__ior__', '__ipow__', '__irshift__', '__isub__', '__iter__', '__itruediv__', '__ixor__', '__le__', '__len__', '__long__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__oct__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdiv__', '__rdivmod__', '__repr__', '__reversed__', '__rfloorfiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setitem__', '__setslice__', '__sub__', '__truediv__', '__xor__', 'next', '__nonzero__', '__str__', '__unicode__', '__index__', '__fspath__', ] def identity(x): return x unary_funcs = { '__unicode__': six.text_type, '__str__': str, '__fspath__': identity, # python3.6+ os.PathLike interface '__repr__': repr, '__nonzero__': bool, # Python2 bool '__bool__': bool, # Python3 bool '__hash__': hash, } def build_class_def(cls): def build_method(name): def method(self, *args, **kwargs): if name in unary_funcs: return unary_funcs[name](self.value) if hasattr(operator, name): return getattr(operator, name)(self.value, *args) return getattr(self.value, name)(*args, **kwargs) return method namespace = dict((name, build_method(name)) for name in _special_names) return type(cls.__name__, (cls,), namespace) def cache_as_field(cache_name): """Cache a functions return value as the field 'cache_name'.""" def cache_wrapper(func): @functools.wraps(func) def inner_wrapper(self, *args, **kwargs): value = getattr(self, cache_name, UndefToken) if value != UndefToken: return value ret = func(self, *args, **kwargs) setattr(self, cache_name, ret) return ret return inner_wrapper return cache_wrapper def extract_value(proxy): """Given a value proxy type, Retrieve a value from a namespace, raising exception if no value is found, or the value does not validate. """ value = proxy.namespace.get(proxy.config_key, proxy.default) if value is UndefToken: raise errors.ConfigurationError("%s is missing value for: %s" % (proxy.namespace, proxy.config_key)) try: return proxy.validator(value) except errors.ValidationError as e: raise errors.ConfigurationError("%s failed to validate %s: %s" % (proxy.namespace, proxy.config_key, e)) class ValueProxy(object): """Proxy a configuration value so it can be loaded after import time.""" __slots__ = [ 'validator', 'config_key', 'default', '_value', 'namespace', '__weakref__' ] @classmethod @cache_as_field('_class_def') def get_class_def(cls): return build_class_def(cls) def __new__(cls, *args, **kwargs): """Create instances of this class with proxied special names.""" klass = cls.get_class_def() instance = object.__new__(klass) klass.__init__(instance, *args, **kwargs) return instance def __init__(self, validator, namespace, key, default=UndefToken): self.validator = validator self.config_key = key self.default = default self.namespace = namespace self._value = UndefToken @cache_as_field('_value') def get_value(self): return extract_value(self) value = property(get_value) def __getattr__(self, item): return getattr(self.value, item) def reset(self): """Clear the cached value so that configuration can be reloaded.""" self._value = UndefToken PyStaticConfiguration-0.10.5/staticconf/readers.py000066400000000000000000000114651365564735600222540ustar00rootroot00000000000000""" Functions to read values directly from a :class:`staticconf.config.ConfigNamespace`. Values will be validated and cast to the requested type. Examples -------- .. code-block:: python import staticconf # read an int max_cycles = staticconf.read_int('max_cycles') start_id = staticconf.read_int('poller.init.start_id', default=0) # start_date will be a datetime.date start_date = staticconf.read_date('start_date') # matcher will be a regex object matcher = staticconf.read_regex('matcher_pattern') # Read a value from a different namespace intervals = staticconf.read_float('intervals', namespace='something') Readers can be attached to a namespace using a :class:`NamespaceReaders` object. .. code-block:: python import staticconf bling_reader = staticconf.NamespaceReaders('bling') # These values are read from the `bling` ConfigNamespace currency = bling_reader.read_string('currency') value = bling_reader.read_float('value') Arguments --------- Readers accept the following kwargs: config_key string configuration key using dotted notation default if no `default` is given, the key must be present in the configuration. If the key is missing a :class:`staticconf.errors.ConfigurationError` is raised. namespace get the value from this namespace instead of DEFAULT. Building custom readers ----------------------- :func:`build_reader` is a factory function which can be used for creating custom readers from a validation function. A validation function should handle all exceptions and raise a :class:`staticconf.errors.ValidationError` if there is a problem. First create a validation function .. code-block:: python def validate_currency(value): try: # Assume a tuple or a list name, decimal_points = value return Currency(name, decimal_points) except Exception, e: raise ValidationErrror(...) Example of a custom reader: .. code-block:: python from staticconf import readers read_currency = readers.build_reader(validate_currency) # Returns a Currency object using the data from the config namespace # at they key `currencies.usd`. usd_currency = read_currency('currencies.usd') """ from staticconf import validation, config, errors from staticconf.proxy import UndefToken def _read_config(config_key, config_namespace, default): value = config_namespace.get(config_key, default=default) if value is UndefToken: msg = '%s missing value for %s' % (config_namespace, config_key) raise errors.ConfigurationError(msg) return value def build_reader(validator, reader_namespace=config.DEFAULT): """A factory method for creating a custom config reader from a validation function. :param validator: a validation function which acceptance one argument (the configuration value), and returns that value casted to the appropriate type. :param reader_namespace: the default namespace to use. Defaults to `DEFAULT`. """ def reader(config_key, default=UndefToken, namespace=None): config_namespace = config.get_namespace(namespace or reader_namespace) return validator(_read_config(config_key, config_namespace, default)) return reader class ReaderNameFactory(object): @staticmethod def get_name(name): return 'read_%s' % name if name else 'read' @staticmethod def get_list_of_name(name): return 'read_list_of_%s' % name def get_all_accessors(name_factory): for name, validator in validation.get_validators(): yield name_factory.get_name(name), validator yield (name_factory.get_list_of_name(name), validation.build_list_type_validator(validator)) class NamespaceAccessor(object): def __init__(self, name, accessor_map, builder): self.accessor_map = accessor_map self.builder = builder self.namespace = name def __getattr__(self, item): if item not in self.accessor_map: raise AttributeError(item) return self.builder(self.accessor_map[item], self.namespace) def get_methods(self): return dict((name, getattr(self, name)) for name in self.accessor_map) def build_accessor_type(name_factory, builder): accessor_map = dict(get_all_accessors(name_factory)) return lambda name: NamespaceAccessor(name, accessor_map, builder) NamespaceReaders = build_accessor_type(ReaderNameFactory, build_reader) """An object with all reader functions which retrieve configuration from a named namespace, instead of `DEFAULT`. """ default_readers = NamespaceReaders(config.DEFAULT) globals().update(default_readers.get_methods()) __all__ = ['NamespaceReaders'] + list(default_readers.get_methods()) PyStaticConfiguration-0.10.5/staticconf/schema.py000066400000000000000000000145431365564735600220670ustar00rootroot00000000000000""" Configuration schemas can be used to group configuration values together. These schemas can be instantiated at import time, and values can be retrieved from them by accessing the attributes of the schema object. Each field on the schema turns into an accessor for a configuration value. These accessors will cache the return value of the validator that they use, so expensive operations are not repeated. Example ------- .. code-block:: python from staticconf import schema class MyClassSchema(object): __metaclass__ = schema.SchemaMeta # Namespace to retrieve configuration values from namespace = 'my_package' # (optional) Config path to prepend to all config keys in this schema config_path = 'my_class.foo' # Attributes accept the same values as a getter (default, help, etc) ratio = schema.float(default=0.2) # configured at my_class.foo.ratio # You can optionally specify a different name from the attribute name max_threshold = schema.int(config_key='max') # configued at my_class.foo.max You can also create your schema objects by subclassing Schema .. code-block:: python from staticconf import schema class MyClassSchema(schema.Schema): ... Access the values from a schema by instantiating the schema class. .. code-block:: python config = MyClassSchema() print config.ratio Arguments --------- Schema accessors accept the following kwargs: config_key string configuration key default if no ``default`` is given, the key must be present in the configuration. Raises :class:`staticconf.errors.ConfigurationError` on missing key. help a help string describing the purpose of the config value. See :func:`staticconf.config.view_help`. Custom schema types ------------------- You can also create your own custom types using :func:`build_value_type`. .. code-block:: python from staticconf import schema def validator(value): try: return do_some_casting(value) except Exception: raise ConfigurationError("%s can't be validated as a foo" % value) foo_type = schema.build_value_type(validator) class MySchema(object): __metaclass__ = schema.SchemaMeta something = foo_type(default=...) """ import functools import six from staticconf import validation, proxy, config, errors, getters class ValueTypeDefinition(object): __slots__ = ['validator', 'config_key', 'default', 'help'] def __init__( self, validator, config_key=None, default=proxy.UndefToken, help=None): self.validator = validator self.config_key = config_key self.default = default self.help = help class ValueToken(object): __slots__ = [ 'validator', 'config_key', 'default', '_value', 'namespace', '__weakref__' ] def __init__(self, validator, namespace, key, default): self.validator = validator self.namespace = namespace self.config_key = key self.default = default self._value = proxy.UndefToken @classmethod def from_definition(cls, value_def, namespace, key): return cls(value_def.validator, namespace, key, value_def.default) @proxy.cache_as_field('_value') def get_value(self): return proxy.extract_value(self) def reset(self): """Clear the cached value so that configuration can be reloaded.""" self._value = proxy.UndefToken def build_property(value_token): """Construct a property from a ValueToken. The callable gets passed an instance of the schema class, which is ignored. """ def caller(_): return value_token.get_value() return property(caller) class SchemaMeta(type): """Metaclass to construct config schema object.""" def __new__(mcs, name, bases, attributes): namespace = mcs.get_namespace(attributes) attributes = mcs.build_attributes(attributes, namespace) return super(SchemaMeta, mcs).__new__(mcs, name, bases, attributes) @classmethod def get_namespace(cls, attributes): if 'namespace' not in attributes: raise errors.ConfigurationError("ConfigSchema requires a namespace.") return config.get_namespace(attributes['namespace']) @classmethod def build_attributes(cls, attributes, namespace): """Return an attributes dictionary with ValueTokens replaced by a property which returns the config value. """ config_path = attributes.get('config_path') tokens = {} def build_config_key(value_def, config_key): key = value_def.config_key or config_key return '%s.%s' % (config_path, key) if config_path else key def build_token(name, value_def): config_key = build_config_key(value_def, name) value_token = ValueToken.from_definition( value_def, namespace, config_key) getters.register_value_proxy(namespace, value_token, value_def.help) tokens[name] = value_token return name, build_property(value_token) def build_attr(name, attribute): if not isinstance(attribute, ValueTypeDefinition): return name, attribute return build_token(name, attribute) attributes = dict(build_attr(*item) for item in six.iteritems(attributes)) attributes['_tokens'] = tokens return attributes @six.add_metaclass(SchemaMeta) class Schema(object): """Base class for configuration schemas, uses :class:`SchemaMeta`.""" namespace = None def build_value_type(validator): """A factory function to create a new schema type. :param validator: a function which accepts one argument and returns that value as the correct type. """ return functools.partial(ValueTypeDefinition, validator) # Backwards compatible with staticconf 0.5.2 create_value_type = build_value_type for name, validator in validation.get_validators(): name = name or 'any' globals()[name] = build_value_type(validator) list_of_validator = validation.build_list_type_validator(validator) globals()['list_of_%s' % name] = build_value_type(list_of_validator) PyStaticConfiguration-0.10.5/staticconf/testing.py000066400000000000000000000057161365564735600223060ustar00rootroot00000000000000""" Facilitate testing of code which uses staticconf. """ import copy from staticconf import config, loader class MockConfiguration(object): """A context manager which replaces the configuration namespace while inside the context. When the context exits the old configuration values will be restored to that namespace. .. code-block:: python import staticconf.testing config = { ... } with staticconf.testing.MockConfiguration(config, namespace='special'): # Run your tests. ... :param namespace: the namespace to patch :param flatten: if True the configuration will be flattened (default True) :param args: passed directly to the constructor of :class:`dict` and used as configuration data :param kwargs: passed directly to the constructor of :class:`dict` and used as configuration data """ def __init__(self, *args, **kwargs): name = kwargs.pop('namespace', config.DEFAULT) flatten = kwargs.pop('flatten', True) config_data = dict(*args, **kwargs) self.namespace = config.get_namespace(name) self.config_data = (dict(loader.flatten_dict(config_data)) if flatten else config_data) self.old_values = None def setup(self): self.old_values = dict(self.namespace.get_config_values()) self.reset_namespace(self.config_data) config.reload(name=self.namespace.name) def teardown(self): self.reset_namespace(self.old_values) config.reload(name=self.namespace.name) def reset_namespace(self, new_values): self.namespace.configuration_values.clear() self.namespace.update_values(new_values) def __enter__(self): return self.setup() def __exit__(self, *args): self.teardown() class PatchConfiguration(MockConfiguration): """A context manager which updates the configuration namespace while inside the context. When the context exits the old configuration values will be restored to that namespace. Unlike MockConfiguration which completely replaces the configuration with the new one, this class instead only updates the keys in the configuration which are passed to it. It preserves all previous values that weren't updated. .. code-block:: python import staticconf.testing config = { ... } with staticconf.testing.PatchConfiguration(config, namespace='special'): # Run your tests. ... The arguments are identical to MockConfiguration. """ def setup(self): self.old_values = copy.deepcopy(dict(self.namespace.get_config_values())) new_configuration = copy.deepcopy(self.old_values) new_configuration.update(self.config_data) self.reset_namespace(new_configuration) config.reload(name=self.namespace.name) PyStaticConfiguration-0.10.5/staticconf/validation.py000066400000000000000000000101421365564735600227500ustar00rootroot00000000000000""" Validate a configuration value by converting it to a specific type. These functions are used by :mod:`staticconf.readers` and :mod:`staticconf.schema` to coerce config values to a type. """ import datetime import logging import re import time import six from staticconf.errors import ValidationError def validate_string(value): return None if value is None else six.text_type(value) def validate_bool(value): return None if value is None else bool(value) def validate_numeric(type_func, value): try: return type_func(value) except ValueError: raise ValidationError("Invalid %s: %s" % (type_func.__name__, value)) def validate_int(value): return validate_numeric(int, value) def validate_float(value): return validate_numeric(float, value) date_formats = [ "%Y-%m-%d %H:%M:%S", "%Y-%m-%d %I:%M:%S %p", "%Y-%m-%d", "%y-%m-%d", "%m/%d/%y", "%m/%d/%Y", ] def validate_datetime(value): if isinstance(value, datetime.datetime): return value for format_ in date_formats: try: return datetime.datetime.strptime(value, format_) except ValueError: pass raise ValidationError("Invalid date format: %s" % value) def validate_date(value): if isinstance(value, datetime.date): return value return validate_datetime(value).date() time_formats = [ "%I %p", "%H:%M", "%I:%M %p", "%H:%M:%S", "%I:%M:%S %p" ] def validate_time(value): if isinstance(value, datetime.time): return value for format_ in time_formats: try: return datetime.time(*time.strptime(value, format_)[3:6]) except ValueError: pass raise ValidationError("Invalid time format: %s" % value) def _validate_iterable(iterable_type, value): """Convert the iterable to iterable_type, or raise a Configuration exception. """ if isinstance(value, six.string_types): msg = "Invalid iterable of type(%s): %s" raise ValidationError(msg % (type(value), value)) try: return iterable_type(value) except TypeError: raise ValidationError("Invalid iterable: %s" % (value)) def validate_list(value): return _validate_iterable(list, value) def validate_set(value): return _validate_iterable(set, value) def validate_tuple(value): return _validate_iterable(tuple, value) def validate_regex(value): try: return re.compile(value) except (re.error, TypeError) as e: raise ValidationError("Invalid regex: %s, %s" % (e, value)) def build_list_type_validator(item_validator): """Return a function which validates that the value is a list of items which are validated using item_validator. """ def validate_list_of_type(value): return [item_validator(item) for item in validate_list(value)] return validate_list_of_type def build_map_type_validator(item_validator): """Return a function which validates that the value is a mapping of items. The function should return pairs of items that will be passed to the `dict` constructor. """ def validate_mapping(value): return dict(item_validator(item) for item in validate_list(value)) return validate_mapping def validate_log_level(value): """Validate a log level from a string value. Returns a constant from the :mod:`logging` module. """ try: return getattr(logging, value) except AttributeError: raise ValidationError("Unknown log level: %s" % value) def validate_any(value): return value validators = { '': validate_any, 'bool': validate_bool, 'date': validate_date, 'datetime': validate_datetime, 'float': validate_float, 'int': validate_int, 'list': validate_list, 'set': validate_set, 'string': validate_string, 'time': validate_time, 'tuple': validate_tuple, 'regex': validate_regex, 'log_level': validate_log_level, } def get_validators(): """Return an iterator of (validator_name, validator) pairs.""" return six.iteritems(validators) PyStaticConfiguration-0.10.5/staticconf/version.py000066400000000000000000000000241365564735600223010ustar00rootroot00000000000000 version = "0.10.5" PyStaticConfiguration-0.10.5/testing/000077500000000000000000000000001365564735600175665ustar00rootroot00000000000000PyStaticConfiguration-0.10.5/testing/__init__.py000066400000000000000000000000001365564735600216650ustar00rootroot00000000000000PyStaticConfiguration-0.10.5/testing/testifycompat.py000066400000000000000000000015371365564735600230410ustar00rootroot00000000000000"""Compatiblity functions for py.test to migrate code from testify. """ try: from unittest import mock # noqa except ImportError: import mock # noqa import pytest def assert_equal(left, right): assert left == right def assert_raises_and_contains(exc, text, func, *args, **kwargs): with pytest.raises(exc) as excinfo: func(*args, **kwargs) assert exc == excinfo.type text = text if isinstance(text, list) else [text] for item in text: assert item in str(excinfo.exconly()) def assert_raises(exc, func, *args, **kwargs): with pytest.raises(exc) as excinfo: func(*args, **kwargs) assert exc == excinfo.type def assert_in(item, container): assert item in container def assert_not_in(item, container): assert item not in container def assert_is(left, right): assert left is right PyStaticConfiguration-0.10.5/tests/000077500000000000000000000000001365564735600172535ustar00rootroot00000000000000PyStaticConfiguration-0.10.5/tests/__init__.py000066400000000000000000000000001365564735600213520ustar00rootroot00000000000000PyStaticConfiguration-0.10.5/tests/config_test.py000066400000000000000000000605751365564735600221460ustar00rootroot00000000000000import gc import os import platform import tempfile import time import sys import functools import pytest from testing.testifycompat import ( assert_equal, assert_raises, mock, ) from staticconf import ( config, errors, proxy, schema, testing, validation, ) import staticconf class TestRemoveByKeys(object): def test_empty_dict(self): keys = range(3) assert_equal([], config.remove_by_keys({}, keys)) def test_no_keys(self): keys = [] testMap = dict(enumerate(range(3))) assert_equal(list(testMap.items()), config.remove_by_keys(testMap, keys)) def test_overlap(self): keys = [1, 3, 5, 7] testMap = dict(enumerate(range(8))) expected = [(0, 0), (2, 2), (4, 4), (6, 6)] assert_equal(expected, config.remove_by_keys(testMap, keys)) class TestConfigMap(object): @pytest.fixture(autouse=True) def setup_config_map(self): self.config_map = config.ConfigMap(one=1, three=3, seven=7) def test_no_iteritems(self): assert not hasattr(self.config_map, 'iteritems') def test_getitem(self): assert_equal(self.config_map['one'], 1) assert_equal(self.config_map['seven'], 7) def test_get(self): assert_equal(self.config_map.get('three'), 3) assert_equal(self.config_map.get('four', 0), 0) def test_contains(self): assert 'one' in self.config_map assert 'two' not in self.config_map def test_len(self): assert_equal(len(self.config_map), 3) class TestConfigurationNamespace(object): @pytest.fixture(autouse=True) def setup_namespace(self): self.name = 'the_name' self.namespace = config.ConfigNamespace(self.name) self.config_data = dict(enumerate(['one', 'two', 'three'], 1)) def test_register_get_value_proxies(self): proxies = [mock.Mock(), mock.Mock()] for mock_proxy in proxies: self.namespace.register_proxy(mock_proxy) assert_equal(self.namespace.get_value_proxies(), proxies) @pytest.mark.skipif( 'PyPy' in platform.python_implementation(), reason="Fails on PyPy", ) def test_get_value_proxies_does_not_contain_out_of_scope_proxies(self): assert not self.namespace.get_value_proxies() def a_scope(): mock_proxy = mock.create_autospec(proxy.ValueProxy) self.namespace.register_proxy(mock_proxy) a_scope() a_scope() gc.collect() assert_equal(len(self.namespace.get_value_proxies()), 0) def test_update_values(self): values = dict(one=1, two=2) self.namespace.update_values(values) assert 'one' in self.namespace assert 'two' in self.namespace def test_get_config_values(self): self.namespace['stars'] = 'foo' values = self.namespace.get_config_values() assert_equal(values, {'stars': 'foo'}) def test_get_config_dict(self): self.namespace['one.two.three.four'] = 5 self.namespace['one.two.three.five'] = 'six' self.namespace['one.b.cats'] = [1, 2, 3] self.namespace['a.two'] = 'c' self.namespace['first'] = True d = self.namespace.get_config_dict() assert_equal(d, { 'one': { 'b': { 'cats': [1, 2, 3], }, 'two': { 'three': { 'four': 5, 'five': 'six', }, }, }, 'a': { 'two': 'c', }, 'first': True, }) def test_get_known_keys(self): proxies = [mock.Mock(), mock.Mock()] for mock_proxy in proxies: self.namespace.register_proxy(mock_proxy) expected = set([mock_proxy.config_key for mock_proxy in proxies]) assert_equal(self.namespace.get_known_keys(), expected) def test_validate_keys_no_unknown_keys(self): proxies = [mock.Mock(config_key=i) for i in self.config_data] for mock_proxy in proxies: self.namespace.register_proxy(mock_proxy) with mock.patch('staticconf.config.log') as mock_log: self.namespace.validate_keys(self.config_data, True) self.namespace.validate_keys(self.config_data, False) assert not mock_log.warn.mock_calls def test_validate_keys_unknown_log(self): with mock.patch('staticconf.config.log') as mock_log: self.namespace.validate_keys(self.config_data, False) assert_equal(len(mock_log.info.mock_calls), 1) def test_validate_keys_unknown_log_keys_only(self): with mock.patch('staticconf.config.log') as mock_log: self.namespace.validate_keys( self.config_data, False, log_keys_only=True, ) assert_equal(len(mock_log.info.mock_calls), 1) log_msg = mock_log.info.call_args[0][0] unknown = config.remove_by_keys( self.config_data, self.namespace.get_known_keys(), ) for k, v in unknown: # Have to cast to strings here, since log_msg is a string key_string, val_string = str(k), str(v) assert key_string in log_msg assert val_string not in log_msg def test_validate_keys_unknown_raise(self): assert_raises(errors.ConfigurationError, self.namespace.validate_keys, self.config_data, True) def test_clear(self): self.namespace.apply_config_data(self.config_data, False, False) assert self.namespace.get_config_values() self.namespace.clear() assert_equal(self.namespace.get_config_values(), {}) class TestGetNamespace(object): @pytest.yield_fixture(autouse=True) def mock_namespaces(self): with mock.patch.dict(config.configuration_namespaces): yield def test_get_namespace_new(self): name = 'some_unlikely_name' assert name not in config.configuration_namespaces config.get_namespace(name) assert name in config.configuration_namespaces def test_get_namespace_existing(self): name = 'the_common_name' namespace = config.get_namespace(name) assert_equal(namespace, config.get_namespace(name)) class TestReload(object): @pytest.yield_fixture(autouse=True) def mock_namespaces(self): with mock.patch.dict(config.configuration_namespaces): yield def test_reload_default(self): staticconf.DictConfiguration(dict(one='three', seven='nine')) one, seven = staticconf.get('one'), staticconf.get('seven') staticconf.DictConfiguration(dict(one='ten', seven='el')) staticconf.reload() assert_equal(one, 'ten') assert_equal(seven, 'el') def test_reload_all(self): name = 'another_one' staticconf.DictConfiguration(dict(one='three')) staticconf.DictConfiguration(dict(two='three'), namespace=name) one, two = staticconf.get('one'), staticconf.get('two', namespace=name) # access the values to set the value_proxy cache one.value, two.value staticconf.DictConfiguration(dict(one='four')) staticconf.DictConfiguration(dict(two='five'), namespace=name) staticconf.reload(all_names=True) assert_equal(one, 'four') assert_equal(two, 'five') def test_reload_single(self): name = 'another_one' staticconf.DictConfiguration(dict(one='three')) staticconf.DictConfiguration(dict(two='three'), namespace=name) one, two = staticconf.get('one'), staticconf.get('two', namespace=name) # access the values to set the value_proxy cache one.value, two.value staticconf.DictConfiguration(dict(one='four')) staticconf.DictConfiguration(dict(two='five'), namespace=name) staticconf.reload() assert_equal(one, 'four') assert_equal(two, 'three') class TestValidateConfig(object): @pytest.yield_fixture(autouse=True) def patch_config(self): with mock.patch.dict(config.configuration_namespaces, clear=True): with testing.MockConfiguration(): yield def test_validate_single_passes(self): staticconf.DictConfiguration({}) config.validate() _ = staticconf.get_string('one.two') # noqa: F841 staticconf.DictConfiguration({'one.two': 'nice'}) config.validate() def test_validate_single_fails(self): _ = staticconf.get_int('one.two') # noqa: F841 assert_raises(errors.ConfigurationError, config.validate) def test_validate_all_passes(self): name = 'yan' staticconf.DictConfiguration({}, namespace=name) staticconf.DictConfiguration({}) config.validate(all_names=True) staticconf.get_string('one.two') staticconf.get_string('foo', namespace=name) staticconf.DictConfiguration({'one.two': 'nice'}) staticconf.DictConfiguration({'foo': 'nice'}, namespace=name) config.validate(all_names=True) def test_validate_all_fails(self): name = 'yan' _ = staticconf.get_string('foo', namespace=name) # noqa: F841 assert_raises(errors.ConfigurationError, config.validate, all_names=True) def test_validate_value_token(self): class ExampleSchema(schema.Schema): namespace = 'DEFAULT' thing = schema.int() assert_raises(errors.ConfigurationError, config.validate, all_names=True) class TestConfigHelp(object): @pytest.fixture(autouse=True) def setup_config_help(self): self.config_help = config.ConfigHelp() self.config_help.add('one', validation.validate_any, None, 'DEFAULT', "the one") self.config_help.add('when', validation.validate_time, 'NOW', 'DEFAULT', "The time") self.config_help.add('you sure', validation.validate_bool, 'No', 'DEFAULT', "Are you?") self.config_help.add('one', validation.validate_any, None, 'Beta', "the one") self.config_help.add('one', validation.validate_any, None, 'Alpha', "the one") self.config_help.add('two', validation.validate_any, None, 'Alpha', "the two") self.lines = self.config_help.view_help().split('\n') def test_view_help_format(self): line, help = self.lines[4:6] assert_equal(help, 'The time') assert_equal(line, 'when (Type: time, Default: NOW)') def test_view_help_format_namespace(self): namespace, one, _, two, _, blank = self.lines[9:15] assert_equal(namespace, 'Namespace: Alpha') assert one.startswith('one') assert two.startswith('two') assert_equal(blank, '') def test_view_help_namespace_sort(self): lines = list(filter(lambda l: l.startswith('Namespace'), self.lines)) expected = ['Namespace: DEFAULT', 'Namespace: Alpha', 'Namespace: Beta'] assert_equal(lines, expected) class TestHasDuplicateKeys(object): @pytest.fixture(autouse=True) def setup_base_conf(self): self.base_conf = {'fear': 'is_the', 'mind': 'killer'} def test_has_dupliacte_keys_false(self): config_data = dict(unique_keys=123) assert not config.has_duplicate_keys(config_data, self.base_conf, True) assert not config.has_duplicate_keys(config_data, self.base_conf, False) def test_has_duplicate_keys_raises(self): config_data = dict(fear=123) assert_raises( errors.ConfigurationError, config.has_duplicate_keys, config_data, self.base_conf, True) def test_has_duplicate_keys_no_raise(self): config_data = dict(mind=123) assert config.has_duplicate_keys(config_data, self.base_conf, False) class TestConfigurationWatcher(object): @pytest.yield_fixture(autouse=True) def setup_mocks_and_config_watcher(self): self.loader = mock.Mock() with mock.patch('staticconf.config.time') as self.mock_time: with mock.patch('staticconf.config.os.stat') as self.mock_stat: with tempfile.NamedTemporaryFile() as file: with mock.patch( 'staticconf.config.os.path.getmtime', ) as self.mock_getmtime: file.flush() self.mtime = 234 self.mock_getmtime.return_value = self.mtime self.mock_stat.return_value.st_ino = 1 self.mock_stat.return_value.st_dev = 2 self.filename = file.name self.watcher = config.ConfigurationWatcher( self.loader, self.filename) yield def test_get_filename_list_from_string(self): with mock.patch('staticconf.config.os.path.abspath') as mock_path_abspath: mock_path_abspath.side_effect = lambda p: p filename = 'thefilename.yaml' filenames = self.watcher.get_filename_list(filename) assert_equal(filenames, [filename]) def test_get_filename_list_from_list(self): with mock.patch('staticconf.config.os.path.abspath') as mock_path_abspath: mock_path_abspath.side_effect = lambda p: p filenames = ['b', 'g', 'z', 'a'] expected = ['a', 'b', 'g', 'z'] assert_equal(self.watcher.get_filename_list(filenames), expected) def test_should_check(self): self.watcher.last_check = 123456789 self.mock_time.time.return_value = 123456789 # Still current, but no min_interval assert self.watcher.should_check # With max interval self.watcher.min_interval = 3 assert not self.watcher.should_check # Time has passed self.mock_time.time.return_value = 123456794 assert self.watcher.should_check def test_file_modified_not_modified(self): self.mock_time.time.return_value = 123460 assert not self.watcher.file_modified() assert_equal(self.watcher.last_check, self.mock_time.time.return_value) def test_file_modified(self): self.watcher.comparators[0].last_max_mtime = 123456 self.mock_getmtime.return_value = 123460 assert self.watcher.file_modified() assert_equal(self.watcher.last_check, self.mock_time.time.return_value) def test_reload_default(self): self.watcher.reload() self.loader.assert_called_with() def test_reload_custom(self): reloader = mock.Mock() watcher = config.ConfigurationWatcher( self.loader, self.filename, reloader=reloader) watcher.reload() reloader.assert_called_with() class TestInodeComparator(object): def test_get_inodes_empty(self): comparator = config.InodeComparator([]) assert comparator.get_inodes() == [] @mock.patch('staticconf.config.os.stat', autospec=True) def test_get_inodes(self, mock_stat): comparator = config.InodeComparator(['./one.file']) inodes = comparator.get_inodes() expected = [(mock_stat.return_value.st_dev, mock_stat.return_value.st_ino)] assert_equal(inodes, expected) class TestMTimeComparator(object): @mock.patch('staticconf.config.os.path.getmtime', autospec=True, return_value=1) def test_no_change(self, mock_mtime): comparator = config.MTimeComparator(['./one.file']) assert not comparator.has_changed() assert not comparator.has_changed() @mock.patch( 'staticconf.config.os.path.getmtime', autospec=True, side_effect=[0, 1, 1, 2], ) def test_changes(self, mock_mtime): comparator = config.MTimeComparator(['./one.file']) assert comparator.has_changed() assert not comparator.has_changed() assert comparator.has_changed() @mock.patch( 'staticconf.config.os.path.getmtime', autospec=True, side_effect=[1, 2, 1], ) def test_change_when_newer_time_before_older_time(self, mock_mtime): comparator = config.MTimeComparator(['./one.file']) # 1 -> 2 assert comparator.has_changed() # 2 -> 1 (can happen as a result of a revert) assert comparator.has_changed() class TestMTimeComparatorWithCompareFunc(object): @pytest.fixture(autouse=True) def setup_comparator(self): self._LoggingMTimeComparator = functools.partial( config.MTimeComparator, compare_func=config.build_compare_func(self._err_logger)) @pytest.fixture(autouse=True) def _reset_err_logger(self): self._err_filename = None self._exc_info = (None, None, None) def _err_logger(self, filename): self._err_filename = filename self._exc_info = sys.exc_info() def test_logs_error(self): self._LoggingMTimeComparator(['./not.a.file']) assert self._err_filename == "./not.a.file" assert all(x is not None for x in self._exc_info) def test_get_most_recent_empty(self): self._LoggingMTimeComparator([]) assert self._err_filename is None assert all(x is None for x in self._exc_info) @mock.patch('staticconf.config.os.path.getmtime', autospec=True, return_value=1) def test_no_change(self, mock_mtime): comparator = self._LoggingMTimeComparator(['./one.file']) assert not comparator.has_changed() assert not comparator.has_changed() assert self._err_filename is None assert all(x is None for x in self._exc_info) @mock.patch( 'staticconf.config.os.path.getmtime', autospec=True, side_effect=[0, 1, 1, 2], ) def test_changes(self, mock_mtime): comparator = self._LoggingMTimeComparator(['./one.file']) assert comparator.has_changed() assert not comparator.has_changed() assert comparator.has_changed() assert self._err_filename is None assert all(x is None for x in self._exc_info) class TestMD5Comparator(object): @pytest.yield_fixture() def comparator(self): self.original_contents = b"abcdefghijkabcd" with tempfile.NamedTemporaryFile() as self.file: self.write_contents(self.original_contents) yield config.MD5Comparator([self.file.name]) def write_contents(self, contents): self.file.seek(0) self.file.write(contents) self.file.flush() def test_get_hashes_empty(self): comparator = config.MD5Comparator([]) assert comparator.get_hashes() == [] def test_has_changed_no_changes(self, comparator): assert not comparator.has_changed() self.write_contents(self.original_contents) assert not comparator.has_changed() def test_has_changed_with_changes(self, comparator): assert not comparator.has_changed() self.write_contents(b"this is the new content") assert comparator.has_changed() class TestReloadCallbackChain(object): @pytest.fixture(autouse=True) def setup_callback_chain(self): self.callbacks = list(enumerate([mock.Mock(), mock.Mock()])) self.callback_chain = config.ReloadCallbackChain(callbacks=self.callbacks) def test_init_with_callbacks(self): assert_equal(self.callback_chain.callbacks, dict(self.callbacks)) def test_add_remove(self): callback = mock.Mock() self.callback_chain.add('one', callback) assert_equal(self.callback_chain.callbacks['one'], callback) self.callback_chain.remove('one') assert 'one' not in self.callback_chain.callbacks def test_call(self): self.callback_chain.namespace = 'the_namespace' with mock.patch('staticconf.config.reload') as mock_reload: self.callback_chain() for _, callback in self.callbacks: callback.assert_called_with() mock_reload.assert_called_with(name='the_namespace', all_names=False) class TestConfigFacade(object): @pytest.fixture(autouse=True) def setup_facade(self): self.mock_watcher = mock.create_autospec(config.ConfigurationWatcher) self.mock_watcher.get_reloader.return_value = mock.create_autospec( config.ReloadCallbackChain) self.facade = config.ConfigFacade(self.mock_watcher) def test_load(self): filename, namespace = "filename", "namespace" loader = mock.Mock() with mock.patch( 'staticconf.config.ConfigurationWatcher', autospec=True) as mock_watcher_class: facade = config.ConfigFacade.load(filename, namespace, loader) facade.watcher.load_config.assert_called_with() assert_equal(facade.watcher, mock_watcher_class.return_value) reloader = facade.callback_chain assert_equal(reloader, facade.watcher.get_reloader()) def test_load_passes_comparators_to_configuration_watcher(self): filename, namespace = "filename", "namespace" loader = mock.Mock() comparator = mock.Mock(name='MockComparator') with mock.patch( 'staticconf.config.ConfigurationWatcher', autospec=True ) as mock_watcher_class: config.ConfigFacade.load( filename, namespace, loader, comparators=[comparator], ) mock_watcher_class.assert_called_with( mock.ANY, filename, min_interval=mock.ANY, reloader=mock.ANY, comparators=[comparator], ) def test_add_callback(self): name, func = 'name', mock.Mock() self.facade.add_callback(name, func) self.facade.callback_chain.add.assert_called_with(name, func) def test_reload_if_changed(self): self.facade.reload_if_changed() self.mock_watcher.reload_if_changed.assert_called_with(force=False) @pytest.mark.acceptance class TestConfigFacadeAcceptance(object): @pytest.fixture(autouse=True) def setup_env(self): self.file = tempfile.NamedTemporaryFile() self.write(b"one: A") def write(self, content, mtime_seconds=0): time.sleep(0.03) self.file.file.seek(0) self.file.write(content) self.file.flush() tstamp = time.time() + mtime_seconds os.utime(self.file.name, (tstamp, tstamp)) @pytest.yield_fixture(autouse=True) def patch_namespace(self): self.namespace = 'testing_namespace' with testing.MockConfiguration(namespace=self.namespace): yield def test_load_end_to_end(self): loader = staticconf.YamlConfiguration callback = mock.Mock() facade = staticconf.ConfigFacade.load(self.file.name, self.namespace, loader) facade.add_callback('one', callback) assert_equal(staticconf.get('one', namespace=self.namespace), "A") self.write(b"one: B", 10) facade.reload_if_changed() assert_equal(staticconf.get('one', namespace=self.namespace), "B") callback.assert_called_with() def test_reload_end_to_end(self): loader = mock.Mock() facade = staticconf.ConfigFacade.load( self.file.name, self.namespace, loader) assert_equal(loader.call_count, 1) time.sleep(1) facade.reload_if_changed() assert_equal(loader.call_count, 1) os.utime(self.file.name, None) facade.reload_if_changed() assert_equal(loader.call_count, 2) class TestBuildLoaderCallable(object): @pytest.yield_fixture(autouse=True) def patch_namespace(self): self.namespace = 'the_namespace' patcher = mock.patch('staticconf.config.get_namespace', autospec=True) with patcher as self.mock_get_namespace: yield def test_build_loader_callable(self): load_func, filename = mock.Mock(), mock.Mock() loader_callable = config.build_loader_callable( load_func, filename, self.namespace) result = loader_callable() self.mock_get_namespace.assert_called_with(self.namespace) mock_namespace = self.mock_get_namespace.return_value mock_namespace.clear.assert_called_with() load_func.assert_called_with(filename, namespace=self.namespace) assert_equal(result, load_func.return_value) PyStaticConfiguration-0.10.5/tests/data/000077500000000000000000000000001365564735600201645ustar00rootroot00000000000000PyStaticConfiguration-0.10.5/tests/data/__init__.py000066400000000000000000000000001365564735600222630ustar00rootroot00000000000000PyStaticConfiguration-0.10.5/tests/getters_test.py000066400000000000000000000063401365564735600223440ustar00rootroot00000000000000import pytest from testing.testifycompat import ( assert_equal, assert_in, assert_is, mock, ) from staticconf import getters, config, testing class TestBuildGetter(object): @pytest.yield_fixture(autouse=True) def teardown_proxies(self): with testing.MockConfiguration(): yield def test_build_getter(self): validator = mock.Mock() getter = getters.build_getter(validator) assert callable(getter), "Getter is not callable" value_proxy = getter('the_name') namespace = config.get_namespace(config.DEFAULT) assert_in(id(value_proxy), namespace.value_proxies) assert_equal(value_proxy.config_key, "the_name") assert_equal(value_proxy.namespace, namespace) def test_build_getter_with_getter_namespace(self): validator = mock.Mock() name = 'the stars' getter = getters.build_getter(validator, getter_namespace=name) assert callable(getter), "Getter is not callable" value_proxy = getter('the_name') namespace = config.get_namespace(name) assert_in(id(value_proxy), namespace.value_proxies) assert_equal(value_proxy.config_key, "the_name") assert_equal(value_proxy.namespace, namespace) class TestNamespaceGetters(object): @pytest.yield_fixture(autouse=True) def teardown_proxies(self): self.namespace = 'the_test_namespace' with testing.MockConfiguration(namespace=self.namespace): yield def test_getters(self): get_conf = getters.NamespaceGetters(self.namespace) proxies = [ get_conf.get_bool('is_it'), get_conf.get_time('when'), get_conf.get_list_of_bool('options') ] namespace = config.get_namespace(get_conf.namespace) for proxy in proxies: assert_in(id(proxy), namespace.value_proxies) class TestProxyFactory(object): @pytest.yield_fixture(autouse=True) def patch_registries(self): patcher = mock.patch('staticconf.getters.register_value_proxy') with patcher as self.mock_register: yield @pytest.fixture(autouse=True) def setup_factory(self): self.factory = getters.ProxyFactory() self.validator = mock.Mock() self.namespace = mock.create_autospec(config.ConfigNamespace) self.config_key = 'some_key' self.default = 'bad_default' self.help = 'some help message no one reads' self.args = ( self.validator, self.namespace, self.config_key, self.default, self.help) def test_build_new(self): value_proxy = self.factory.build(*self.args) self.mock_register.assert_called_with( self.namespace, value_proxy, self.help) def test_build_existing(self): value_proxy = self.factory.build(*self.args) self.mock_register.reset_mock() assert_is(value_proxy, self.factory.build(*self.args)) assert not self.mock_register.mock_calls def test_build_with_immutable_default(self): args = self.validator, self.namespace, self.config_key, [], self.help self.factory.build(*args) assert_in(repr(args[:-1]), self.factory.proxies) PyStaticConfiguration-0.10.5/tests/integration_test.py000066400000000000000000000071031365564735600232100ustar00rootroot00000000000000import logging from testing.testifycompat import ( assert_equal, assert_raises, ) import staticconf from staticconf import testing, errors class SomeClass(object): namespace = 'UniqueNamespaceForEndToEndTesting' alt_name = 'UniqueNamespaceForEndToEndTestingAlternative' getters = staticconf.NamespaceGetters(namespace) max = getters.get_int('SomeClass.max') min = getters.get_int('SomeClass.min') ratio = getters.get_float('SomeClass.ratio') alt_ratio = getters.get_float('SomeClass.alt_ratio', 6.0) msg = getters.get_string('SomeClass.msg', None) real_max = staticconf.get_int('SomeClass.max', namespace=alt_name) real_min = staticconf.get_int('SomeClass.min', namespace=alt_name) class TestEndToEnd(object): config = { 'SomeClass': { 'max': 100, 'min': '0', 'ratio': '7.7' }, 'globals': False, 'enable': 'True', 'matcher': r'\d+', 'options': ['1', '7', '3', '9'], 'level': 'INFO', } def test_load_and_validate(self): staticconf.DictConfiguration(self.config, namespace=SomeClass.namespace) some_class = SomeClass() assert_equal(some_class.max, 100) assert_equal(some_class.min, 0) assert_equal(some_class.ratio, 7.7) assert_equal(some_class.alt_ratio, 6.0) assert_equal(SomeClass.getters.get('globals'), False) assert_equal(SomeClass.getters.get('enable'), 'True') assert_equal(SomeClass.getters.get_bool('enable'), True) assert_equal(some_class.msg, None) assert SomeClass.getters.get_regex('matcher').match('12345') assert not SomeClass.getters.get_regex('matcher').match('a') assert_equal(SomeClass.getters.get_list_of_int('options'), [1, 7, 3, 9]) assert_equal(SomeClass.getters.get_log_level('level'), logging.INFO) def test_load_and_validate_namespace(self): real_config = {'SomeClass.min': 20, 'SomeClass.max': 25} staticconf.DictConfiguration(self.config, namespace=SomeClass.namespace) staticconf.DictConfiguration(real_config, namespace=SomeClass.alt_name) some_class = SomeClass() assert_equal(some_class.max, 100) assert_equal(some_class.min, 0) assert_equal(some_class.real_min, 20) assert_equal(some_class.real_max, 25) def test_readers(self): staticconf.DictConfiguration(self.config) assert_equal(staticconf.read_float('SomeClass.ratio'), 7.7) assert_equal(staticconf.read_bool('globals'), False) assert_equal(staticconf.read_list_of_int('options'), [1, 7, 3, 9]) class TestMockConfiguration(object): namespace = 'UniqueNamespaceForMockConfigurationTesting' getters = staticconf.NamespaceGetters(namespace) def test_mock_configuration_context_manager(self): one = self.getters.get('one') three = self.getters.get_int('three', default=3) with testing.MockConfiguration(dict(one=7), namespace=self.namespace): assert_equal(one, 7) assert_equal(three, 3) assert_raises(errors.ConfigurationError, self.getters.get('one')) def test_mock_configuration(self): two = self.getters.get_string('two') stars = self.getters.get_bool('stars') mock_config = testing.MockConfiguration( dict(two=2, stars=False), namespace=self.namespace) mock_config.setup() assert_equal(two, '2') assert not stars mock_config.teardown() assert_raises(errors.ConfigurationError, self.getters.get('two')) PyStaticConfiguration-0.10.5/tests/loader_test.py000066400000000000000000000235661365564735600221460ustar00rootroot00000000000000import os import platform import tempfile import textwrap import pytest import six from six.moves import range from testing.testifycompat import ( assert_equal, assert_raises, assert_not_in, mock, ) from staticconf import loader, errors def get_bytecode_filename(module_name): if six.PY2: return module_name + '.pyc' return __import__(module_name).__cached__ class LoaderTestCase(object): content = None @pytest.yield_fixture(autouse=True) def mock_config(self): with mock.patch('staticconf.config') as self.mock_config: yield @pytest.fixture(autouse=True) def content_to_file(self): self.write_content_to_file() def write_content_to_file(self, content=None): content = content or self.content if not content: return self.tmpfile = tempfile.NamedTemporaryFile() self.tmpfile.write(content.encode('utf8')) self.tmpfile.flush() class TestListConfiguration(LoaderTestCase): def test_loader(self): overrides = ['something=1', 'max=two'] expected = dict(something='1', max='two') config_data = loader.ListConfiguration(overrides) assert_equal(config_data, expected) class TestFlattenDict(LoaderTestCase): source = { 'zero': 0, 'first': { 'star': 1, 'another': { 'depth': 2 } }, } expected = { 'zero': 0, 'first.star': 1, 'first.another.depth': 2 } def test_flatten(self): actual = dict(loader.flatten_dict(self.source)) assert_equal(actual, self.expected) class TestBuildLoader(LoaderTestCase): def test_build_loader(self): loader_func = mock.Mock() assert callable(loader.build_loader(loader_func)) def test_build_loader_optional(self): err_msg = "Failed to do" loader_func = mock.Mock() loader_func.side_effect = ValueError(err_msg) config_loader = loader.build_loader(loader_func) config_loader(optional=True) assert_raises(ValueError, config_loader) def test_build_loader_without_flatten(self): source = {'base': {'one': 'thing', 'two': 'foo'}} loader_func = mock.Mock(return_value=source) config_loader = loader.build_loader(loader_func) config = config_loader(source, flatten=False) assert_equal(config, source) class TestYamlConfiguration(LoaderTestCase): content = textwrap.dedent(""" somekey: token: "smarties" another: blind """) def test_loader(self): config_data = loader.YamlConfiguration(self.tmpfile.name) assert_equal(config_data['another'], 'blind') assert_equal(config_data['somekey.token'], 'smarties') class TestJSONConfiguration(LoaderTestCase): content = '{"somekey": {"token": "smarties"}, "another": "blind"}' def test_loader(self): config_data = loader.JSONConfiguration(self.tmpfile.name) assert_equal(config_data['another'], 'blind') assert_equal(config_data['somekey.token'], 'smarties') class TestAutoConfiguration(LoaderTestCase): @pytest.fixture(autouse=True) def setup_filename(self): self.filename = None @pytest.yield_fixture(autouse=True) def cleanup_file(self): yield if self.filename: os.unlink(self.filename) def test_auto_json(self): self.filename = os.path.join(tempfile.gettempdir(), 'config.json') with open(self.filename, 'w') as tmpfile: tmpfile.write('{"key": "1", "second.value": "two"}') tmpfile.flush() config_data = loader.AutoConfiguration(base_dir=tempfile.gettempdir()) assert_equal(config_data['key'], '1') def test_auto_yaml(self): self.filename = os.path.join(tempfile.gettempdir(), 'config.yaml') with open(self.filename, 'w') as tmpfile: tmpfile.write('key: 1') tmpfile.flush() config_data = loader.AutoConfiguration(base_dir=tempfile.gettempdir()) assert_equal(config_data['key'], 1) def test_auto_failed(self): assert_raises(errors.ConfigurationError, loader.AutoConfiguration) class TestPythonConfiguration(LoaderTestCase): module = 'example_mod' module_file = 'example_mod.py' module_content = textwrap.dedent(""" some_value = "test" more_values = { "depth": "%s" } """) @pytest.yield_fixture(autouse=True) def teardown_module(self): yield self.remove_module() def remove_module(self): compiled_file = get_bytecode_filename(self.module) for filename in [self.module_file, compiled_file]: os.remove(filename) if os.path.exists(filename) else None @pytest.fixture(autouse=True) def setup_module(self): self.create_module('one') def create_module(self, value): with open(self.module_file, 'w') as fh: fh.write(self.module_content % value) def test_python_configuration(self): config_data = loader.PythonConfiguration(self.module) assert_equal(config_data['some_value'], 'test') assert_equal(config_data['more_values.depth'], 'one') @pytest.mark.skipif(platform.python_implementation() == 'PyPy', reason="Unexpected results on pypy") def test_python_configuration_reload(self): config_data = loader.PythonConfiguration(self.module) assert_equal(config_data['more_values.depth'], 'one') self.remove_module() self.create_module('two') config_data = loader.PythonConfiguration(self.module) assert config_data['more_values.depth'] == 'two' class TestINIConfiguration(LoaderTestCase): content = textwrap.dedent(""" [Something] mars=planet stars=sun [Business] is_good=True always=False why=not today """) def test_prop_configuration(self): config_data = loader.INIConfiguration(self.tmpfile.name) assert_equal(config_data['Something.mars'], 'planet') assert_equal(config_data['Business.why'], 'not today') class TestXMLConfiguration(LoaderTestCase): content = """ 1 ok foo """ def test_xml_configuration(self): config_data = loader.XMLConfiguration(self.tmpfile.name) assert_equal(config_data['something.a'], 'here') assert_equal(config_data['something.stars.value'], 'ok') assert_equal(config_data['something.stars.b'], 'there') assert_equal(config_data['another.value'], 'foo') def test_xml_configuration_safe_load(self): config_data = loader.XMLConfiguration(self.tmpfile.name, safe=True) assert_equal(config_data['something.a'], 'here') assert_equal(config_data['empty.value'], 'E') def test_xml_configuration_safe_override(self): content = """ E """ self.write_content_to_file(content) assert_raises( errors.ConfigurationError, loader.XMLConfiguration, self.tmpfile.name, safe=True) def test_xml_configuration_safe_value_tag(self): content = """ E """ self.write_content_to_file(content) assert_raises( errors.ConfigurationError, loader.XMLConfiguration, self.tmpfile.name, safe=True) class TestPropertiesConfiguration(LoaderTestCase): content = textwrap.dedent(""" stars = in the sky blank.key = first.second=1 first.depth.then.more= j=t # Ignore the comment key with spaces = the value more.props = the end key.with.col : a value """) def test_properties_configuration(self): config_data = loader.PropertiesConfiguration(self.tmpfile.name) assert_equal(len(config_data), 7) assert_equal(config_data['stars'], 'in the sky') assert_equal(config_data['blank.key'], '') assert_equal(config_data['first.second'], '1') assert_equal(config_data['first.depth.then.more'], 'j=t') assert_equal(config_data['key with spaces'], 'the value') assert_equal(config_data['more.props'], 'the end') assert_equal(config_data['key.with.col'], 'a value') def test_invalid_line(self): self.tmpfile.write('justkey\n'.encode('utf8')) self.tmpfile.flush() assert_raises( errors.ConfigurationError, loader.PropertiesConfiguration, self.tmpfile.name) class TestCompositeConfiguration(object): def test_load(self): loaders = [(mock.Mock(return_value={i: 0}), 1, 2) for i in range(3)] composite = loader.CompositeConfiguration(loaders) assert_equal(composite.load(), {0: 0, 1: 0, 2: 0}) for loader_call, arg_one, arg_two in loaders: loader_call.assert_called_with(arg_one, arg_two) class StubObject(object): year = 2012 month = 3 hour = 15 _private = 'something' __really_private = 'hidden' class TestObjectConfiguration(LoaderTestCase): def test_load(self): config_data = loader.ObjectConfiguration(StubObject) assert_equal(config_data['year'], 2012) assert_equal(config_data['month'], 3) assert_equal(config_data['hour'], 15) assert_not_in('_private', config_data) assert_not_in('__really_private', config_data) PyStaticConfiguration-0.10.5/tests/proxy_test.py000066400000000000000000000125421365564735600220510ustar00rootroot00000000000000import datetime import os.path import pytest from testing.testifycompat import ( assert_equal, assert_raises_and_contains, assert_in, mock, ) from staticconf import proxy, validation, errors, config class TestExtractValue(object): @pytest.fixture(autouse=True) def setup_configuration_values(self): validator = mock.Mock(return_value=2) self.name = 'test_namespace' self.namespace = config.get_namespace(self.name) self.config_key = 'something' self.value_proxy = proxy.ValueProxy( validator, self.namespace, self.config_key) def test_extract_value_unset(self): expected = [self.name, self.config_key] assert_raises_and_contains( errors.ConfigurationError, expected, lambda: self.value_proxy.value) def test_get_value_fails_validation(self): expected = [self.name, self.config_key] validator = mock.Mock(side_effect=validation.ValidationError) _ = proxy.ValueProxy( # noqa: F841 validator, self.namespace, 'something.broken') assert_raises_and_contains( errors.ConfigurationError, expected, lambda: self.value_proxy.value) class TestValueProxy(object): @pytest.fixture(autouse=True) def setup_configuration_values(self): self.value_cache = { 'something': 2, 'something.string': 'the stars', 'zero': 0, 'the_date': datetime.datetime(2012, 3, 14, 4, 4, 4), 'the_list': range(3), } def test_proxy(self): validator = mock.Mock(return_value=2) value_proxy = proxy.ValueProxy(validator, self.value_cache, 'something') assert_equal(value_proxy, 2) assert value_proxy < 4 assert value_proxy > 1 assert_equal(value_proxy + 5, 7) assert_equal(repr(value_proxy), "2") assert_equal(str(value_proxy), "2") assert_equal(3 % value_proxy, 1) assert_equal(3 ** value_proxy, 9) assert_equal(value_proxy ** 3, 8) assert_equal(hash(value_proxy), 2) assert_equal(abs(value_proxy), 2) assert_equal(hex(value_proxy), "0x2") assert bool(value_proxy) assert_equal(range(5)[value_proxy], 2) assert_equal(range(5)[:value_proxy], range(2)) def test_proxy_with_string(self): validator = mock.Mock(return_value='one%s') value_proxy = proxy.ValueProxy(validator, self.value_cache, 'something') assert_equal(value_proxy, 'one%s') assert value_proxy < 'two' assert value_proxy > 'ab' assert os.path.join(value_proxy, 'a') == os.path.join('one%s', 'a') assert_equal(value_proxy + '!', 'one%s!') assert_equal(value_proxy % '!', 'one!') assert_equal(repr(value_proxy), "'one%s'") assert_equal(str(value_proxy), "one%s") assert_equal(hash(value_proxy), hash("one%s")) assert bool(value_proxy) def test_proxy_with_datetime(self): the_date = datetime.datetime(2012, 12, 1, 5, 5, 5) validator = mock.Mock(return_value=the_date) value_proxy = proxy.ValueProxy(validator, self.value_cache, 'something') assert_equal(value_proxy, the_date) assert value_proxy < datetime.datetime(2012, 12, 3) assert value_proxy > datetime.datetime(2012, 1, 4) four_days_ahead = datetime.datetime(2012, 12, 4, 5, 5, 5) assert_equal(value_proxy + datetime.timedelta(days=3), four_days_ahead) assert_equal(repr(value_proxy), repr(the_date)) assert_equal(str(value_proxy), str(the_date)) assert_equal(hash(value_proxy), hash(the_date)) assert bool(value_proxy) def test_proxy_zero(self): validator = mock.Mock(return_value=0) self.value_proxy = proxy.ValueProxy(validator, self.value_cache, 'zero') assert_equal(self.value_proxy, 0) assert not self.value_proxy assert not self.value_proxy and True assert not self.value_proxy or False assert not self.value_proxy ^ 0 assert ~ self.value_proxy def test_get_value(self): expected = "the stars" validator = mock.Mock(return_value=expected) value_proxy = proxy.ValueProxy( validator, self.value_cache, 'something.string') assert_equal(value_proxy, expected) def test_get_value_cached(self): expected = "the other stars" validator = mock.Mock() value_proxy = proxy.ValueProxy( validator, self.value_cache, 'something.string') value_proxy._value = expected assert_equal(value_proxy.value, expected) validator.assert_not_called() def test_proxied_attributes(self): validator = mock.Mock(return_value=self.value_cache['the_date']) value_proxy = proxy.ValueProxy(validator, self.value_cache, 'the_date') assert_equal(value_proxy.date(), datetime.date(2012, 3, 14)) assert_equal(value_proxy.hour, 4) def test_proxy_list(self): the_list = range(3) validator = mock.Mock(return_value=the_list) value_proxy = proxy.ValueProxy(validator, self.value_cache, 'the_list') assert_equal(value_proxy, the_list) assert_in(2, value_proxy) assert_equal(value_proxy[:1], range(0, 1)) assert_equal(len(value_proxy), 3) PyStaticConfiguration-0.10.5/tests/readers_test.py000066400000000000000000000040071365564735600223120ustar00rootroot00000000000000import pytest from testing.testifycompat import ( assert_equal, assert_raises, mock, ) from staticconf import config, readers, proxy, errors, testing class TestBuildReader(object): @pytest.fixture(autouse=True) def namespace(self): self.namespace = mock.create_autospec(config.ConfigNamespace) def test_read_config_success(self): config_key = 'the_key' value = readers._read_config(config_key, self.namespace, None) self.namespace.get.assert_called_with(config_key, default=None) assert_equal(value, self.namespace.get.return_value) def test_read_config_failed(self): self.namespace.get.return_value = proxy.UndefToken assert_raises( errors.ConfigurationError, readers._read_config, 'some_key', self.namespace, None) @mock.patch('staticconf.readers.config.get_namespace', autospec=True) def test_build_reader(self, mock_get_namespace): config_key, validator, self.namespace = 'the_key', mock.Mock(), 'the_name' reader = readers.build_reader(validator, self.namespace) value = reader(config_key) mock_get_namespace.assert_called_with(self.namespace) validator.assert_called_with( mock_get_namespace.return_value.get.return_value) assert_equal(value, validator.return_value) class TestNamespaceReader(object): config = { 'one': '1', 'three': '3.0', 'options': ['seven', 'stars'] } @pytest.yield_fixture(autouse=True) def patch_config(self): self.namespace = 'the_name' with testing.MockConfiguration(self.config, namespace=self.namespace): yield def test_readers(self): read_conf = readers.NamespaceReaders(self.namespace) assert_equal(read_conf.read_int('one'), 1) assert_equal(read_conf.read_float('three'), 3.0) assert_equal( read_conf.read_list_of_string('options'), ['seven', 'stars']) PyStaticConfiguration-0.10.5/tests/schema_test.py000066400000000000000000000066001365564735600221260ustar00rootroot00000000000000import pytest import six from testing.testifycompat import ( assert_equal, assert_raises, mock, ) from staticconf import testing, schema, validation, config, errors class TestCreateValueType(object): def test_build_value_type(self): help_text = 'what?' config_key = 'one' float_type = schema.build_value_type(validation.validate_float) assert callable(float_type) value_def = float_type(default=5, config_key=config_key, help=help_text) assert_equal(value_def.default, 5) assert_equal(value_def.help, help_text) assert_equal(value_def.config_key, config_key) @six.add_metaclass(schema.SchemaMeta) class ATestingSchema(object): namespace = 'my_testing_namespace' config_path = 'my.thing' one = schema.int(default=5) two = schema.string(help='the value for two') some_value = schema.any(config_key='three.four') when = schema.date() really_when = schema.datetime() at_time = schema.time() really = schema.bool() ratio = schema.float() all_of_them = schema.list() some_of_them = schema.set() wrapped = schema.tuple() options = schema.list_of_bool() @pytest.yield_fixture def meta_schema(): with mock.patch('staticconf.schema.config', autospec=True) as mock_config: with mock.patch('staticconf.schema.getters', autospec=True) as mock_getters: schema_object = ATestingSchema() yield schema_object.__class__, mock_config, mock_getters class TestSchemaMeta(object): def test_get_namespace_missing(self, meta_schema): meta, _, _ = meta_schema assert_raises(errors.ConfigurationError, meta.get_namespace, {}) def test_get_namespace_present(self, meta_schema): meta, mock_config, _ = meta_schema name = 'the_namespace' namespace = meta.get_namespace({'namespace': name}) mock_config.get_namespace.assert_called_with(name) assert_equal(namespace, mock_config.get_namespace.return_value) def test_build_attributes(self, meta_schema): meta, _, mock_getters = meta_schema value_def = mock.create_autospec(schema.ValueTypeDefinition) attributes = { 'not_a_token': None, 'a_token': value_def } namespace = mock.create_autospec(config.ConfigNamespace) attributes = meta.build_attributes(attributes, namespace) assert_equal(attributes['not_a_token'], None) assert_equal(list(attributes['_tokens'].keys()), ['a_token']) token = attributes['_tokens']['a_token'] assert_equal(token.config_key, value_def.config_key) assert_equal(token.default, value_def.default) assert isinstance(attributes['a_token'], property) mock_getters.register_value_proxy.assert_called_with( namespace, token, value_def.help) @pytest.yield_fixture def testing_schema_namespace(): conf = { 'my.thing.one': '1', 'my.thing.two': 'another', 'my.thing.three.four': 'deeper' } with testing.MockConfiguration(conf, namespace=ATestingSchema.namespace): yield class TestSchemaAcceptance(object): def test_schema(self, testing_schema_namespace): config_schema = ATestingSchema() assert_equal(config_schema.some_value, 'deeper') assert_equal(config_schema.one, 1) assert_equal(config_schema.two, 'another') PyStaticConfiguration-0.10.5/tests/testing_test.py000066400000000000000000000023201365564735600223360ustar00rootroot00000000000000 import staticconf from staticconf import testing from testing.testifycompat import assert_equal class TestMockConfiguration(object): def test_init(self): with testing.MockConfiguration(a='one', b='two'): assert_equal(staticconf.get('a'), 'one') assert_equal(staticconf.get('b'), 'two') def test_init_nested(self): conf = { 'a': { 'b': 'two', }, 'c': 'three' } with testing.MockConfiguration(conf): assert_equal(staticconf.get('a.b'), 'two') assert_equal(staticconf.get('c'), 'three') class TestPatchConfiguration(object): def test_nested(self): with testing.MockConfiguration(a='one', b='two'): with testing.PatchConfiguration(a='three'): assert_equal(staticconf.get('a'), 'three') assert_equal(staticconf.get('b'), 'two') assert_equal(staticconf.get('a'), 'one') assert_equal(staticconf.get('b'), 'two') def test_not_nested(self): with testing.PatchConfiguration(a='one', b='two'): assert_equal(staticconf.get('a'), 'one') assert_equal(staticconf.get('b'), 'two') PyStaticConfiguration-0.10.5/tests/validation_test.py000066400000000000000000000116571365564735600230300ustar00rootroot00000000000000import datetime import logging from staticconf import validation, errors from testing.testifycompat import ( assert_equal, assert_raises_and_contains, ) class TestValidation(object): def test_validate_string(self): assert_equal(None, validation.validate_string(None)) assert_equal('asd', validation.validate_string('asd')) assert_equal('123', validation.validate_string(123)) class TestDateTimeValidation(object): def test_validate_datetime(self): actual = validation.validate_datetime("2012-03-14 05:05:05") expected = datetime.datetime(2012, 3, 14, 5, 5, 5) assert_equal(actual, expected) def test_validate_datetime_pm_format(self): actual = validation.validate_datetime("2012-03-14 6:05:05 PM") expected = datetime.datetime(2012, 3, 14, 18, 5, 5) assert_equal(actual, expected) def test_validate_datetime_already_datetime(self): expected = datetime.datetime(2012, 3, 14, 18, 5, 5) actual = validation.validate_datetime(expected) assert_equal(actual, expected) def test_validate_date(self): actual = validation.validate_date("03/14/12") expected = datetime.date(2012, 3, 14) assert_equal(actual, expected) def test_validate_date_already_date(self): expected = datetime.date(2012, 3, 14) actual = validation.validate_date(expected) assert_equal(actual, expected) def test_validate_time(self): actual = validation.validate_time("4:12:14") expected = datetime.time(4, 12, 14) assert_equal(actual, expected) def test_validate_time_pm_format(self): actual = validation.validate_time("4:12:14 pm") expected = datetime.time(16, 12, 14) assert_equal(actual, expected) def test_validate_time_already_time(self): expected = datetime.time(16, 12, 14) actual = validation.validate_time(expected) assert_equal(actual, expected) class TestIterableValidation(object): def test_validate_list(self): assert_equal([0, 1, 2], validation.validate_list((0, 1, 2))) def test_validate_set(self): expected = set([3, 2, 1]) actual = validation.validate_set([1, 3, 2, 2, 1, 3, 2]) assert_equal(expected, actual) class TestRegexValidation(object): def test_validate_regex_success(self): pattern = r'^(:?what)\s+could\s+go\s+(wrong)[!?.,]$' actual = validation.validate_regex(pattern) assert_equal(pattern, actual.pattern) def test_validate_regex_failed(self): pattern = "((this) regex is broken" assert_raises_and_contains( errors.ValidationError, pattern, validation.validate_regex, pattern) def test_validate_regex_none(self): assert_raises_and_contains( errors.ValidationError, 'None', validation.validate_regex, None) class TestBuildListOfTypeValidator(object): def test_build_list_of_type_ints_success(self): validator = validation.build_list_type_validator( validation.validate_int) assert_equal(validator(['0', '1', '2']), [0, 1, 2]) def test_build_list_of_type_float_failed(self): validator = validation.build_list_type_validator( validation.validate_float) assert_raises_and_contains( errors.ValidationError, 'Invalid float: a', validator, ['0.1', 'a']) def test_build_list_of_type_empty_list(self): validator = validation.build_list_type_validator( validation.validate_string) assert_equal(validator([]), []) def test_build_list_of_type_not_a_list(self): validator = validation.build_list_type_validator( validation.validate_any) assert_raises_and_contains( errors.ValidationError, 'Invalid iterable', validator, None) class TestBuildMappingTypeValidator(object): def test_build_map_from_list_of_pairs(self): pair_validator = validation.build_map_type_validator(lambda i: i) expected = {0: 0, 1: 1, 2: 2} assert_equal(pair_validator(enumerate(range(3))), expected) def test_build_map_from_list_of_dicts(self): def map_by_id(d): return d['id'], d['value'] map_validator = validation.build_map_type_validator(map_by_id) expected = {'a': 'b', 'c': 'd'} source = [dict(id='a', value='b'), dict(id='c', value='d')] assert_equal(map_validator(source), expected) class TestValidateLogLevel(object): def test_valid_log_level(self): assert_equal(validation.validate_log_level('WARN'), logging.WARN) assert_equal(validation.validate_log_level('DEBUG'), logging.DEBUG) def test_invalid_log_level(self): assert_raises_and_contains( errors.ValidationError, 'UNKNOWN', validation.validate_log_level, 'UNKNOWN') PyStaticConfiguration-0.10.5/tox.ini000066400000000000000000000010651365564735600174260ustar00rootroot00000000000000[tox] envlist = py26,py27,py34,py35,py36,pypy,docs,coverage [testenv] deps = pyyaml pytest mock <= 1.0.1 flake8 commands = py.test {posargs:tests} flake8 staticconf tests testing [testenv:py26] commands = py.test {posargs:tests} [testenv:docs] deps = {[testenv]deps} sphinx >= 1.0 sphinx_rtd_theme changedir = docs commands = sphinx-build -b html -d build/doctrees source build/html [testenv:coverage] deps = {[testenv]deps} coverage commands = coverage run --source=staticconf {envbindir}/py.test tests