pypuppetdb-master/0000755000175000017500000000000012370463077013562 5ustar jonasjonaspypuppetdb-master/README.rst0000644000175000017500000002047412370425211015244 0ustar jonasjonas########## pypuppetdb ########## .. image:: https://api.travis-ci.org/nedap/pypuppetdb.png :target: https://travis-ci.org/nedap/pypuppetdb .. image:: https://coveralls.io/repos/nedap/pypuppetdb/badge.png :target: https://coeralls.io/r/nedap/pypuppetdb .. image:: https://pypip.in/d/pypuppetdb/badge.png :target: https://crate.io/packages/pypuppetdb .. image:: https://pypip.in/v/pypuppetdb/badge.png :target: https://crate.io/packages/pypuppetdb pypuppetdtb is a library to work with PuppetDB's REST API. It is implemented using the `requests`_ library. .. _requests: http://docs.python-requests.org/en/latest/ This library is a thin wrapper around the REST API providing some convinience functions and objects to request and hold data from PuppetDB. To use this library you will need: * Python 2.6 or 2.7 * Python 3.3 Installation ============ You can install this package from source or from PyPi. .. code-block:: bash $ pip install pypuppetdb .. code-block:: bash $ git clone https://github.com/nedap/pypuppetdb $ python setup.py install If you wish to hack on it clone the repository but after that run: .. code-block:: bash $ pip install -r requirements.txt This will install all the runtime requirements of pypuppetdb and the dependencies for the test suite and generation of documentation. Packages -------- Native packages for your operating system will be provided in the near future. +------------------+-----------+--------------------------------------------+ | OS | Status | | +==================+===========+============================================+ | Debian 6/Squeeze | planned | Requires Backports | +------------------+-----------+--------------------------------------------+ | Debian 7/Wheezy | planned | | +------------------+-----------+--------------------------------------------+ | Ubuntu 13.04 | planned | | +------------------+-----------+--------------------------------------------+ | Ubuntu 13.10 | planned | | +------------------+-----------+--------------------------------------------+ | CentOS/RHEL 5 | n/a | Python 2.4 | +------------------+-----------+--------------------------------------------+ | CentOS/RHEL 6 | planned | | +------------------+-----------+--------------------------------------------+ | `ArchLinux`_ | available | Maintained by `Niels Abspoel`_ | +------------------+-----------+--------------------------------------------+ | `OpenBSD`_ | available | Maintained by `Jasper Lievisse Adriaanse`_ | +------------------+-----------+--------------------------------------------+ .. _ArchLinux: https://aur.archlinux.org/packages/python2-pypuppetdb/ .. _Niels Abspoel: https://github.com/aboe76 .. _Jasper Lievisse Adriaanse: https://github.com/jasperla .. _OpenBSD: http://www.openbsd.org/cgi-bin/cvsweb/ports/databases/py-puppetdb/ Usage ===== Once you have pypuppetdb installed you can configure it to connect to PuppetDB and take it from there. Connecting ---------- The first thing you need to do is to connect with PuppetDB: .. code-block:: python >>> from pypuppetdb import connect >>> db = connect() Nodes ----- The following will return a generator object yielding Node objects for every returned node from PuppetDB. .. code-block:: python >>> nodes = db.nodes() >>> for node in nodes: >>> print(node) host1 host2 ... To query a single node the singular `node()` can be used: .. code-block:: python >>> node = db.node('hostname') >>> print(node) hostname Node scope ~~~~~~~~~~ The Node objects are a bit more special in that they can query for facts and resources themselves. Using those methods from a node object will automatically add a query to the request scoping the request to the node. .. code-block:: python >>> node = db.node('hostname') >>> print(node.fact('osfamily')) osfamily/hostname Facts ----- .. code-block:: python >>> facts = db.facts('osfamily') >>> for fact in facts: >>> print(fact) osfamily/host1 osfamily/host2 That queries PuppetDB for the 'osfamily' fact and will yield Fact objects, one per node this fact is known for. Resources --------- .. code-block:: python >>> resources = db.resources('file') Will return a generator object containing all file resources you're managing across your infrastructure. This is probably a bad idea if you have a big number of nodes as the response will be huge. Catalogs --------- .. code-block:: python >>> catalog = db.catalog('hostname') >>> for res in catalog.get_resources(): >>> print(res) Will return a Catalog object with the latest Catalog of the definded host. This catalog contains the defined Resources and Edges. .. code-block:: python >>> catalog = db.catalog('hostname') >>> resource = catalog.get_resource('Service','ntp') >>> for rel in resource.relationships: >>> print(rel) Class[Ntp] - contains - Service[ntp] File[/etc/ntp.conf] - notifies - Service[ntp] File[/etc/ntp.conf] - required-by - Service[ntp] Will return all Relationships of a given Resource defined by type and title. This will list all linked other Resources and the type of relationship. Getting Help ============ This project is still very new so it's not inconceivable you'll run into issues. For bug reports you can file an `issue`_. If you need help with something feel free to hit up `@daenney`_ by e-mail or find him on IRC. He can usually be found on `IRCnet`_ and `Freenode`_ and idles in #puppet. There's now also the #puppetboard channel on `Freenode`_ where we hang out and answer questions related to pypuppetdb and Puppetboard. .. _issue: https://github.com/nedap/pypuppetdb/issues .. _@daenney: https://github.com/daenney .. _IRCnet: http://www.ircnet.org .. _Freenode: http://freenode.net Documentation ============= API documentation is automatically generated from the docstrings using Sphinx's autodoc feature. Documentation will automatically be rebuilt on every push thanks to the Read The Docs webhook. You can `find it here`_. .. _find it here: https://pypuppetdb.readthedocs.org/en/latest/ You can build the documentation manually by doing: .. code-block:: bash $ cd docs $ make html Doing so will only work if you have Sphinx installed, which you can acheive through: .. code-block:: bash $ pip install -r requirements.txt Contributing ============ We welcome contributions to this library. However, there are a few ground rules contributors should be aware of. License ------- This project is licensed under the Apache v2.0 License. As such, your contributions, once accepted, are automatically covered by this license. Copyright (c) 2013-2014 Daniele Sluijters Commit messages --------------- Write decent commit messages. Don't use swear words and refrain from uninformative commit messages as 'fixed typo'. The preferred format of a commit message: :: docs/quickstart: Fixed a typo in the Nodes section. If needed, elaborate further on this commit. Feel free to write a complete blog post here if that helps us understand what this is all about. Fixes #4 and resolves #2. If you'd like a more elaborate guide on how to write and format your commit messages have a look at this post by `Tim Pope`_. .. _Tim Pope: http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html Tests ----- Commits are expected to contain tests or updates to tests if they add to or modify the current behaviour. The test suite is powered by `pytest`_ and requires `pytest`_, `pytest-pep8`_, `httpretty`_ and `pytest-httpretty`_ which will be installed for you if you run: .. code-block:: bash $ pip install -r requirements.txt .. _pytest: http://pytest.org/latest/ .. _pytest-pep8: https://pypi.python.org/pypi/pytest-pep8 .. _httpretty: https://pypi.python.org/pypi/httpretty/ .. _pytest-httpretty: https://github.com/papaeye/pytest-httpretty To run the unit tests (the ones that don't require a live PuppetDB): .. code-block:: bash $ py.test -v -m unit If the tests pass, you're golden. If not we'll have to figure out why and fix that. Feel free to ask for help on this. pypuppetdb-master/docs/0000755000175000017500000000000012370425211014476 5ustar jonasjonaspypuppetdb-master/docs/make.bat0000644000175000017500000001176012370425211016110 0ustar jonasjonas@ECHO OFF REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set BUILDDIR=_build set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . set I18NSPHINXOPTS=%SPHINXOPTS% . if NOT "%PAPER%" == "" ( set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% ) if "%1" == "" goto help if "%1" == "help" ( :help echo.Please use `make ^` where ^ is one of echo. html to make standalone HTML files echo. dirhtml to make HTML files named index.html in directories echo. singlehtml to make a single large HTML file echo. pickle to make pickle files echo. json to make JSON files echo. htmlhelp to make HTML files and a HTML help project echo. qthelp to make HTML files and a qthelp project echo. devhelp to make HTML files and a Devhelp project echo. epub to make an epub echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter echo. text to make text files echo. man to make manual pages echo. texinfo to make Texinfo files echo. gettext to make PO message catalogs echo. changes to make an overview over all changed/added/deprecated items echo. linkcheck to check all external links for integrity echo. doctest to run all doctests embedded in the documentation if enabled goto end ) if "%1" == "clean" ( for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i del /q /s %BUILDDIR%\* goto end ) if "%1" == "html" ( %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/html. goto end ) if "%1" == "dirhtml" ( %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. goto end ) if "%1" == "singlehtml" ( %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. goto end ) if "%1" == "pickle" ( %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the pickle files. goto end ) if "%1" == "json" ( %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the JSON files. goto end ) if "%1" == "htmlhelp" ( %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run HTML Help Workshop with the ^ .hhp project file in %BUILDDIR%/htmlhelp. goto end ) if "%1" == "qthelp" ( %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run "qcollectiongenerator" with the ^ .qhcp project file in %BUILDDIR%/qthelp, like this: echo.^> qcollectiongenerator %BUILDDIR%\qthelp\PyPuppetDB.qhcp echo.To view the help file: echo.^> assistant -collectionFile %BUILDDIR%\qthelp\PyPuppetDB.ghc goto end ) if "%1" == "devhelp" ( %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp if errorlevel 1 exit /b 1 echo. echo.Build finished. goto end ) if "%1" == "epub" ( %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub if errorlevel 1 exit /b 1 echo. echo.Build finished. The epub file is in %BUILDDIR%/epub. goto end ) if "%1" == "latex" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex if errorlevel 1 exit /b 1 echo. echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. goto end ) if "%1" == "text" ( %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text if errorlevel 1 exit /b 1 echo. echo.Build finished. The text files are in %BUILDDIR%/text. goto end ) if "%1" == "man" ( %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man if errorlevel 1 exit /b 1 echo. echo.Build finished. The manual pages are in %BUILDDIR%/man. goto end ) if "%1" == "texinfo" ( %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo if errorlevel 1 exit /b 1 echo. echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. goto end ) if "%1" == "gettext" ( %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale if errorlevel 1 exit /b 1 echo. echo.Build finished. The message catalogs are in %BUILDDIR%/locale. goto end ) if "%1" == "changes" ( %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes if errorlevel 1 exit /b 1 echo. echo.The overview file is in %BUILDDIR%/changes. goto end ) if "%1" == "linkcheck" ( %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck if errorlevel 1 exit /b 1 echo. echo.Link check complete; look for any errors in the above output ^ or in %BUILDDIR%/linkcheck/output.txt. goto end ) if "%1" == "doctest" ( %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest if errorlevel 1 exit /b 1 echo. echo.Testing of doctests in the sources finished, look at the ^ results in %BUILDDIR%/doctest/output.txt. goto end ) :end pypuppetdb-master/docs/index.rst0000644000175000017500000000144212370425211016340 0ustar jonasjonasWelcome to pypuppetdb's documentation! ====================================== .. note:: This is a very new project and still changing at a rapid pace. As such the only documentation currently available is the API documentation and a brief Getting Started guide. Once this settles down tutorials and other documentation will be added over time. Getting started --------------- The quickstart should get you up and running with pypuppetdb and familiarise you with how this library works. .. toctree:: :maxdepth: 2 quickstart API Documentation ----------------- This part of the documentation focusses on the classes, methods and functions that make up this library. .. toctree:: :maxdepth: 2 api Indices and tables ================== * :ref:`genindex` * :ref:`search` pypuppetdb-master/docs/quickstart.rst0000644000175000017500000000721612370425211017430 0ustar jonasjonas.. _quickstart: Quickstart ========== Once you have pypuppetdb installed you can configure it to connect to PuppetDB and take it from there. Connecting ---------- The first thing you need to do is to connect with PuppetDB: .. code-block:: python >>> from pypuppetdb import connect >>> db = connect() Nodes ----- The following will return a generator object yielding Node objects for every returned node from PuppetDB. .. code-block:: python >>> nodes = db.nodes() >>> for node in nodes: >>> print(node) host1 host2 ... To query a single node the singular `node()` can be used: .. code-block:: python >>> node = db.node('hostname') >>> print(node) hostname Node scope ~~~~~~~~~~ The Node objects are a bit more special in that they can query for facts and resources themselves. Using those methods from a node object will automatically add a query to the request scoping the request to the node. .. code-block:: python >>> node = db.node('hostname') >>> print(node.fact('osfamily')) osfamily/hostname Facts ----- .. code-block:: python >>> facts = db.facts('osfamily') >>> for fact in facts: >>> print(fact) osfamily/host1 osfamily/host2 That queries PuppetDB for the 'osfamily' fact and will yield Fact objects, one per node this fact is known for. Resources --------- .. code-block:: python >>> resources = db.resources('file') Will return a generator object containing all file resources you're managing across your infrastructure. This is probably a bad idea if you have a big number of nodes as the response will be huge. SSL --- If PuppetDB and the tool that's using pypuppetdb aren't located on the same machine you will have to connect securely to PuppetDB using client certificates according to PuppetDB's default configuration. You can also tell PuppetDB to accept plain connections from anywhere instead of just the local machine but **don't do that**. Pypuppetdb can handle this easily for you. It requires two things: * Generate with your Puppet CA a key pair that you want to use * Tell pypuppetdb to use this keypair. Generate keypair ~~~~~~~~~~~~~~~~ On your Puppet Master or dedicated Puppet CA server: .. code-block:: console $ puppet cert generate Once that's done you'll need to get the public and private keyfile and copy them over. You can find those in Puppet's ``$ssldir``, usually ``/var/lib/puppet/ssl``: * private key: ``$ssldir/private_keys/.pem`` * public key: ``$ssldir/ca/signed/.pem`` Configure pypuppetdb for SSL ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Once you have those you can pass them to pypuppetdb's ``connect()``: .. code-block:: python >>> db = connect(ssl_key='/path/to/private.pem', ssl_cert='/path/to/public.pem') If both ``ssl_key`` and ``ssl_cert`` are provided pypuppetdb will automatically switch over to using HTTPS instead. By default pypuppetdb will also verify the certificate PuppetDB is serving. This means that the authority that signed PuppetDB's server certificate, most likely your Puppet Master, must be part of the trusted set of certificates for your OS or must be added to that set. Those certificates are usually found in ``/etc/ssl/certs`` on Linux-y machines. For Debian, install your Puppet Master's certificate in ``/usr/local/share/ca-certifiactes`` with a ``.crt`` extension and then run ``dpkg-reconfigure ca-certificates`` as per ``/usr/share/doc/ca-certificates/README.Debian``. This of course requires the ``ca-certificates`` package to be installed. If you do not wish to do so or for whatever reason want to disable the verification of PuppetDB's certificate you can pass in ``ssl_verify=False``. pypuppetdb-master/docs/Makefile0000644000175000017500000001271412370425211016143 0ustar jonasjonas# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: -rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/PyPuppetDB.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/PyPuppetDB.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/PyPuppetDB" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/PyPuppetDB" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." pypuppetdb-master/docs/api.rst0000644000175000017500000000710112370425211016000 0ustar jonasjonas.. _api: Developer Interface =================== .. module:: pypuppetdb This part of the documentation covers all the interfaces of PyPuppetDB. It will cover how the API is set up and how to configure which version of the API to use. Lazy objects ------------ .. note:: Reading in the response from PuppetDB is currently greedy, it will read in the complete response no matter the size. This will change once streaming and pagination support are added to PuppetDB's endpoints. In order for pypuppetdb to be able to deal with big datasets those functions that are expected to return more than a single item are implemented as generators. This is usually the case for functions with a plural name like :func:`~pypuppetdb.api.v2.API.nodes` or :func:`~pypuppetdb.api.v2.API.facts`. Because of this we'll only query PuppetDB once you start iterating over the generator object. Until that time not a single request is fired at PuppetDB. Most singular functions are implemented by calling their plural counterpart and then iterating over the generator, immediately exhausting the generator and returning a single/the first object. Main Interface -------------- What you'll usually need to do is use the :func:`connect` method to set up a connection with PuppetDB and indicate which version of the API you want to talk. .. autofunction:: connect API objects ----------- The PuppetDB API is versioned. We currently have a v1, v2 and v3. In order to work with this structure PyPuppetDB consists of a :class:`BaseAPI ` class that factors out identical code between different versions. Every version of the API has its own class which inherits from our :class:`BaseAPI `. .. data:: API_VERSIONS :obj:`dict` of :obj:`int`::obj:`string` pairs representing the API version and it's URL prefix. We currently only handle API version 2 though it should be fairly easy to support version 1 should we want to. BaseAPI ^^^^^^^ .. autoclass:: pypuppetdb.api.BaseAPI :members: :private-members: v2.API ^^^^^^ .. autoclass:: pypuppetdb.api.v2.API :members: :inherited-members: :private-members: :show-inheritance: v3.API ^^^^^^ .. autoclass:: pypuppetdb.api.v3.API :members: :inherited-members: :private-members: :show-inheritance: Types ----- In order to facilitate working with the API most methods like :meth:`~pypuppetdb.api.v2.API.nodes` don't return the decoded JSON response but return an object representation of the querried endpoints data. .. autoclass:: pypuppetdb.types.Node :members: .. autoclass:: pypuppetdb.types.Fact .. autoclass:: pypuppetdb.types.Resource .. autoclass:: pypuppetdb.types.Event .. autoclass:: pypuppetdb.types.Report .. autoclass:: pypuppetdb.types.Catalog .. autoclass:: pypuppetdb.types.Edge Errors ------ Unfortunately things can go haywire. PuppetDB might not be reachable or complain about our query, requests might have to wait too long to recieve a response or the body is just too big to handle. In that case, we'll throw an exception at you. .. autoexception:: pypuppetdb.errors.APIError .. autoexception:: pypuppetdb.errors.ImproperlyConfiguredError :show-inheritance: .. autoexception:: pypuppetdb.errors.UnsupportedVersionError :show-inheritance: .. autoexception:: pypuppetdb.errors.DoesNotComputeError :show-inheritance: .. autoexception:: pypuppetdb.errors.EmptyResponseError :show-inheritance: Utilities --------- A few functions that are used across this library have been put into their own :mod:`utils` module. .. autoclass:: pypuppetdb.utils.UTC .. autofunction:: pypuppetdb.utils.json_to_datetime pypuppetdb-master/docs/conf.py0000644000175000017500000000274312370425211016003 0ustar jonasjonas# -*- coding: utf-8 -*- import sys, os pypuppetdb_root = os.path.dirname(os.path.abspath('.')) sys.path.insert(0, pypuppetdb_root) import pypuppetdb.package # -- General configuration ----------------------------------------------------- extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode'] templates_path = ['_templates'] source_suffix = '.rst' master_doc = 'index' project = pypuppetdb.package.__title__ copyright = '{0}, {1}'.format(pypuppetdb.package.__year__, pypuppetdb.package.__author__) version = pypuppetdb.package.__version__ release = version language = 'en' exclude_patterns = ['_build'] pygments_style = 'sphinx' # -- Options for HTML output --------------------------------------------------- html_theme = 'default' html_static_path = ['_static'] htmlhelp_basename = 'pypuppetdbdoc' # -- Options for LaTeX output -------------------------------------------------- latex_documents = [ ('index', 'pypuppetdb.tex', u'pypuppetdb Documentation', u'Daniele Sluijters', 'manual'), ] # -- Options for manual page output -------------------------------------------- man_pages = [ ('index', 'pypuppetdb', u'pypuppetdb Documentation', [u'Daniele Sluijters'], 1) ] # -- Options for Texinfo output ------------------------------------------------ texinfo_documents = [ ('index', 'pypuppetdb', u'pypuppetdb Documentation', u'Daniele Sluijters', 'pypuppetdb', 'Library to work with the PuppetDB REST API.', 'Miscellaneous'), ] pypuppetdb-master/.travis.yml0000644000175000017500000000043212370425211015656 0ustar jonasjonaslanguage: python python: - 2.6 - 2.7 - 3.3 install: - pip install -q -r test-requirements.txt --use-wheel - pip install -q coverage coveralls --use-wheel script: coverage run --source pypuppetdb -m py.test --pep8 after_success: - coveralls notifications: email: false pypuppetdb-master/tox.ini0000644000175000017500000000033712370425211015064 0ustar jonasjonas[tox] envlist = py26, py27, py33, pypy, pep8 [testenv] commands = py.test -m 'unit' deps = -r{toxinidir}/requirements.txt [testenv:pep8] basepython = python2.6 commands = py.test --pep8 -m 'not (unit or integration)' pypuppetdb-master/LICENSE0000644000175000017500000002404112370425211014554 0ustar jonasjonasApache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: You must give any other recipients of the Work or Derivative Works a copy of this License; and You must cause any modified files to carry prominent notices stating that You changed the files; and You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. pypuppetdb-master/test-requirements.txt0000644000175000017500000000023712370425211020011 0ustar jonasjonasrequests==2.1.0 pytest==2.5.0 mock==1.0.1 git+https://github.com/gabrielfalcao/HTTPretty.git@python-3.3-support#egg=httpretty pytest-pep8==1.0.5 coverage==3.7 pypuppetdb-master/setup.py0000644000175000017500000000341512370463077015277 0ustar jonasjonasimport sys import os import codecs from setuptools import setup, find_packages from setuptools.command.test import test as TestCommand if sys.argv[-1] == 'publish': os.system('python setup.py sdist upload') sys.exit() class Tox(TestCommand): def finalize_options(self): TestCommand.finalize_options(self) self.test_args = [] self.test_suite = True def run_tests(self): # import here, cause outside the eggs aren't loaded import tox errno = tox.cmdline(self.test_args) sys.exit(errno) with codecs.open('README.rst', encoding='utf-8') as f: README = f.read() with codecs.open('CHANGELOG.rst', encoding='utf-8') as f: CHANGELOG = f.read() setup( name='pypuppetdb', version='0.1.1+git080614', author='Daniele Sluijters', author_email='daniele.sluijters+pypi@gmail.com', packages=find_packages(), url='https://github.com/nedap/pypuppetdb', license='Apache License 2.0', description='Library for working with the PuppetDB REST API.', long_description='\n'.join((README, CHANGELOG)), keywords='puppet puppetdb', tests_require=['tox'], cmdclass={'test': Tox}, install_requires=[ "requests >= 1.2.3", ], classifiers=[ 'Development Status :: 4 - Beta', 'Intended Audience :: Developers', 'Natural Language :: English', 'License :: OSI Approved :: Apache Software License', 'Operating System :: POSIX', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.6', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.3', 'Topic :: Software Development :: Libraries' ], ) pypuppetdb-master/conftest.py0000644000175000017500000000102112370425211015737 0ustar jonasjonasimport pytest import pypuppetdb # Set up our API objects @pytest.fixture(scope='session') def api2(): """Set up a connection to PuppetDB with API version 2.""" return pypuppetdb.connect(api_version=2) @pytest.fixture(scope='session') def api3(): """Set up a connection to PuppetDB with API version 3.""" return pypuppetdb.connect(api_version=3) @pytest.fixture def baseapi(): return pypuppetdb.api.BaseAPI(3) @pytest.fixture def utc(): """Create a UTC object.""" return pypuppetdb.utils.UTC() pypuppetdb-master/python-pypuppetdb.spec0000644000175000017500000000271612370425211020143 0ustar jonasjonas%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")} Name: python-pypuppetdb Version: 0.0.4 Release: 2%{?dist} Summary: A Python puppetdb API Group: Development/Languages License: Apache URL: https://github.com/nedap/pypuppetdb Source0: http://pypi.python.org/packages/source/p/pypuppetdb/pypuppetdb-%{version}.tar.gz BuildRoot: %{_tmppath}/%{name}-%{version}-%{release} BuildArch: noarch BuildRequires: python-setuptools Requires: python-requests >= 1.1.0 Requires: python >= 2.6 %description pypuppetdb is a library to work with PuppetDB's REST API. It is implemented using the requests library. This library is a thin wrapper around the REST API providing some convinience functions and objects to request and hold data from PuppetDB. To use this library you will need: Python 2.6 or 2.7 or Python 3.3. %prep %setup -q -n pypuppetdb-%{version} %{__rm} -rf *.egg-info %{__sed} -i 's,^#!.*env python.*$,#!/usr/bin/python,' setup.py %build %install rm -rf %{buildroot} %{__python} setup.py install -O1 --root %{buildroot} %clean rm -rf %{buildroot} %files %defattr(-,root,root,-) %doc %{python_sitelib}/* %changelog * Tue Oct 15 2013 Klavs Klavsen - 0.0.4-2 - Add requirements, description and other small cleanups. * Mon Oct 14 2013 Klavs Klavsen - 0.0.4-1 - Initial release. pypuppetdb-master/setup.cfg0000644000175000017500000000036012370425211015366 0ustar jonasjonas[pytest] norecursedirs = docs .tox minversion = 2.3 markers = unit: mark test as a unit test. Runs without PuppetDB. integration: mark test as an integration test. Needs a live PuppetDB with testdata loaded. [wheel] universal = 1 pypuppetdb-master/.gitignore0000644000175000017500000000064412370425211015542 0ustar jonasjonas*.py[cod] # C extensions *.so # Pytest .cache __pycache__ # Packages *.egg *.egg-info dist build eggs parts bin var sdist develop-eggs .installed.cfg lib lib64 # Installer logs pip-log.txt # Unit test / coverage reports .coverage .tox nosetests.xml htmlcov # Translations *.mo # Virtualenv / pyenv .python-version .venv # Sphinx docs/_build # Mr Developer .mr.developer.cfg .project .pydevproject # OS X .DS_* pypuppetdb-master/.coveragerc0000644000175000017500000000005412370425211015666 0ustar jonasjonas[report] exclude_lines = pragma: notest pypuppetdb-master/MANIFEST.in0000644000175000017500000000012212370425211015277 0ustar jonasjonasinclude README.rst include CHANGELOG.rst include LICENSE include requirements.txt pypuppetdb-master/CHANGELOG.rst0000644000175000017500000000734512370425211015600 0ustar jonasjonas######### Changelog ######### 0.1.1 ===== * Fix the license in our ``setup.py``. The license shouldn't be longer than 200 characters. We were including the full license tripping up tools like bdist_rpm. 0.1.0 ===== Significant changes have been made in this release. The complete v3 API is now supported except for query pagination. Most changes are backwards compatible except for a change in the SSL configuration. The previous behaviour was buggy and slightly misleading in the names the options took: * ``ssl`` has been renamed to ``ssl_verify`` and now defaults to ``True``. * Automatically use HTTPS if ``ssl_key`` and ``ssl_cert`` are provided. For additional instructions about getting SSL to work see the Quickstart in the documentation. Deprecation ------------ Support for API v2 will be dropped in the 0.2.x release series. New features ------------ The following features are **only** supported for **API v3**. The ``node()`` and ``nodes()`` function have gained the following options: * ``with_status=False`` * ``unreported=2`` When ``with_status`` is set to ``True`` an additional query will be made using the ``events-count`` endpoint scoped to the latest report. This will result in an additional ``events`` and ``status`` keys on the node object. ``status`` will be either of ``changed``, ``unchanged`` or ``failed`` depending on if ``events`` contains ``successes`` or ``failures`` or none. By default ``unreported`` is set to ``2``. This is only in effect when ``with_status`` is set to ``True``. It means that if a node hasn't checked in for two hours it will get a ``status`` of ``unreported`` instead. New endpoints: * ``events-count``: ``events_count()`` * ``aggregate-event-counts``: ``aggregate_event_counts()`` * ``server-time``: ``server_time()`` * ``version``: ``current_version()`` * ``catalog``: ``catalog()`` New types: * ``pypuppetdb.types.Catalog`` * ``pypuppetdb.types.Edge`` Changes to types: * ``pypuppetdb.types.Node`` now has: * ``status`` defaulting to ``None`` * ``events`` defaulting to ``None`` * ``unreported_time`` defaulting to ``None`` 0.0.4 ===== Due to a fairly serious bug 0.0.3 was pulled from PyPi minutes after release. When a bug was fixed to be able to query for all facts we accidentally introduced a different bug that caused the ``facts()`` call on a node to query for all facts because we were resetting the query. * Fix a bug where ``node.facts()`` was causing us to query all facts because the query to scope our request was being reset. 0.0.3 ===== With the introduction of PuppetDB 1.5 a new API version, v3, was also introduced. In that same release the old ``/experimental`` endpoints were removed, meaning that as of PuppetDB 1.5 with the v2 API you can no longer get access to reports or events. In light of this the support for the experimental endpoints has been completely removed from pypuppetdb. As of this release you can only get to reports and/or events through v3 of the API. This release includes preliminary support for the v3 API. Everything that could be done with v2 plus the experimental endpoints is now possible on v3. However, more advanced funtionality has not yet been implemented. That will be the focus of the next release. * Removed dependency on pytz. * Fixed the behaviour of ``facts()`` and ``resources()``. We can now correctly query for all facts or resources. * Fixed an issue with catalog timestampless nodes. * Pass along the ``timeout`` option to ``connect()``. * Added preliminary PuppetDB API v3 support. * Removed support for the experimental endpoints. * The ``connect()`` method defaults to API v3 now. 0.0.2 ===== * Fix a bug in ``setup.py`` preventing successful installation. 0.0.1 ===== Initial release. Implements most of the v2 API. pypuppetdb-master/tests/0000755000175000017500000000000012370425211014710 5ustar jonasjonaspypuppetdb-master/tests/test_types.py0000644000175000017500000001637512370425211017501 0ustar jonasjonasimport sys from pypuppetdb.utils import json_to_datetime from pypuppetdb.types import ( Node, Fact, Resource, Report, Event, Catalog, Edge ) if sys.version_info >= (3, 0): unicode = str class TestNode(object): """Test the Node object.""" def test_without_status(self): node = Node('_', 'node', report_timestamp='2013-08-01T09:57:00.000Z', catalog_timestamp='2013-08-01T09:57:00.000Z', facts_timestamp='2013-08-01T09:57:00.000Z',) assert node.name == 'node' assert node.deactivated is False assert node.report_timestamp is not None assert node.facts_timestamp is not None assert node.catalog_timestamp is not None assert str(node) == str('node') assert unicode(node) == unicode('node') assert repr(node) == str('') def test_with_status_unreported(self): node = Node('_', 'node', report_timestamp='2013-08-01T09:57:00.000Z', catalog_timestamp='2013-08-01T09:57:00.000Z', facts_timestamp='2013-08-01T09:57:00.000Z', status='unreported', unreported_time='0d 5h 20m',) assert node.name == 'node' assert node.deactivated is False assert node.report_timestamp is not None assert node.facts_timestamp is not None assert node.catalog_timestamp is not None assert node.status is 'unreported' assert node.unreported_time is '0d 5h 20m' assert str(node) == str('node') assert unicode(node) == unicode('node') assert repr(node) == str('') class TestFact(object): """Test the Fact object.""" def test_fact(self): fact = Fact('node', 'osfamily', 'Debian') assert fact.node == 'node' assert fact.name == 'osfamily' assert fact.value == 'Debian' assert str(fact) == str('osfamily/node') assert unicode(fact) == unicode('osfamily/node') assert repr(fact) == str('Fact: osfamily/node') class TestResource(object): "Test the Resource object.""" def test_resource(self): resource = Resource('node', '/etc/ssh/sshd_config', 'file', ['class', 'ssh'], False, '/ssh/manifests/init.pp', 15, parameters={ 'ensure': 'present', 'owner': 'root', 'group': 'root', 'mode': '0600', }) assert resource.node == 'node' assert resource.name == '/etc/ssh/sshd_config' assert resource.type_ == 'file' assert resource.tags == ['class', 'ssh'] assert resource.exported is False assert resource.sourcefile == '/ssh/manifests/init.pp' assert resource.sourceline == 15 assert resource.parameters['ensure'] == 'present' assert resource.parameters['owner'] == 'root' assert resource.parameters['group'] == 'root' assert resource.parameters['mode'] == '0600' assert str(resource) == str('file[/etc/ssh/sshd_config]') assert unicode(resource) == unicode('file[/etc/ssh/sshd_config]') assert repr(resource) == str( '') class TestReport(object): """Test the Report object.""" def test_report(self): report = Report('node1.puppet.board', 'hash#', '2013-08-01T09:57:00.000Z', '2013-08-01T10:57:00.000Z', '2013-08-01T10:58:00.000Z', '1351535883', 3, '3.2.1', 'af9f16e3-75f6-4f90-acc6-f83d6524a6f3') assert report.node == 'node1.puppet.board' assert report.hash_ == 'hash#' assert report.start == json_to_datetime('2013-08-01T09:57:00.000Z') assert report.end == json_to_datetime('2013-08-01T10:57:00.000Z') assert report.received == json_to_datetime('2013-08-01T10:58:00.000Z') assert report.version == '1351535883' assert report.format_ == 3 assert report.agent_version == '3.2.1' assert report.run_time == report.end - report.start assert report.transaction == 'af9f16e3-75f6-4f90-acc6-f83d6524a6f3' assert str(report) == str('hash#') assert unicode(report) == unicode('hash#') assert repr(report) == str('Report: hash#') class TestEvent(object): """Test the Event object.""" def test_event(self): event = Event('node', 'failure', '2013-08-01T10:57:00.000Z', 'hash#', '/etc/ssh/sshd_config', 'ensure', 'Nothing to say', 'present', 'absent', 'file') assert event.node == 'node' assert event.status == 'failure' assert event.failed is True assert event.timestamp == json_to_datetime('2013-08-01T10:57:00.000Z') assert event.hash_ == 'hash#' assert event.item['title'] == '/etc/ssh/sshd_config' assert event.item['type'] == 'file' assert event.item['property'] == 'ensure' assert event.item['message'] == 'Nothing to say' assert event.item['old'] == 'absent' assert event.item['new'] == 'present' assert str(event) == str('file[/etc/ssh/sshd_config]/hash#') assert unicode(event) == unicode('file[/etc/ssh/sshd_config]/hash#') assert repr(event) == str('Event: file[/etc/ssh/sshd_config]/hash#') def test_event_failed(self): event = Event('node', 'success', '2013-08-01T10:57:00.000Z', 'hash#', '/etc/ssh/sshd_config', 'ensure', 'Nothing to say', 'present', 'absent', 'file') assert event.status == 'success' assert event.failed is False class TestCatalog(object): """Test the Catalog object.""" def test_catalog(self): catalog = Catalog('node', [], [], 'unique', None) assert catalog.node == 'node' assert catalog.version == 'unique' assert catalog.transaction_uuid is None assert catalog.resources == {} assert catalog.edges == [] assert str(catalog) == str('node/None') assert unicode(catalog) == unicode('node/None') assert repr(catalog) == str( '') class TestEdge(object): """Test the Edge object.""" def test_edge(self): resource_a = Resource('node', '/etc/ssh/sshd_config', 'file', ['class', 'ssh'], False, '/ssh/manifests/init.pp', 15, parameters={}) resource_b = Resource('node', 'sshd', 'service', ['class', 'ssh'], False, '/ssh/manifests/init.pp', 30, parameters={}) edge = Edge(resource_a, resource_b, 'notify') assert edge.source == resource_a assert edge.target == resource_b assert edge.relationship == 'notify' assert str(edge) == str( 'file[/etc/ssh/sshd_config] - notify - service[sshd]') assert unicode(edge) == unicode( 'file[/etc/ssh/sshd_config] - notify - service[sshd]') assert repr(edge) == str( '') pypuppetdb-master/tests/test_connect.py0000644000175000017500000000063312370425211017754 0ustar jonasjonasimport pytest import pypuppetdb def test_connect_unknown_api_version(): with pytest.raises(pypuppetdb.errors.UnsupportedVersionError): pypuppetdb.connect(api_version=1) def test_connect_api_v2(): puppetdb = pypuppetdb.connect(api_version=2) assert puppetdb.version == 'v2' def test_connect_api_v3(): puppetdb = pypuppetdb.connect(api_version=3) assert puppetdb.version == 'v3' pypuppetdb-master/tests/test_package.py0000644000175000017500000000055612370425211017722 0ustar jonasjonasfrom pypuppetdb.package import ( __title__, __author__, __license__, __year__, __copyright__) def test_package(): assert __title__ == 'pypuppetdb' assert __author__ == 'Daniele Sluijters' assert __license__ == 'Apache License 2.0' assert __year__ == '2013, 2014' assert __copyright__ == 'Copyright {0} {1}'.format(__year__, __author__) pypuppetdb-master/tests/test_utils.py0000644000175000017500000000257512370425211017472 0ustar jonasjonasfrom __future__ import unicode_literals import sys import pytest import pypuppetdb import datetime if sys.version_info >= (3, 0): unicode = str class TestUTC(object): """Test the UTC class.""" def test_utc_offset(self, utc): assert datetime.timedelta(0) == utc.utcoffset(300) def test_tzname(self, utc): assert str('UTC') == utc.tzname(300) def test_dst(self, utc): assert datetime.timedelta(0) == utc.dst(300) def test_magic_str(self, utc): assert str('UTC') == str(utc) def test_magic_unicode(self, utc): assert 'UTC' == unicode(utc) def test_magic_repr(self, utc): assert str('') == repr(utc) class TestJSONToDateTime(object): """Test the json_to_datetime function.""" def test_json_to_datetime(self): json_datetime = '2013-08-01T09:57:00.000Z' python_datetime = pypuppetdb.utils.json_to_datetime(json_datetime) assert python_datetime.dst() == datetime.timedelta(0) assert python_datetime.date() == datetime.date(2013, 8, 1) assert python_datetime.tzname() == 'UTC' assert python_datetime.utcoffset() == datetime.timedelta(0) assert python_datetime.dst() == datetime.timedelta(0) def test_json_to_datetime_invalid(self): with pytest.raises(ValueError): pypuppetdb.utils.json_to_datetime('2013-08-0109:57:00.000Z') pypuppetdb-master/tests/test_baseapi.py0000644000175000017500000002162012370425211017726 0ustar jonasjonasimport json import mock import httpretty import pytest import requests import pypuppetdb def stub_request(url, data=None, **kwargs): if data is None: body = '[]' else: with open(data, 'r') as d: body = json.load(d.read()) return httpretty.register_uri(httpretty.GET, url, body=body, status=200, **kwargs) class TestBaseAPIVersion(object): def test_init_v2_defaults(self): v2 = pypuppetdb.api.BaseAPI(2) assert v2.api_version == 'v2' def test_init_v3_defaults(self): v3 = pypuppetdb.api.BaseAPI(3) assert v3.api_version == 'v3' def test_init_invalid_version(self): with pytest.raises(pypuppetdb.errors.UnsupportedVersionError): vderp = pypuppetdb.api.BaseAPI(10000) class TestBaseAPIInitOptions(object): def test_defaults(self, baseapi): assert baseapi.host == 'localhost' assert baseapi.port == 8080 assert baseapi.ssl_verify is True assert baseapi.ssl_key is None assert baseapi.ssl_cert is None assert baseapi.timeout == 10 assert baseapi.protocol == 'http' def test_host(self): api = pypuppetdb.api.BaseAPI(3, host='127.0.0.1') assert api.host == '127.0.0.1' def test_port(self): api = pypuppetdb.api.BaseAPI(3, port=8081) assert api.port == 8081 def test_ssl_verify(self): api = pypuppetdb.api.BaseAPI(3, ssl_verify=False) assert api.ssl_verify is False assert api.protocol == 'http' def test_ssl_key(self): api = pypuppetdb.api.BaseAPI(3, ssl_key='/a/b/c.pem') assert api.ssl_key == '/a/b/c.pem' assert api.protocol == 'http' def test_ssl_cert(self): api = pypuppetdb.api.BaseAPI(3, ssl_cert='/d/e/f.pem') assert api.ssl_cert == '/d/e/f.pem' assert api.protocol == 'http' def test_ssl_key_and_cert(self): api = pypuppetdb.api.BaseAPI(3, ssl_cert='/d/e/f.pem', ssl_key='/a/b/c.pem') assert api.ssl_key == '/a/b/c.pem' assert api.ssl_cert == '/d/e/f.pem' assert api.protocol == 'https' def test_timeout(self): api = pypuppetdb.api.BaseAPI(3, timeout=20) assert api.timeout == 20 class TestBaseAPIProperties(object): def test_version(self, baseapi): assert baseapi.version == 'v3' def test_base_url(self, baseapi): assert baseapi.base_url == 'http://localhost:8080' def test_base_url_ssl(self, baseapi): baseapi.protocol = 'https' # slightly evil assert baseapi.base_url == 'https://localhost:8080' def test_total(self, baseapi): baseapi.last_total = 10 # slightly evil assert baseapi.total == 10 class TestBaseAPIURL(object): def test_without_path(self, baseapi): assert baseapi._url('nodes') == 'http://localhost:8080/v3/nodes' def test_with_invalid_endpoint(self, baseapi): with pytest.raises(pypuppetdb.errors.APIError): baseapi._url('this_will-Never+Ex1s7') def test_with_path(self, baseapi): url = baseapi._url('nodes', path='node1.example.com') assert url == 'http://localhost:8080/v3/nodes/node1.example.com' class TesteAPIQuery(object): @mock.patch.object(requests.Session, 'request') def test_timeout(self, get, baseapi): get.side_effect = requests.exceptions.Timeout with pytest.raises(requests.exceptions.Timeout): baseapi._query('nodes') @mock.patch.object(requests.Session, 'request') def test_connectionerror(self, get, baseapi): get.side_effect = requests.exceptions.ConnectionError with pytest.raises(requests.exceptions.ConnectionError): baseapi._query('nodes') @mock.patch.object(requests.Session, 'request') def test_httperror(self, get, baseapi): get.side_effect = requests.exceptions.HTTPError( response=requests.Response()) with pytest.raises(requests.exceptions.HTTPError): baseapi._query('nodes') def test_setting_headers(self, baseapi): httpretty.enable() stub_request('http://localhost:8080/v3/nodes') baseapi._query('nodes') # need to query some endpoint request_headers = dict(httpretty.last_request().headers) assert request_headers['Accept'] == 'application/json' assert request_headers['Content-Type'] == 'application/json' assert request_headers['Accept-Charset'] == 'utf-8' assert request_headers['Host'] == 'localhost:8080' assert httpretty.last_request().path == '/v3/nodes' httpretty.disable() httpretty.reset() def test_with_path(self, baseapi): httpretty.enable() stub_request('http://localhost:8080/v3/nodes/node1') baseapi._query('nodes', path='node1') assert httpretty.last_request().path == '/v3/nodes/node1' httpretty.disable() httpretty.reset() def test_with_query(self, baseapi): httpretty.enable() stub_request('http://localhost:8080/v3/nodes') baseapi._query('nodes', query='["certname", "=", "node1"]') assert httpretty.last_request().querystring == { 'query': ['["certname", "=", "node1"]']} httpretty.disable() httpretty.reset() def test_with_order(self, baseapi): httpretty.enable() stub_request('http://localhost:8080/v3/nodes') baseapi._query('nodes', order_by='ted') assert httpretty.last_request().querystring == { 'order-by': ['ted']} httpretty.disable() httpretty.reset() def test_with_limit(self, baseapi): httpretty.enable() stub_request('http://localhost:8080/v3/nodes') baseapi._query('nodes', limit=1) assert httpretty.last_request().querystring == { 'limit': ['1']} httpretty.disable() httpretty.reset() def test_with_include_total(self, baseapi): httpretty.enable() stub_request('http://localhost:8080/v3/nodes') baseapi._query('nodes', include_total=True) assert httpretty.last_request().querystring == { 'include-total': ['true']} httpretty.disable() httpretty.reset() def test_with_offset(self, baseapi): httpretty.enable() stub_request('http://localhost:8080/v3/nodes') baseapi._query('nodes', offset=1) assert httpretty.last_request().querystring == { 'offset': ['1']} httpretty.disable() httpretty.reset() def test_with_summarize_by(self, baseapi): httpretty.enable() stub_request('http://localhost:8080/v3/nodes') baseapi._query('nodes', summarize_by=1) assert httpretty.last_request().querystring == { 'summarize-by': ['1']} httpretty.disable() httpretty.reset() def test_with_count_by(self, baseapi): httpretty.enable() stub_request('http://localhost:8080/v3/nodes') baseapi._query('nodes', count_by=1) assert httpretty.last_request().querystring == { 'count-by': ['1']} httpretty.disable() httpretty.reset() def test_with_count_filter(self, baseapi): httpretty.enable() stub_request('http://localhost:8080/v3/nodes') baseapi._query('nodes', count_filter=1) assert httpretty.last_request().querystring == { 'count-filter': ['1']} httpretty.disable() httpretty.reset() def test_response_empty(self, baseapi): httpretty.enable() httpretty.register_uri(httpretty.GET, 'http://localhost:8080/v3/nodes', body=json.dumps(None)) with pytest.raises(pypuppetdb.errors.EmptyResponseError): baseapi._query('nodes') def test_response_x_records(self, baseapi): httpretty.enable() httpretty.register_uri(httpretty.GET, 'http://localhost:8080/v3/nodes', adding_headers={ 'X-Records': 256}, body='[]', ) baseapi._query('nodes', include_total=True) assert baseapi.total == 256 class TestAPIMethods(object): def test_nodes(self, baseapi): with pytest.raises(NotImplementedError): baseapi.nodes() def test_node(self, baseapi): with pytest.raises(NotImplementedError): baseapi.node() def test_facts(self, baseapi): with pytest.raises(NotImplementedError): baseapi.facts() def test_resources(self, baseapi): with pytest.raises(NotImplementedError): baseapi.resources() def test_metric(self, baseapi): httpretty.enable() stub_request('http://localhost:8080/v3/metrics/mbean/test') baseapi.metric('test') assert httpretty.last_request().path == '/v3/metrics/mbean/test' httpretty.disable() httpretty.reset() pypuppetdb-master/pypuppetdb/0000755000175000017500000000000012370425211015742 5ustar jonasjonaspypuppetdb-master/pypuppetdb/utils.py0000644000175000017500000000173012370425211017455 0ustar jonasjonasfrom __future__ import unicode_literals from __future__ import absolute_import import warnings import datetime # A UTC class, see: # http://docs.python.org/2/library/datetime.html#tzinfo-objects class UTC(datetime.tzinfo): """UTC""" def utcoffset(self, dt): return datetime.timedelta(0) def tzname(self, dt): return str('UTC') def dst(self, dt): return datetime.timedelta(0) def __repr__(self): return str('') def __str__(self): return str('UTC') def __unicode__(self): return 'UTC' def json_to_datetime(date): """Tranforms a JSON datetime string into a timezone aware datetime object with a UTC tzinfo object. :param date: The datetime representation. :type date: :obj:`string` :returns: A timezone aware datetime object. :rtype: :class:`datetime.datetime` """ return datetime.datetime.strptime(date, '%Y-%m-%dT%H:%M:%S.%fZ').replace( tzinfo=UTC()) pypuppetdb-master/pypuppetdb/api/0000755000175000017500000000000012370425211016513 5ustar jonasjonaspypuppetdb-master/pypuppetdb/api/v3.py0000644000175000017500000002300112370425211017411 0ustar jonasjonasfrom __future__ import unicode_literals from __future__ import absolute_import import logging from pypuppetdb.api import BaseAPI from pypuppetdb.utils import json_to_datetime from datetime import datetime, timedelta from pypuppetdb.types import ( Node, Fact, Resource, Report, Event, Catalog ) log = logging.getLogger(__name__) class API(BaseAPI): """The API object for version 3 of the PuppetDB API. This object contains all v3 specific methods and ways of doing things. :param \*\*kwargs: Rest of the keywoard arguments passed on to our parent\ :class:`~pypuppetdb.api.BaseAPI`. """ def __init__(self, *args, **kwargs): """Initialise the API object.""" super(API, self).__init__(api_version=3, **kwargs) log.debug('API initialised with {0}.'.format(kwargs)) def node(self, name): """Gets a single node from PuppetDB.""" nodes = self.nodes(name=name) return next(node for node in nodes) def nodes(self, name=None, query=None, unreported=2, with_status=False): """Query for nodes by either name or query. If both aren't provided this will return a list of all nodes. This method also fetches the nodes status and event counts of the latest report from puppetdb. :param name: (optional) :type name: :obj:`None` or :obj:`string` :param query: (optional) :type query: :obj:`None` or :obj:`string` :param with_status: (optional) include the node status in the\ returned nodes :type with_status: :bool: :param unreported: (optional) amount of hours when a node gets marked as unreported :type unreported: :obj:`None` or integer :returns: A generator yieling Nodes. :rtype: :class:`pypuppetdb.types.Node` """ nodes = self._query('nodes', path=name, query=query) # If we happen to only get one node back it # won't be inside a list so iterating over it # goes boom. Therefor we wrap a list around it. if type(nodes) == dict: nodes = [nodes, ] if with_status: latest_events = self._query( 'event-counts', query='["=","latest-report?",true]', summarize_by='certname') for node in nodes: node['unreported_time'] = None node['status'] = None if with_status: status = [s for s in latest_events if s['subject']['title'] == node['name']] # node status from events if with_status and status: node['events'] = status = status[0] if status['successes'] > 0: node['status'] = 'changed' if status['noops'] > 0: node['status'] = 'noop' if status['failures'] > 0: node['status'] = 'failed' else: if with_status: node['status'] = 'unchanged' node['events'] = None # node report age if with_status and node['report_timestamp'] is not None: try: last_report = json_to_datetime(node['report_timestamp']) last_report = last_report.replace(tzinfo=None) now = datetime.utcnow() unreported_border = now-timedelta(hours=unreported) if last_report < unreported_border: delta = (datetime.utcnow()-last_report) node['status'] = 'unreported' node['unreported_time'] = '{0}d {1}h {2}m'.format( delta.days, int(delta.seconds/3600), int((delta.seconds % 3600)/60) ) except AttributeError: node['status'] = 'unreported' if not node['report_timestamp'] and with_status: node['status'] = 'unreported' yield Node(self, node['name'], deactivated=node['deactivated'], report_timestamp=node['report_timestamp'], catalog_timestamp=node['catalog_timestamp'], facts_timestamp=node['facts_timestamp'], status=node['status'], events=node['events'], unreported_time=node['unreported_time'] ) def facts(self, name=None, value=None, query=None): """Query for facts limited by either name, value and/or query. This will yield a single Fact object at a time.""" log.debug('{0}, {1}, {2}'.format(name, value, query)) if name is not None and value is not None: path = '{0}/{1}'.format(name, value) elif name is not None and value is None: path = name elif name is None and value is None and query is not None: path = None else: log.debug("We want to query for all facts.") query = '' path = None facts = self._query('facts', path=path, query=query) for fact in facts: yield Fact( fact['certname'], fact['name'], fact['value'], ) def fact_names(self): """Get a list of all known facts.""" return self._query('fact-names') def resources(self, type_=None, title=None, query=None): """Query for resources limited by either type and/or title or query. This will yield a Resources object for every returned resource.""" path = None if type_ is not None: type_ = self._normalize_resource_type(type_) if title is not None: path = '{0}/{1}'.format(type_, title) elif title is None: path = type_ elif query is None: log.debug('Going to query for all resources. This is usually a ' 'bad idea as it might return enormous amounts of ' 'resources.') resources = self._query('resources', path=path, query=query) for resource in resources: yield Resource( resource['certname'], resource['title'], resource['type'], resource['tags'], resource['exported'], resource['file'], resource['line'], resource['parameters'], ) def reports(self, query): """Get reports for our infrastructure. Currently reports can only be filtered through a query which requests a specific certname. If not it will return all reports. This yields a Report object for every returned report.""" reports = self._query('reports', query=query) for report in reports: yield Report( report['certname'], report['hash'], report['start-time'], report['end-time'], report['receive-time'], report['configuration-version'], report['report-format'], report['puppet-version'], report['transaction-uuid'] ) def events(self, query, order_by=None, limit=None): """A report is made up of events. This allows to query for events based on the reprt hash. This yields an Event object for every returned event.""" events = self._query('events', query=query, order_by=order_by, limit=limit) for event in events: yield Event( event['certname'], event['status'], event['timestamp'], event['report'], event['resource-title'], event['property'], event['message'], event['new-value'], event['old-value'], event['resource-type'], ) def event_counts(self, query, summarize_by, count_by=None, count_filter=None): """Get event counts from puppetdb""" return self._query('event-counts', query=query, summarize_by=summarize_by, count_by=count_by, count_filter=count_filter) def aggregate_event_counts(self, query, summarize_by, count_by=None, count_filter=None): """Get event counts from puppetdb""" return self._query('aggregate-event-counts', query=query, summarize_by=summarize_by, count_by=count_by, count_filter=count_filter) def server_time(self): """Get the current time of the clock on the PuppetDB server""" return self._query('server-time')['server-time'] def current_version(self): """Get version information about the running PuppetDB server""" return self._query('version')['version'] def catalog(self, node): """Get the most recent catalog for a given node""" c = self._query('catalogs', path=node) return Catalog(c['data']['name'], c['data']['edges'], c['data']['resources'], c['data']['version'], c['data']['transaction-uuid']) pypuppetdb-master/pypuppetdb/api/v2.py0000644000175000017500000001006712370425211017420 0ustar jonasjonasfrom __future__ import unicode_literals from __future__ import absolute_import import logging from pypuppetdb.api import BaseAPI from pypuppetdb.types import ( Node, Fact, Resource, ) log = logging.getLogger(__name__) class API(BaseAPI): """The API object for version 2 of the PuppetDB API. This object contains all v2 specific methods and ways of doing things. :param \*\*kwargs: Rest of the keywoard arguments passed on to our parent\ :class:`~pypuppetdb.api.BaseAPI`. """ def __init__(self, *args, **kwargs): """Initialise the API object.""" super(API, self).__init__(api_version=2, **kwargs) log.debug('API initialised with {0}'.format(kwargs)) def node(self, name): """Gets a single node from PuppetDB.""" nodes = self.nodes(name=name) return next(node for node in nodes) def nodes(self, name=None, query=None): """Query for nodes by either name or query. If both aren't provided this will return a list of all nodes. :param name: (optional) :type name: :obj:`None` or :obj:`string` :param query: (optional) :type query: :obj:`None` or :obj:`string` :returns: A generator yieling Nodes. :rtype: :class:`pypuppetdb.types.Node` """ nodes = self._query('nodes', path=name, query=query) # If we happen to only get one node back it # won't be inside a list so iterating over it # goes boom. Therefor we wrap a list around it. if type(nodes) == dict: log.debug("Request returned a single node.") nodes = [nodes, ] for node in nodes: yield Node(self, node['name'], deactivated=node['deactivated'], report_timestamp=node['report_timestamp'], catalog_timestamp=node['catalog_timestamp'], facts_timestamp=node['facts_timestamp'], ) def facts(self, name=None, value=None, query=None): """Query for facts limited by either name, value and/or query. This will yield a single Fact object at a time.""" log.debug('{0}, {1}, {2}'.format(name, value, query)) if name is not None and value is not None: path = '{0}/{1}'.format(name, value) elif name is not None and value is None: path = name elif name is None and value is None and query is not None: path = None else: log.debug("We want to query for all facts.") query = '' path = None facts = self._query('facts', path=path, query=query) for fact in facts: yield Fact( fact['certname'], fact['name'], fact['value'], ) def fact_names(self): """Get a list of all known facts.""" return self._query('fact-names') def resources(self, type_=None, title=None, query=None): """Query for resources limited by either type and/or title or query. This will yield a Resources object for every returned resource.""" if type_ is not None: type_ = self._normalize_resource_type(type_) if title is not None: path = '{0}/{1}'.format(type_, title) elif title is None: path = type_ else: log.debug('Going to query for all resources. This is usually a ' 'bad idea as it might return enormous amounts of ' 'resources.') query = '' path = None resources = self._query('resources', path=path, query=query) for resource in resources: yield Resource( resource['certname'], resource['title'], resource['type'], resource['tags'], resource['exported'], resource['sourcefile'], resource['sourceline'], resource['parameters'], ) pypuppetdb-master/pypuppetdb/api/__init__.py0000644000175000017500000002707612370425211020640 0ustar jonasjonasfrom __future__ import unicode_literals from __future__ import absolute_import import logging import json import requests from pypuppetdb.errors import ( ImproperlyConfiguredError, EmptyResponseError, UnsupportedVersionError, APIError, ) log = logging.getLogger(__name__) API_VERSIONS = { 2: 'v2', 3: 'v3', } ENDPOINTS = { 2: { 'facts': 'facts', 'fact-names': 'fact-names', 'nodes': 'nodes', 'resources': 'resources', 'metrics': 'metrics', 'mbean': 'metrics/mbean', }, 3: { 'facts': 'facts', 'fact-names': 'fact-names', 'nodes': 'nodes', 'resources': 'resources', 'catalogs': 'catalogs', 'metrics': 'metrics', 'mbean': 'metrics/mbean', 'reports': 'reports', 'events': 'events', 'event-counts': 'event-counts', 'aggregate-event-counts': 'aggregate-event-counts', 'server-time': 'server-time', 'version': 'version', }, } ERROR_STRINGS = { 'timeout': 'Connection to PuppetDB timed out on', 'refused': 'Could not reach PuppetDB on', } class BaseAPI(object): """This is a Base or Abstract class and is not meant to be instantiated or used directly. The BaseAPI object defines a set of methods that can be reused across different versions of the PuppetDB API. If querying for a certain resource is done in an identical fashion across different versions it will be implemented here and should be overridden in their respective versions if they deviate. If :attr:`ssl` is set to `True` but either :attr:`ssl_key` or\ :attr:`ssl_cert` are `None` this will raise an error. When at initialisation :obj:`api_version` isn't found in\ :const:`API_VERSIONS` this will raise an error. :param api_version: Version of the API we're initialising. :type api_version: :obj:`int` :param host: (optional) Hostname or IP of PuppetDB. :type host: :obj:`string` :param port: (optional) Port on which to talk to PuppetDB. :type port: :obj:`int` :param ssl_verify: (optional) Verify PuppetDB server certificate. :type ssl_verify: :obj:`bool` :param ssl_key: (optional) Path to our client secret key. :type ssl_key: :obj:`None` or :obj:`string` representing a filesystem\ path. :param ssl_cert: (optional) Path to our client certificate. :type ssl_cert: :obj:`None` or :obj:`string` representing a filesystem\ path. :param timeout: (optional) Number of seconds to wait for a response. :type timeout: :obj:`int` :raises: :class:`~pypuppetdb.errors.ImproperlyConfiguredError` :raises: :class:`~pypuppetdb.errors.UnsupportedVersionError` """ def __init__(self, api_version, host='localhost', port=8080, ssl_verify=True, ssl_key=None, ssl_cert=None, timeout=10): """Initialises our BaseAPI object passing the parameters needed in order to be able to create the connection strings, set up SSL and timeouts and so forth.""" if api_version in API_VERSIONS: self.api_version = API_VERSIONS[api_version] else: raise UnsupportedVersionError self.host = host self.port = port self.ssl_verify = ssl_verify self.ssl_key = ssl_key self.ssl_cert = ssl_cert self.timeout = timeout self.endpoints = ENDPOINTS[api_version] self._session = requests.Session() self._session.headers = { 'content-type': 'application/json', 'accept': 'application/json', 'accept-charset': 'utf-8' } if self.ssl_key is not None and self.ssl_cert is not None: self.protocol = 'https' else: self.protocol = 'http' @property def version(self): """The version of the API we're querying against. :returns: Current API version. :rtype: :obj:`string`""" return self.api_version @property def base_url(self): """A base_url that will be used to construct the final URL we're going to query against. :returns: A URL of the form: ``proto://host:port``. :rtype: :obj:`string` """ return '{proto}://{host}:{port}'.format( proto=self.protocol, host=self.host, port=self.port, ) @property def total(self): """The total-count of the last request to PuppetDB if enabled as parameter in _query method :returns Number of total results :rtype :obj:`int` """ if self.last_total is not None: return int(self.last_total) def _normalize_resource_type(self, type_): """Normalizes the type passed to the api by capitalizing each part of the type. For example: sysctl::value -> Sysctl::Value user -> User """ return '::'.join([s.capitalize() for s in type_.split('::')]) def _url(self, endpoint, path=None): """The complete URL we will end up querying. Depending on the endpoint we pass in this will result in different URL's with different prefixes. :param endpoint: The PuppetDB API endpoint we want to query. :type endpoint: :obj:`string` :param path: An additional path if we don't wish to query the\ bare endpoint. :type path: :obj:`string` :returns: A URL constructed from :func:`base_url` with the\ apropraite API version/prefix and the rest of the path added\ to it. :rtype: :obj:`string` """ log.debug('_url called with endpoint: {0} and path: {1}'.format( endpoint, path)) if endpoint in self.endpoints: api_prefix = self.api_version endpoint = self.endpoints[endpoint] else: # If we reach this we're trying to query an endpoint that doesn't # exist. This shouldn't happen unless someone made a booboo. raise APIError url = '{base_url}/{api_prefix}/{endpoint}'.format( base_url=self.base_url, api_prefix=api_prefix, endpoint=endpoint, ) if path is not None: url = '{0}/{1}'.format(url, path) return url def _query(self, endpoint, path=None, query=None, order_by=None, limit=None, offset=None, include_total=False, summarize_by=None, count_by=None, count_filter=None): """This method actually querries PuppetDB. Provided an endpoint and an optional path and/or query it will fire a request at PuppetDB. If PuppetDB can be reached and answers within the timeout we'll decode the response and give it back or raise for the HTTP Status Code PuppetDB gave back. :param endpoint: The PuppetDB API endpoint we want to query. :type endpoint: :obj:`string` :param path: An additional path if we don't wish to query the\ bare endpoint. :type path: :obj:`string` :param query: (optional) A query to further narrow down the resultset. :type query: :obj:`string` :param order_by: (optional) Set the order parameters for the resultset. :type order_by: :obj:`string` :param limit: (optional) Tell PuppetDB to limit it's response to this\ number of objects. :type limit: :obj:`int` :param offset: (optional) Tell PuppetDB to start it's response from\ the given offset. This is useful for implementing pagination\ but is not supported just yet. :type offset: :obj:`string` :param include_total: (optional) Include the total number of results :type order_by: :obj:`bool` :param summarize_by: (optional) Specify what type of object you'd like\ to see counts at the event-counts and aggregate-event-counts \ endpoints :type summarize_by: :obj:`string` :param count_by: (optional) Specify what type of object is counted :type count_by: :obj:`string` :param count_filter: (optional) Specify a filter for the results :type count_filter: :obj:`string` :raises: :class:`~pypuppetdb.errors.EmptyResponseError` :returns: The decoded response from PuppetDB :rtype: :obj:`dict` or :obj:`list` """ log.debug('_query called with endpoint: {0}, path: {1}, query: {2}, ' 'limit: {3}, offset: {4}, summarize_by {5}, count_by {6}, ' 'count_filter: {7}'.format(endpoint, path, query, limit, offset, summarize_by, count_by, count_filter)) url = self._url(endpoint, path=path) payload = {} if query is not None: payload['query'] = query if order_by is not None: payload['order-by'] = order_by if limit is not None: payload['limit'] = limit if include_total is True: payload['include-total'] = json.dumps(include_total) if offset is not None: payload['offset'] = offset if summarize_by is not None: payload['summarize-by'] = summarize_by if count_by is not None: payload['count-by'] = count_by if count_filter is not None: payload['count-filter'] = count_filter if not (payload): payload = None try: r = self._session.get(url, params=payload, verify=self.ssl_verify, cert=(self.ssl_cert, self.ssl_key), timeout=self.timeout) r.raise_for_status() # get total number of results if requested with include-total # just a quick hack - needs improvement if 'X-Records' in r.headers: self.last_total = r.headers['X-Records'] else: self.last_total = None json_body = r.json() if json_body is not None: return json_body else: del json_body raise EmptyResponseError except requests.exceptions.Timeout: log.error("{0} {1}:{2} over {3}.".format(ERROR_STRINGS['timeout'], self.host, self.port, self.protocol.upper())) raise except requests.exceptions.ConnectionError: log.error("{0} {1}:{2} over {3}.".format(ERROR_STRINGS['refused'], self.host, self.port, self.protocol.upper())) raise except requests.exceptions.HTTPError as err: log.error("{0} {1}:{2} over {3}.".format(err.response.text, self.host, self.port, self.protocol.upper())) raise # Method stubs def nodes(self): raise NotImplementedError def node(self): raise NotImplementedError def facts(self): raise NotImplementedError def resources(self): raise NotImplementedError def metric(self, metric): """Query for a specific metrc. :param metric: The name of the metric we want. :type metric: :obj:`string` :returns: The return of :meth:`~pypuppetdb.api.BaseAPI._query`. """ return self._query('mbean', path=metric) pypuppetdb-master/pypuppetdb/__init__.py0000644000175000017500000000673112370425211020062 0ustar jonasjonasfrom __future__ import unicode_literals from __future__ import absolute_import """ pypuppetdb PuppetDB API library ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pypuppetdb is a library to work with PuppetDB's REST API. It provides a way to query PuppetDB and a set of additional methods and objects to make working with PuppetDB's API and the responses easier: >>> from pypuppetdb import connect >>> db = connect() >>> nodes = db.nodes() >>> print(nodes) >>> for node in nodes: >>> print(node) host1 host2 ... This will return a generator object yielding Node objects for every returned node from PuppetDB. To query a single node the singular db.node() can be used: >>> node = db.node('hostname') >>> print(node) hostname The Node objects are a bit more special in that they can query for facts and resources themselves. Using those methods from a node object will automatically add a query to the request scoping the request to the node. >>> node = db.node('hostname') >>> print node.fact('osfamily') osfamily/hostname We can also query for facts: >>> facts = db.facts('osfamily') >>> print(facts) >> for fact in facts: >>> print(fact) osfamily/host1 osfamily/host2 That querries PuppetDB for the 'osfamily' fact and will yield Fact objects, one per node this fact is found on. >>> resources = db.resources('file') Will return a generator object containing all file resources you're managing across your infrastructure. This is probably a bad idea if you have a big number of nodes as the response will be huge. """ import logging from pypuppetdb.api import v2 from pypuppetdb.api import v3 from pypuppetdb.errors import UnsupportedVersionError try: # Python 2.7+ from logging import NullHandler except ImportError: # pragma: notest class NullHandler(logging.Handler): def emit(self, record): pass logging.getLogger(__name__).addHandler(NullHandler()) def connect(api_version=3, host='localhost', port=8080, ssl_verify=False, ssl_key=None, ssl_cert=None, timeout=10): """Connect with PuppetDB. This will return an object allowing you to query the API through its methods. :param api_version: Version of the API we're initialising. :type api_version: :obj:`int` :param host: (optional) Hostname or IP of PuppetDB. :type host: :obj:`string` :param port: (optional) Port on which to talk to PuppetDB. :type port: :obj:`int` :param ssl: (optional) Talk with PuppetDB over SSL. :type ssl: :obj:`bool` :param ssl_key: (optional) Path to our client secret key. :type ssl_key: :obj:`None` or :obj:`string` representing a filesystem\ path. :param ssl_cert: (optional) Path to our client certificate. :type ssl_cert: :obj:`None` or :obj:`string` representing a filesystem\ path. :param timeout: (optional) Number of seconds to wait for a response. :type timeout: :obj:`int` :raises: :class:`~pypuppetdb.errors.UnsupportedVersionError` """ if api_version == 3: return v3.API(host=host, port=port, timeout=timeout, ssl_verify=ssl_verify, ssl_key=ssl_key, ssl_cert=ssl_cert) if api_version == 2: return v2.API(host=host, port=port, timeout=timeout, ssl_verify=ssl_verify, ssl_key=ssl_key, ssl_cert=ssl_cert) else: raise UnsupportedVersionError pypuppetdb-master/pypuppetdb/errors.py0000644000175000017500000000152712370425211017635 0ustar jonasjonasclass APIError(Exception): """Our base exception the other errors inherit from.""" pass class ImproperlyConfiguredError(APIError): """This exception is thrown when the API is initialised and it detects incompatbile configuration such as SSL turned on but no certificates provided.""" pass class EmptyResponseError(APIError): """Will be thrown when we did recieve a response but the response is empty.""" pass class UnsupportedVersionError(APIError): """Triggers when using the :func:`connect` function and providing it with an unknown API version.""" pass class DoesNotComputeError(APIError): """This error will be thrown when a function is called with an incompatible set of optional parameters. This is the 'you are being a naughty developer, go read the docs' error. """ pass pypuppetdb-master/pypuppetdb/package.py0000644000175000017500000000052612370425211017712 0ustar jonasjonasfrom __future__ import unicode_literals __title__ = 'pypuppetdb' __version_info__ = (0, 1, 1) # notest __version__ = '.'.join("{0}".format(num) for num in __version_info__) # notest __author__ = 'Daniele Sluijters' __license__ = 'Apache License 2.0' __year__ = '2013, 2014' __copyright__ = 'Copyright {0} {1}'.format(__year__, __author__) pypuppetdb-master/pypuppetdb/types.py0000644000175000017500000003766612370425211017502 0ustar jonasjonasfrom __future__ import unicode_literals from __future__ import absolute_import import logging from pypuppetdb.utils import json_to_datetime log = logging.getLogger(__name__) class Event(object): """This object represents an event. :param node: The hostname of the node this event fired on. :param status: The status for the event. :param timestamp: A timestamp of when this event occured. :param hash\_: The hash of this event. :param title: The resource title this event was fired for. :param property\_: The property of the resource this event was fired for. :param message: A message associated with this event. :param new_value: The new value/state of the resource. :param old_value: The old value/state of the resource. :param type\_: The type of the resource this event fired for. :ivar status: A :obj:`string` of this event's status. :ivar failed: The :obj:`bool` equivalent of `status`. :ivar timestamp: A :obj:`datetime.datetime` of when this event happend. :ivar node: The hostname of the machine this event\ occured on. :ivar hash\_: The hash of this event. :ivar item: :obj:`dict` with information about the item/resource this\ event was triggered for. """ def __init__(self, node, status, timestamp, hash_, title, property_, message, new_value, old_value, type_): self.node = node self.status = status if self.status == 'failure': self.failed = True else: self.failed = False self.timestamp = json_to_datetime(timestamp) self.hash_ = hash_ self.item = {} self.item['title'] = title self.item['type'] = type_ self.item['property'] = property_ self.item['message'] = message self.item['old'] = old_value self.item['new'] = new_value self.__string = '{0}[{1}]/{2}'.format(self.item['type'], self.item['title'], self.hash_) def __repr__(self): return str('Event: {0}'.format(self.__string)) def __str__(self): return str('{0}').format(self.__string) def __unicode__(self): return self.__string class Report(object): """This object represents a report. :param node: The hostname of the node this report originated on. :param hash\_: A string uniquely identifying this report. :param start: The start time of the agent run. :type start: :obj:`string` formatted as ``%Y-%m-%dT%H:%M:%S.%fZ`` :param end: The time the agent finished its run. :type end: :obj:`string` formatted as ``%Y-%m-%dT%H:%M:%S.%fZ`` :param received: The time PuppetDB received the report. :type received: :obj:`string` formatted as ``%Y-%m-%dT%H:%M:%S.%fZ`` :param version: The catalog / configuration version. :type version: :obj:`string` :param format\_: The catalog format version. :type format\_: :obj:`int` :param agent_version: The Puppet agent version. :type agent_version: :obj:`string` :param transaction: The UUID of this transaction. :type transaction: :obj:`string` :ivar node: The hostname this report originated from. :ivar hash\_: Unique identifier of this report. :ivar start: :obj:`datetime.datetime` when the Puppet agent run started. :ivar end: :obj:`datetime.datetime` when the Puppet agent run ended. :ivar received: :obj:`datetime.datetime` when the report finished\ uploading. :ivar version: :obj:`string` catalog configuration version. :ivar format\_: :obj:`int` catalog format version. :ivar agent_version: :obj:`string` Puppet Agent version. :ivar run_time: :obj:`datetime.timedelta` of **end** - **start**. :ivar transaction: UUID identifying this transaction. """ def __init__(self, node, hash_, start, end, received, version, format_, agent_version, transaction): self.node = node self.hash_ = hash_ self.start = json_to_datetime(start) self.end = json_to_datetime(end) self.received = json_to_datetime(received) self.version = version self.format_ = format_ self.agent_version = agent_version self.run_time = self.end - self.start self.transaction = transaction self.__string = '{0}'.format(self.hash_) def __repr__(self): return str('Report: {0}'.format(self.__string)) def __str__(self): return str('{0}').format(self.__string) def __unicode__(self): return self.__string class Fact(object): """his object represents a fact. :param node: The hostname this fact was collected from. :param name: The fact's name, such as 'osfamily' :param value: The fact's value, such as 'Debian' :ivar node: :obj:`string` holding the hostname. :ivar name: :obj:`string` holding the fact's name. :ivar value: :obj:`string` holding the fact's value. """ def __init__(self, node, name, value): self.node = node self.name = name self.value = value self.__string = '{0}/{1}'.format(self.name, self.node) def __repr__(self): return str('Fact: {0}'.format(self.__string)) def __str__(self): return str('{0}').format(self.__string) def __unicode__(self): return self.__string class Resource(object): """This object represents a resource. :param node: The hostname this resource is located on. :param name: The name of the resource in the Puppet manifest. :param type\_: Type of the Puppet resource. :param tags: Tags associated with this resource. :type tags: :obj:`list` :param exported: If it's an exported resource. :type exported: :obj:`bool` :param sourcefile: The Puppet manifest this resource is declared in. :param sourceline: The line this resource is declared at. :param parameters: The parameters this resource has been declared with. :type parameters: :obj:`dict` :ivar node: The hostname this resources is located on. :ivar name: The name of the resource in the Puppet manifest. :ivar type\_: The type of Puppet resource. :ivar exported: :obj:`bool` if the resource is exported. :ivar sourcefile: The Puppet manifest this resource is declared in. :ivar sourceline: The line this resource is declared at. :ivar parameters: :obj:`dict` with key:value pairs of parameters. :ivar relationships: :obj:`list` Contains all relationships to other\ resources """ def __init__(self, node, name, type_, tags, exported, sourcefile, sourceline, parameters={}): self.node = node self.name = name self.type_ = type_ self.tags = tags self.exported = exported self.sourcefile = sourcefile self.sourceline = sourceline self.parameters = parameters self.relationships = [] self.__string = '{0}[{1}]'.format(self.type_, self.name) def __repr__(self): return str('').format(self.__string) def __str__(self): return str('{0}').format(self.__string) def __unicode__(self): return self.__string class Node(object): """This object represents a node. It additionally has some helper methods so that you can query for resources or facts directly from the node scope. :param api: API object. :param name: Hostname of this node. :param deactivated: (default `None`) Time this node was deactivated at. :type deactivated: :obj:`string` formatted as ``%Y-%m-%dT%H:%M:%S.%fZ`` :param report_timestamp: (default `None`) Time of the last report. :type report_timestamp: :obj:`string` formatted as\ ``%Y-%m-%dT%H:%M:%S.%fZ`` :param catalog_timestamp: (default `None`) Time the last time a catalog\ was compiled. :type catalog_timestamp: :obj:`string` formatted as\ ``%Y-%m-%dT%H:%M:%S.%fZ`` :param facts_timestamp: (default `None`) Time the last time facts were\ collected. :type facts_timestamp: :obj:`string` formatted as\ ``%Y-%m-%dT%H:%M:%S.%fZ`` :param status: (default `None`) Status of the node\ changed | unchanged | unreported | failed :type status: :obj:`string` :param events: (default `None`) Counted events from latest Report :type events: :obj:`dict` :param unreported_time: (default `None`) Time since last report :type unreported_time: :obj:`string` :ivar name: Hostname of this node. :ivar deactivated: :obj:`datetime.datetime` when this host was\ deactivated or `False`. :ivar report_timestamp: :obj:`datetime.datetime` when the last run\ occured or `None`. :ivar catalog_timestamp: :obj:`datetime.datetime` last time a catalog was\ compiled or `None`. :ivar facts_timestamp: :obj:`datetime.datetime` last time when facts were\ collected or `None`. """ def __init__(self, api, name, deactivated=None, report_timestamp=None, catalog_timestamp=None, facts_timestamp=None, status=None, events=None, unreported_time=None): self.name = name self.status = status self.events = events self.unreported_time = unreported_time if deactivated is not None: self.deactivated = json_to_datetime(deactivated) else: self.deactivated = False if report_timestamp is not None: self.report_timestamp = json_to_datetime(report_timestamp) else: self.report_timestamp = report_timestamp if facts_timestamp is not None: self.facts_timestamp = json_to_datetime(facts_timestamp) else: self.facts_timestamp = facts_timestamp if catalog_timestamp is not None: self.catalog_timestamp = json_to_datetime(catalog_timestamp) else: self.catalog_timestamp = catalog_timestamp self.__api = api self.__query_scope = '["=", "certname", "{0}"]'.format(self.name) self.__string = self.name def __repr__(self): return str('').format(self.__string) def __str__(self): return str('{0}').format(self.__string) def __unicode__(self): return self.__string def facts(self): """Get all facts of this node.""" return self.__api.facts(query=self.__query_scope) def fact(self, name): """Get a single fact from this node.""" facts = self.__api.facts(name=name, query=self.__query_scope) return next(fact for fact in facts) def resources(self, type_=None, title=None): """Get all resources of this node or all resources of the specified type.""" if type_ is None: resources = self.__api.resources(query=self.__query_scope) elif type_ is not None and title is None: resources = self.__api.resources(type_=type_, query=self.__query_scope) else: resources = self.__api.resources(type_=type_, title=title, query=self.__query_scope) return resources def resource(self, type_, title): """Get a resource matching the supplied type and title.""" resources = self.__api.resources(type_=type_, title=title, query=self.__query_scope) return next(resource for resource in resources) def reports(self): """Get all reports for this node.""" return self.__api.reports(self.__query_scope) class Catalog(object): """ This object represents a compiled catalog from puppet. It contains Resource and Edge object that represent the dependency graph. :param node: Name of the host :type edges: :obj:`string` :param edges: Edges returned from Catalog data :type edges: :obj:`list` containing :obj:`dict` with Edge information :param resources: Resources returned from Catalog data :type resources: :obj:`list` containing :obj:`dict` with Resources :param version: Catalog version from Puppet (unique for each node) :type version: :obj:`string` :param transaction_uuid: A string used to match the catalog with the corresponding report that was issued during the same puppet run :type transaction_uuid: :obj:`string` :ivar node: :obj:`string` Name of the host :ivar version: :obj:`string` Catalog version from Puppet (unique for each node) :ivar transaction_uuid: :obj:`string` used to match the catalog with corresponding report :ivar edges: :obj:`list` of :obj:`Edge` The source Resource object\ of the relationship :ivar resources: :obj:`dict` of :obj:`Resource` The source Resource\ object of the relationship """ def __init__(self, node, edges, resources, version, transaction_uuid): self.node = node self.version = version self.transaction_uuid = transaction_uuid self.resources = dict() for resource in resources: if 'file' not in resource: resource['file'] = None if 'line' not in resource: resource['line'] = None identifier = resource['type']+'['+resource['title']+']' res = Resource(node, resource['title'], resource['type'], resource['tags'], resource['exported'], resource['file'], resource['line'], resource['parameters']) self.resources[identifier] = res self.edges = [] for edge in edges: identifier_source = edge['source']['type'] + \ '[' + edge['source']['title'] + ']' identifier_target = edge['target']['type'] + \ '[' + edge['target']['title'] + ']' e = Edge(self.resources[identifier_source], self.resources[identifier_target], edge['relationship']) self.edges.append(e) self.resources[identifier_source].relationships.append(e) self.resources[identifier_target].relationships.append(e) self.__string = '{0}/{1}'.format(self.node, self.transaction_uuid) def __repr__(self): return str('').format(self.__string) def __str__(self): return str('{0}').format(self.__string) def __unicode__(self): return self.__string def get_resources(self): return self.resources.itervalues() def get_resource(self, resource_type, resource_title): identifier = resource_type + \ '[' + resource_title + ']' return self.resources[identifier] def get_edges(self): return iter(self.edges) class Edge(object): """ This object represents the connection between two Resource objects :param source: The source Resource object of the relationship :type source: :obj:`Resource` :param target: The target Resource object of the relationship :type target: :obj:`Resource` :param relaptionship: Name of the Puppet Ressource Relationship :type relationship: :obj:`string` :ivar source: :obj:`Resource` The source Resource object :ivar target: :obj:`Resource` The target Resource object :ivar relationship: :obj:`string` Name of the Puppet Resource relationship """ def __init__(self, source, target, relationship): self.source = source self.target = target self.relationship = relationship self.__string = '{0} - {1} - {2}'.format(self.source, self.relationship, self.target) def __repr__(self): return str('').format(self.__string) def __str__(self): return str('{0}').format(self.__string) def __unicode__(self): return self.__string