pax_global_header00006660000000000000000000000064126336254550014525gustar00rootroot0000000000000052 comment=d29eabf0d798aadfcf7cb881d3f9ad403c99542b python-etcd-0.4.3/000077500000000000000000000000001263362545500137675ustar00rootroot00000000000000python-etcd-0.4.3/.gitignore000066400000000000000000000001611263362545500157550ustar00rootroot00000000000000*.pyc .installed.cfg bin develop-eggs eggs .eggs .idea *.egg-info tmp build dist docs .coverage python-etcd-0.4.3/.travis.yml000066400000000000000000000007231263362545500161020ustar00rootroot00000000000000language: python python: - "2.7" - "3.5" before_install: - ./build_etcd.sh v2.2.0 - pip install --upgrade setuptools # command to install dependencies install: - pip install coveralls - pip install coverage - python bootstrap.py - bin/buildout # command to run tests script: PATH=$PATH:./etcd/bin coverage run --source=src/etcd --omit="src/etcd/tests/*" bin/test after_success: coveralls # Add env var to detect it during build env: TRAVIS=True python-etcd-0.4.3/AUTHORS000066400000000000000000000010341263362545500150350ustar00rootroot00000000000000Maintainers: ----------- Jose Plana (jplana) Giuseppe Lavagetto (lavagetto) Contributors: ------------ Aleksandar Veselinovic Alex Chan Alex Ianchici Bartlomiej Biernacki Bradley Cicenas Christoph Heer Hogenmiller Jimmy Zelinskie Jim Rollenhagen John Kristensen Joshua Conner Matthias Urlichs Michal Witkowski Nick Bartos Peter Wagner Roberto Aguilar Roy Smith Ryan Fowler Samuel Marks Sergio Castaño Arteaga Shaun Crampton Sigmund Augdal Simeon Visser Simon Gomizelj SkyLothar Spike Curtis Tomas Kral Tom Denham WillPlatnick WooParadog python-etcd-0.4.3/LICENSE.txt000066400000000000000000000023301263362545500156100ustar00rootroot00000000000000The MIT License (MIT) Copyright (c) 2013,2014,2015 Jose Plana Mario, Giuseppe Lavagetto Modifications, Copyright (c) 2015 Metaswitch Networks Limited Modifications, Copyright (c) 2015 The Wikimedia Foundation, Inc. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. python-etcd-0.4.3/MANIFEST.in000066400000000000000000000000441263362545500155230ustar00rootroot00000000000000include README.rst include NEWS.txt python-etcd-0.4.3/NEWS.txt000066400000000000000000000055071263362545500153130ustar00rootroot00000000000000News ==== 0.4.3 ----- *Release date: 14-Dec-2015* * Fix check for parameters in case of connection error * Python 3.5 compatibility and general python3 cleanups * Added authentication and module for managing ACLs * Added srv record-based DNS discovery * Fixed (again) logging of cluster id changes * Fixed leader lookup * Properly retry request on exception * Client: clean up open connections when deleting 0.4.2 ----- *Release date: 8-Oct-2015* * Fixed lock documentation * Fixed lock sequences due to etcd 2.2 change * Better exception management during response processing * Fixed logging of cluster ID changes * Fixed subtree results * Do not check cluster ID if etcd responses don't contain the ID * Added a cause to EtcdConnectionFailed 0.4.1 ----- *Release date: 1-Aug-2015* * Added client-side leader election * Added stats endpoints * Added logging * Better exception handling * Check for cluster ID on each request * Added etcd.Client.members and fixed etcd.Client.leader * Removed locking and election etcd support * Allow the use of etcd proxies with reconnections * Implement pop: Remove key from etc and return the corresponding value. * Eternal watcher can be now recursive * Fix etcd.Client machines * Do not send parameters with `None` value to etcd * Support ttl=0 in write. * Moved pyOpenSSL into test requirements. * Always set certificate information so redirects from http to https work. 0.3.3 ----- *Release date: 12-Apr-2015* * Forward leaves_only value in get_subtree() recursive calls * Fix README prevExists->prevExist * Added configurable version_prefix * Added support for recursive watch * Better error handling support (more detailed exceptions) * Fixed some unreliable tests 0.3.2 ----- *Release date: 4-Aug-2014* * Fixed generated documentation version. 0.3.1 ----- *Release date: 4-Aug-2014* * Added consisten read option * Fixed timeout parameter in read() * Added atomic delete parameter support * Fixed delete behaviour * Added update method that allows atomic updated on results * Fixed checks on write() * Added leaves generator to EtcdResult and get_subtree for recursive fetch * Added etcd_index to EtcdResult * Changed ethernal -> eternal * Updated urllib3 & pyOpenSSL libraries * Several performance fixes * Better parsing of etcd_index and raft_index * Removed duplicated tests * Added several integration and unit tests * Use etcd v0.3.0 in travis * Execute test using `python setup.py test` and nose 0.3.0 ----- *Release date: 18-Jan-2014* * API v2 support * Python 3.3 compatibility 0.2.1 ----- *Release data: 30-Nov-2013* * SSL support * Added support for subdirectories in results. * Improve test * Added support for reconnections, allowing death node tolerance. 0.2.0 ----- *Release date: 30-Sep-2013* * Allow fetching of multiple keys (sub-nodes) 0.1 --- *Release date: 18-Sep-2013* * Initial release python-etcd-0.4.3/README.rst000066400000000000000000000136501263362545500154630ustar00rootroot00000000000000python-etcd documentation ========================= A python client for Etcd https://github.com/coreos/etcd Official documentation: http://python-etcd.readthedocs.org/ .. image:: https://travis-ci.org/jplana/python-etcd.png?branch=master :target: https://travis-ci.org/jplana/python-etcd .. image:: https://coveralls.io/repos/jplana/python-etcd/badge.svg?branch=master&service=github :target: https://coveralls.io/github/jplana/python-etcd?branch=master Installation ------------ Pre-requirements ~~~~~~~~~~~~~~~~ Install etcd (2.0.1 or later). This version of python-etcd will only work correctly with the etcd version 2.0.x or later. If you are running an older version of etcd, please use python-etcd 0.3.3 or earlier. This client is known to work with python 2.7 and with python 3.3 or above. It is not tested or expected to work in more outdated versions of python. From source ~~~~~~~~~~~ .. code:: bash $ python setup.py install Usage ----- The basic methods of the client have changed compared to previous versions, to reflect the new API structure; however a compatibility layer has been maintained so that you don't necessarily need to rewrite all your existing code. Create a client object ~~~~~~~~~~~~~~~~~~~~~~ .. code:: python import etcd client = etcd.Client() # this will create a client against etcd server running on localhost on port 4001 client = etcd.Client(port=4002) client = etcd.Client(host='127.0.0.1', port=4003) client = etcd.Client(host='127.0.0.1', port=4003, allow_redirect=False) # wont let you run sensitive commands on non-leader machines, default is true # If you have defined a SRV record for _etcd._tcp.example.com pointing to the clients client = etcd.Client(srv_domain='example.com', protocol="https") # create a client against https://api.example.com:443/etcd client = etcd.Client(host='api.example.com', protocol='https', port=443, version_prefix='/etcd') Write a key ~~~~~~~~~ .. code:: python client.write('/nodes/n1', 1) # with ttl client.write('/nodes/n2', 2, ttl=4) # sets the ttl to 4 seconds client.set('/nodes/n2', 1) # Equivalent, for compatibility reasons. Read a key ~~~~~~~~~ .. code:: python client.read('/nodes/n2').value client.read('/nodes', recursive = True) #get all the values of a directory, recursively. client.get('/nodes/n2').value Delete a key ~~~~~~~~~~~~ .. code:: python client.delete('/nodes/n1') Atomic Compare and Swap ~~~~~~~~~~~~ .. code:: python client.write('/nodes/n2', 2, prevValue = 4) # will set /nodes/n2 's value to 2 only if its previous value was 4 and client.write('/nodes/n2', 2, prevExist = False) # will set /nodes/n2 's value to 2 only if the key did not exist before client.write('/nodes/n2', 2, prevIndex = 30) # will set /nodes/n2 's value to 2 only if the key was last modified at index 30 client.test_and_set('/nodes/n2', 2, 4) #equivalent to client.write('/nodes/n2', 2, prevValue = 4) You can also atomically update a result: .. code:: python result = client.read('/foo') print(result.value) # bar result.value += u'bar' updated = client.update(result) # if any other client wrote '/foo' in the meantime this will fail print(updated.value) # barbar Watch a key ~~~~~~~~~~~ .. code:: python client.read('/nodes/n1', wait = True) # will wait till the key is changed, and return once its changed client.read('/nodes/n1', wait = True, timeout=30) # will wait till the key is changed, and return once its changed, or exit with an exception after 30 seconds. client.read('/nodes/n1', wait = True, waitIndex = 10) # get all changes on this key starting from index 10 client.watch('/nodes/n1') #equivalent to client.read('/nodes/n1', wait = True) client.watch('/nodes/n1', index = 10) Locking module ~~~~~~~~~~~~~~ .. code:: python # Initialize the lock object: # NOTE: this does not acquire a lock yet client = etcd.Client() lock = etcd.Lock(client, 'my_lock_name') # Use the lock object: lock.acquire(blocking=True, # will block until the lock is acquired lock_ttl=None) # lock will live until we release it lock.is_acquired() # lock.acquire(lock_ttl=60) # renew a lock lock.release() # release an existing lock lock.is_acquired() # False # The lock object may also be used as a context manager: client = etcd.Client() with etcd.Lock(client, 'customer1') as my_lock: do_stuff() my_lock.is_acquired() # True my_lock.acquire(lock_ttl = 60) my_lock.is_acquired() # False Get machines in the cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: python client.machines Get leader of the cluster ~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: python client.leader Generate a sequential key in a directory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: python x = client.write("/dir/name", "value", append=True) print("generated key: " + x.key) print("stored value: " + x.value) List contents of a directory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: python #stick a couple values in the directory client.write("/dir/name", "value1", append=True) client.write("/dir/name", "value2", append=True) directory = client.get("/dir/name") # loop through directory children for result in directory.children: print(result.key + ": " + result.value) # or just get the first child value print(directory.children.next().value) Development setup ----------------- To create a buildout, .. code:: bash $ python bootstrap.py $ bin/buildout to test you should have etcd available in your system path: .. code:: bash $ bin/test to generate documentation, .. code:: bash $ cd docs $ make Release HOWTO ------------- To make a release 1) Update release date/version in NEWS.txt and setup.py 2) Run 'python setup.py sdist' 3) Test the generated source distribution in dist/ 4) Upload to PyPI: 'python setup.py sdist register upload' python-etcd-0.4.3/bootstrap.py000066400000000000000000000130661263362545500163640ustar00rootroot00000000000000############################################################################## # # Copyright (c) 2006 Zope Foundation and Contributors. # All Rights Reserved. # # This software is subject to the provisions of the Zope Public License, # Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution. # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS # FOR A PARTICULAR PURPOSE. # ############################################################################## """Bootstrap a buildout-based project Simply run this script in a directory containing a buildout.cfg. The script accepts buildout command-line options, so you can use the -c option to specify an alternate configuration file. """ import os import shutil import sys import tempfile from optparse import OptionParser tmpeggs = tempfile.mkdtemp() usage = '''\ [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options] Bootstraps a buildout-based project. Simply run this script in a directory containing a buildout.cfg, using the Python that you want bin/buildout to use. Note that by using --find-links to point to local resources, you can keep this script from going over the network. ''' parser = OptionParser(usage=usage) parser.add_option("-v", "--version", help="use a specific zc.buildout version") parser.add_option("-t", "--accept-buildout-test-releases", dest='accept_buildout_test_releases', action="store_true", default=False, help=("Normally, if you do not specify a --version, the " "bootstrap script and buildout gets the newest " "*final* versions of zc.buildout and its recipes and " "extensions for you. If you use this flag, " "bootstrap and buildout will get the newest releases " "even if they are alphas or betas.")) parser.add_option("-c", "--config-file", help=("Specify the path to the buildout configuration " "file to be used.")) parser.add_option("-f", "--find-links", help=("Specify a URL to search for buildout releases")) options, args = parser.parse_args() ###################################################################### # load/install setuptools to_reload = False try: import pkg_resources import setuptools except ImportError: ez = {} try: from urllib.request import urlopen except ImportError: from urllib2 import urlopen # XXX use a more permanent ez_setup.py URL when available. exec(urlopen('https://bitbucket.org/pypa/setuptools/raw/0.7.2/ez_setup.py' ).read(), ez) setup_args = dict(to_dir=tmpeggs, download_delay=0) ez['use_setuptools'](**setup_args) if to_reload: reload(pkg_resources) import pkg_resources # This does not (always?) update the default working set. We will # do it. for path in sys.path: if path not in pkg_resources.working_set.entries: pkg_resources.working_set.add_entry(path) ###################################################################### # Install buildout ws = pkg_resources.working_set cmd = [sys.executable, '-c', 'from setuptools.command.easy_install import main; main()', '-mZqNxd', tmpeggs] find_links = os.environ.get( 'bootstrap-testing-find-links', options.find_links or ('http://downloads.buildout.org/' if options.accept_buildout_test_releases else None) ) if find_links: cmd.extend(['-f', find_links]) setuptools_path = ws.find( pkg_resources.Requirement.parse('setuptools')).location requirement = 'zc.buildout' version = options.version if version is None and not options.accept_buildout_test_releases: # Figure out the most recent final version of zc.buildout. import setuptools.package_index _final_parts = '*final-', '*final' def _final_version(parsed_version): for part in parsed_version: if (part[:1] == '*') and (part not in _final_parts): return False return True index = setuptools.package_index.PackageIndex( search_path=[setuptools_path]) if find_links: index.add_find_links((find_links,)) req = pkg_resources.Requirement.parse(requirement) if index.obtain(req) is not None: best = [] bestv = None for dist in index[req.project_name]: distv = dist.parsed_version if _final_version(distv): if bestv is None or distv > bestv: best = [dist] bestv = distv elif distv == bestv: best.append(dist) if best: best.sort() version = best[-1].version if version: requirement = '=='.join((requirement, version)) cmd.append(requirement) import subprocess if subprocess.call(cmd, env=dict(os.environ, PYTHONPATH=setuptools_path)) != 0: raise Exception( "Failed to execute command:\n%s", repr(cmd)[1:-1]) ###################################################################### # Import and run buildout ws.add_entry(tmpeggs) ws.require(requirement) import zc.buildout.buildout if not [a for a in args if '=' not in a]: args.append('bootstrap') # if -c was provided, we push it back into args for buildout' main function if options.config_file is not None: args[0:0] = ['-c', options.config_file] zc.buildout.buildout.main(args) shutil.rmtree(tmpeggs) python-etcd-0.4.3/build_etcd.sh000077500000000000000000000006061263362545500164260ustar00rootroot00000000000000#!/bin/sh if [ $# -gt 0 ] then ETCD_VERSION="$1"; else ETCD_VERSION="master"; fi echo "Using ETCD version $ETCD_VERSION" git clone https://github.com/coreos/etcd.git cd etcd git checkout $ETCD_VERSION ./build ${TRAVIS:?"This is not a Travis build. All Done"} #Temporal solution to travis issue #155 sudo rm -rf /dev/shm && sudo ln -s /run/shm /dev/shm echo "All Done" python-etcd-0.4.3/buildout.cfg000066400000000000000000000012531263362545500163000ustar00rootroot00000000000000[buildout] parts = python sphinxbuilder test coverage develop = . eggs = urllib3==1.7.1 pyOpenSSL==0.13.1 ${deps:extraeggs} [python] recipe = zc.recipe.egg interpreter = python eggs = ${buildout:eggs} [test] recipe = pbp.recipe.noserunner eggs = ${python:eggs} mock [coverage] recipe = pbp.recipe.noserunner eggs = ${test:eggs} coverage defaults = --with-coverage --cover-package=etcd [sphinxbuilder] recipe = collective.recipe.sphinxbuilder source = ${buildout:directory}/docs-source build = ${buildout:directory}/docs [deps:python2] extraeggs = dnspython==1.12.0 [deps:python3] extraeggs = dnspython3==1.12.0 python-etcd-0.4.3/docs-source/000077500000000000000000000000001263362545500162155ustar00rootroot00000000000000python-etcd-0.4.3/docs-source/api.rst000066400000000000000000000003761263362545500175260ustar00rootroot00000000000000API Documentation ========================= .. automodule:: etcd :members: .. autoclass:: Client :special-members: :members: :exclude-members: __weakref__ .. autoclass:: Lock :special-members: :members: :exclude-members: __weakref__ python-etcd-0.4.3/docs-source/conf.py000066400000000000000000000174311263362545500175220ustar00rootroot00000000000000# -*- coding: utf-8 -*- import sys, os class Mock(object): def __init__(self, *args, **kwargs): pass def __call__(self, *args, **kwargs): return Mock() @classmethod def __getattr__(cls, name): if name in ('__file__', '__path__'): return '/dev/null' elif name[0] == name[0].upper(): mockType = type(name, (), {}) mockType.__module__ = __name__ return mockType else: return Mock() MOCK_MODULES = ['urllib3'] for mod_name in MOCK_MODULES: sys.modules[mod_name] = Mock() # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. sys.path.insert(0, os.path.abspath('../src')) # -- General configuration ----------------------------------------------------- # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = ['sphinx.ext.autodoc'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = u'python-etcd' copyright = u'2013-2015 Jose Plana, Giuseppe Lavagetto' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = '0.4' # The full version, including alpha/beta/rc tags. release = '0.4.3' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build'] # The reST default role (used for this markup: `text`) to use for all documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # -- Options for HTML output --------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'sphinxdoc' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'python-etcddoc' # -- Options for LaTeX output -------------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ ('index', 'python-etcd.tex', u'python-etcd Documentation', u'Jose Plana', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output -------------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'python-etcd', u'python-etcd Documentation', [u'Jose Plana'], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ------------------------------------------------ # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ('index', 'python-etcd', u'python-etcd Documentation', u'Jose Plana', 'python-etcd', 'One line description of project.', 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' python-etcd-0.4.3/docs-source/index.rst000066400000000000000000000110401263362545500200520ustar00rootroot00000000000000Python-etcd documentation ========================= A python client for Etcd https://github.com/coreos/etcd Installation ------------ Pre-requirements ................ Install etcd From source ........... .. code-block:: bash $ python setup.py install Usage ----- Create a client object ...................... .. code-block:: python import etcd client = etcd.Client() # this will create a client against etcd server running on localhost on port 4001 client = etcd.Client(port=4002) client = etcd.Client(host='127.0.0.1', port=4003) client = etcd.Client(host='127.0.0.1', port=4003, allow_redirect=False) # wont let you run sensitive commands on non-leader machines, default is true client = etcd.Client( host='127.0.0.1', port=4003, allow_reconnect=True, protocol='https',) Set a key ......... .. code-block:: python client.write('/nodes/n1', 1) # with ttl client.write('/nodes/n2', 2, ttl=4) # sets the ttl to 4 seconds # create only client.write('/nodes/n3', 'test', prevExist=False) # Compare and swap values atomically client.write('/nodes/n3', 'test2', prevValue='test1') #this fails to write client.write('/nodes/n3', 'test2', prevIndex=10) #this fails to write # mkdir client.write('/nodes/queue', dir=True) # Append a value to a queue dir client.write('/nodes/queue', 'test', append=True) #will write i.e. /nodes/queue/11 client.write('/nodes/queue', 'test2', append=True) #will write i.e. /nodes/queue/12 You can also atomically update a result: .. code:: python result = client.read('/foo') print(result.value) # bar result.value += u'bar' updated = client.update(result) # if any other client wrote '/foo' in the meantime this will fail print(updated.value) # barbar Get a key ......... .. code-block:: python client.read('/nodes/n2').value #recursively read a directory r = client.read('/nodes', recursive=True, sorted=True) for child in r.children: print("%s: %s" % (child.key,child.value)) client.read('/nodes/n2', wait=True) #Waits for a change in value in the key before returning. client.read('/nodes/n2', wait=True, waitIndex=10) Delete a key ............ .. code-block:: python client.delete('/nodes/n1') client.delete('/nodes', dir=True) #spits an error if dir is not empty client.delete('/nodes', recursive=True) #this works recursively Use lock primitives ................... .. code-block:: python # Initialize the lock object: # NOTE: this does not acquire a lock yet client = etcd.Client() lock = client.get_lock('/customer1', ttl=60) # Use the lock object: lock.acquire() lock.is_locked() # True lock.renew(60) lock.release() lock.is_locked() # False # The lock object may also be used as a context manager: client = etcd.Client() lock = client.get_lock('/customer1', ttl=60) with lock as my_lock: do_stuff() lock.is_locked() # True lock.renew(60) lock.is_locked() # False Use the leader election primitives .................................. .. code-block:: python # Set a leader object with a name; if no name is given, the local hostname # is used. # Zero or no ttl means the leader object is persistent. client = etcd.Client() client.election.set('/mysql', name='foo.example.com', ttl=120) # returns the etcd index # Get the name print(client.election.get('/mysql')) # 'foo.example.com' # Delete it! print(client.election.delete('/mysql', name='foo.example.com')) Get machines in the cluster ........................... .. code-block:: python client.machines Get leader of the cluster ......................... .. code-block:: python client.leader Development setup ----------------- To create a buildout, .. code-block:: bash $ python bootstrap.py $ bin/buildout to test you should have etcd available in your system path: .. code-block:: bash $ bin/test to generate documentation, .. code-block:: bash $ cd docs $ make Release HOWTO ------------- To make a release, 1) Update release date/version in NEWS.txt and setup.py 2) Run 'python setup.py sdist' 3) Test the generated source distribution in dist/ 4) Upload to PyPI: 'python setup.py sdist register upload' 5) Increase version in setup.py (for next release) List of contributors at https://github.com/jplana/python-etcd/graphs/contributors Code documentation ------------------ .. toctree:: :maxdepth: 2 api.rst python-etcd-0.4.3/setup.py000066400000000000000000000025351263362545500155060ustar00rootroot00000000000000from setuptools import setup, find_packages import sys, os here = os.path.abspath(os.path.dirname(__file__)) README = open(os.path.join(here, 'README.rst')).read() NEWS = open(os.path.join(here, 'NEWS.txt')).read() version = '0.4.3' # Dnspython is two different packages depending on python version if sys.version_info.major == 2: dns = 'dnspython' else: dns = 'dnspython3' install_requires = [ 'urllib3>=1.7.1', dns ] test_requires = [ 'mock', 'nose', 'pyOpenSSL>=0.14' ] setup( name='python-etcd', version=version, description="A python client for etcd", long_description=README + '\n\n' + NEWS, classifiers=[ "Topic :: System :: Distributed Computing", "Topic :: Software Development :: Libraries", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3", "Topic :: Database :: Front-Ends", ], keywords='etcd raft distributed log api client', author='Jose Plana', author_email='jplana@gmail.com', url='http://github.com/jplana/python-etcd', license='MIT', packages=find_packages('src'), package_dir = {'': 'src'}, include_package_data=True, zip_safe=False, install_requires=install_requires, tests_require=test_requires, test_suite='nose.collector', ) python-etcd-0.4.3/src/000077500000000000000000000000001263362545500145565ustar00rootroot00000000000000python-etcd-0.4.3/src/etcd/000077500000000000000000000000001263362545500154755ustar00rootroot00000000000000python-etcd-0.4.3/src/etcd/__init__.py000066400000000000000000000171711263362545500176150ustar00rootroot00000000000000import logging from .client import Client from .lock import Lock _log = logging.getLogger(__name__) # Prevent "no handler" warnings to stderr in projects that do not configure # logging. try: from logging import NullHandler except ImportError: # Python <2.7, just define it. class NullHandler(logging.Handler): def emit(self, record): pass _log.addHandler(NullHandler()) class EtcdResult(object): _node_props = { 'key': None, 'value': None, 'expiration': None, 'ttl': None, 'modifiedIndex': None, 'createdIndex': None, 'newKey': False, 'dir': False, } def __init__(self, action=None, node=None, prevNode=None, **kwdargs): """ Creates an EtcdResult object. Args: action (str): The action that resulted in key creation node (dict): The dictionary containing all node information. prevNode (dict): The dictionary containing previous node information. """ self.action = action for (key, default) in self._node_props.items(): if key in node: setattr(self, key, node[key]) else: setattr(self, key, default) self._children = [] if self.dir and 'nodes' in node: # We keep the data in raw format, converting them only when needed self._children = node['nodes'] if prevNode: self._prev_node = EtcdResult(None, node=prevNode) # See issue 38: when returning a write() op etcd has a bogus result. if self._prev_node.dir and not self.dir: self.dir = True def parse_headers(self, response): headers = response.getheaders() self.etcd_index = int(headers.get('x-etcd-index', 1)) self.raft_index = int(headers.get('x-raft-index', 1)) def get_subtree(self, leaves_only=False): """ Get all the subtree resulting from a recursive=true call to etcd. Args: leaves_only (bool): if true, only value nodes are returned """ if not self._children: #if the current result is a leaf, return itself yield self return else: # node is not a leaf if not leaves_only: yield self for n in self._children: node = EtcdResult(None, n) for child in node.get_subtree(leaves_only=leaves_only): yield child return @property def leaves(self): return self.get_subtree(leaves_only=True) @property def children(self): """ Deprecated, use EtcdResult.leaves instead """ return self.leaves def __eq__(self, other): if not (type(self) is type(other)): return False for k in self._node_props.keys(): try: a = getattr(self, k) b = getattr(other, k) if a != b: return False except: return False return True def __ne__(self, other): return not self.__eq__(other) def __repr__(self): return "%s(%r)" % (self.__class__, self.__dict__) class EtcdException(Exception): """ Generic Etcd Exception. """ def __init__(self, message=None, payload=None): super(EtcdException, self).__init__(message) self.payload = payload class EtcdValueError(EtcdException, ValueError): """ Base class for Etcd value-related errors. """ pass class EtcdCompareFailed(EtcdValueError): """ Compare-and-swap failure """ pass class EtcdClusterIdChanged(EtcdException): """ The etcd cluster ID changed. This may indicate the cluster was replaced with a backup. Raised to prevent waiting on an etcd_index that was only valid on the old cluster. """ pass class EtcdKeyError(EtcdException): """ Etcd Generic KeyError Exception """ pass class EtcdKeyNotFound(EtcdKeyError): """ Etcd key not found exception (100) """ pass class EtcdNotFile(EtcdKeyError): """ Etcd not a file exception (102) """ pass class EtcdNotDir(EtcdKeyError): """ Etcd not a directory exception (104) """ pass class EtcdAlreadyExist(EtcdKeyError): """ Etcd already exist exception (105) """ pass class EtcdEventIndexCleared(EtcdException): """ Etcd event index is outdated and cleared exception (401) """ pass class EtcdConnectionFailed(EtcdException): """ Connection to etcd failed. """ def __init__(self, message=None, payload=None, cause=None): super(EtcdConnectionFailed, self).__init__(message=message, payload=payload) self.cause = cause class EtcdInsufficientPermissions(EtcdException): """ Request failed because of insufficient permissions. """ pass class EtcdWatchTimedOut(EtcdConnectionFailed): """ A watch timed out without returning a result. """ pass class EtcdWatcherCleared(EtcdException): """ Watcher is cleared due to etcd recovery. """ pass class EtcdLeaderElectionInProgress(EtcdException): """ Request failed due to in-progress leader election. """ pass class EtcdRootReadOnly(EtcdKeyError): """ Operation is not valid on the root, which is read only. """ pass class EtcdDirNotEmpty(EtcdValueError): """ Directory not empty. """ pass class EtcdLockExpired(EtcdException): """ Our lock apparently expired while we were trying to acquire it. """ class EtcdError(object): # See https://github.com/coreos/etcd/blob/master/Documentation/errorcode.md error_exceptions = { 100: EtcdKeyNotFound, 101: EtcdCompareFailed, 102: EtcdNotFile, # 103: Non-public: no more peers. 104: EtcdNotDir, 105: EtcdAlreadyExist, # 106: Non-public: key is preserved. 107: EtcdRootReadOnly, 108: EtcdDirNotEmpty, # 109: Non-public: existing peer addr. 110: EtcdInsufficientPermissions, 200: EtcdValueError, 201: EtcdValueError, 202: EtcdValueError, 203: EtcdValueError, 204: EtcdValueError, 205: EtcdValueError, 206: EtcdValueError, 207: EtcdValueError, 208: EtcdValueError, 209: EtcdValueError, 210: EtcdValueError, # 300: Non-public: Raft internal error. 301: EtcdLeaderElectionInProgress, 400: EtcdWatcherCleared, 401: EtcdEventIndexCleared, } @classmethod def handle(cls, payload): """ Decodes the error and throws the appropriate error message :param payload: The decoded JSON error payload as a dict. """ error_code = payload.get("errorCode") message = payload.get("message") cause = payload.get("cause") msg = '{} : {}'.format(message, cause) status = payload.get("status") # Some general status handling, as # not all endpoints return coherent error messages if status == 404: error_code = 100 elif status == 401: error_code = 110 exc = cls.error_exceptions.get(error_code, EtcdException) if issubclass(exc, EtcdException): raise exc(msg, payload) else: raise exc(msg) # Attempt to enable urllib3's SNI support, if possible # Blatantly copied from requests. try: from urllib3.contrib import pyopenssl pyopenssl.inject_into_urllib3() except ImportError: pass python-etcd-0.4.3/src/etcd/auth.py000066400000000000000000000203421263362545500170110ustar00rootroot00000000000000import json import logging import etcd _log = logging.getLogger(__name__) class EtcdAuthBase(object): entity = 'example' def __init__(self, client, name): self.client = client self.name = name self.uri = "{}/auth/{}s/{}".format(self.client.version_prefix, self.entity, self.name) @property def names(self): key = "{}s".format(self.entity) uri = "{}/auth/{}".format(self.client.version_prefix, key) response = self.client.api_execute(uri, self.client._MGET) return json.loads(response.data.decode('utf-8'))[key] def read(self): try: response = self.client.api_execute(self.uri, self.client._MGET) except etcd.EtcdInsufficientPermissions as e: _log.error("Any action on the authorization requires the root role") raise except etcd.EtcdKeyNotFound: _log.info("%s '%s' not found", self.entity, self.name) raise except Exception as e: _log.error("Failed to fetch %s in %s%s: %r", self.entity, self.client._base_uri, self.client.version_prefix, e) raise etcd.EtcdException( "Could not fetch {} '{}'".format(self.entity, self.name)) self._from_net(response.data) def write(self): try: r = self.__class__(self.client, self.name) r.read() except etcd.EtcdKeyNotFound: r = None try: for payload in self._to_net(r): response = self.client.api_execute_json(self.uri, self.client._MPUT, params=payload) # This will fail if the response is an error self._from_net(response.data) except etcd.EtcdInsufficientPermissions as e: _log.error("Any action on the authorization requires the root role") raise except Exception as e: _log.error("Failed to write %s '%s'", self.entity, self.name) # TODO: fine-grained exception handling raise etcd.EtcdException( "Could not write {} '{}': {}".format(self.entity, self.name, e)) def delete(self): try: _ = self.client.api_execute(self.uri, self.client._MDELETE) except etcd.EtcdInsufficientPermissions as e: _log.error("Any action on the authorization requires the root role") raise except etcd.EtcdKeyNotFound: _log.info("%s '%s' not found", self.entity, self.name) raise except Exception as e: _log.error("Failed to delete %s in %s%s: %r", self.entity, self._base_uri, self.version_prefix, e) raise etcd.EtcdException( "Could not delete {} '{}'".format(self.entity, self.name)) def _from_net(self, data): raise NotImplementedError() def _to_net(self, old=None): raise NotImplementedError() @classmethod def new(cls, client, data): c = cls(client, data[cls.entity]) c._from_net(data) return c class EtcdUser(EtcdAuthBase): """Class to manage in a orm-like way etcd users""" entity = 'user' def __init__(self, client, name): super(EtcdUser, self).__init__(client, name) self._roles = set() self._password = None def _from_net(self, data): d = json.loads(data.decode('utf-8')) self.roles = d.get('roles', []) self.name = d.get('user') def _to_net(self, prevobj=None): if prevobj is None: retval = [{"user": self.name, "password": self._password, "roles": list(self.roles)}] else: retval = [] if self._password: retval.append({"user": self.name, "password": self._password}) to_grant = list(self.roles - prevobj.roles) to_revoke = list(prevobj.roles - self.roles) if to_grant: retval.append({"user": self.name, "grant": to_grant}) if to_revoke: retval.append({"user": self.name, "revoke": to_revoke}) # Let's blank the password now # Even if the user can't be written we don't want it to leak anymore. self._password = None return retval @property def roles(self): return self._roles @roles.setter def roles(self, val): self._roles = set(val) @property def password(self): """Empty property for password.""" return None @password.setter def password(self, new_password): """Change user's password.""" self._password = new_password def __str__(self): return json.dumps(self._to_net()[0]) class EtcdRole(EtcdAuthBase): entity = 'role' def __init__(self, client, name): super(EtcdRole, self).__init__(client, name) self._read_paths = set() self._write_paths = set() def _from_net(self, data): d = json.loads(data.decode('utf-8')) self.name = d.get('role') try: kv = d["permissions"]["kv"] except: self._read_paths = set() self._write_paths = set() return self._read_paths = set(kv.get('read', [])) self._write_paths = set(kv.get('write', [])) def _to_net(self, prevobj=None): retval = [] if prevobj is None: retval.append({ "role": self.name, "permissions": { "kv": { "read": list(self._read_paths), "write": list(self._write_paths) } } }) else: to_grant = { 'read': list(self._read_paths - prevobj._read_paths), 'write': list(self._write_paths - prevobj._write_paths) } to_revoke = { 'read': list(prevobj._read_paths - self._read_paths), 'write': list(prevobj._write_paths - self._write_paths) } if [path for sublist in to_revoke.values() for path in sublist]: retval.append({'role': self.name, 'revoke': {'kv': to_revoke}}) if [path for sublist in to_grant.values() for path in sublist]: retval.append({'role': self.name, 'grant': {'kv': to_grant}}) return retval def grant(self, path, permission): if permission.upper().find('R') >= 0: self._read_paths.add(path) if permission.upper().find('W') >= 0: self._write_paths.add(path) def revoke(self, path, permission): if permission.upper().find('R') >= 0 and \ path in self._read_paths: self._read_paths.remove(path) if permission.upper().find('W') >= 0 and \ path in self._write_paths: self._write_paths.remove(path) @property def acls(self): perms = {} try: for path in self._read_paths: perms[path] = 'R' for path in self._write_paths: if path in perms: perms[path] += 'W' else: perms[path] = 'W' except: pass return perms @acls.setter def acls(self, acls): self._read_paths = set() self._write_paths = set() for path, permission in acls.items(): self.grant(path, permission) def __str__(self): return json.dumps({"role": self.name, 'acls': self.acls}) class Auth(object): def __init__(self, client): self.client = client self.uri = "{}/auth/enable".format(self.client.version_prefix) @property def active(self): resp = self.client.api_execute(self.uri, self.client._MGET) return json.loads(resp.data.decode('utf-8'))['enabled'] @active.setter def active(self, value): if value != self.active: method = value and self.client._MPUT or self.client._MDELETE self.client.api_execute(self.uri, method) python-etcd-0.4.3/src/etcd/client.py000066400000000000000000000755241263362545500173420ustar00rootroot00000000000000""" .. module:: python-etcd :synopsis: A python etcd client. .. moduleauthor:: Jose Plana """ import logging try: # Python 3 from http.client import HTTPException except ImportError: # Python 2 from httplib import HTTPException import socket import urllib3 import urllib3.util import json import ssl import dns.resolver from functools import wraps import etcd try: from urlparse import urlparse except ImportError: from urllib.parse import urlparse _log = logging.getLogger(__name__) class Client(object): """ Client for etcd, the distributed log service using raft. """ _MGET = 'GET' _MPUT = 'PUT' _MPOST = 'POST' _MDELETE = 'DELETE' _comparison_conditions = set(('prevValue', 'prevIndex', 'prevExist')) _read_options = set(('recursive', 'wait', 'waitIndex', 'sorted', 'quorum')) _del_conditions = set(('prevValue', 'prevIndex')) http = None def __init__( self, host='127.0.0.1', port=4001, srv_domain=None, version_prefix='/v2', read_timeout=60, allow_redirect=True, protocol='http', cert=None, ca_cert=None, username=None, password=None, allow_reconnect=False, use_proxies=False, expected_cluster_id=None, per_host_pool_size=10 ): """ Initialize the client. Args: host (mixed): If a string, IP to connect to. If a tuple ((host, port), (host, port), ...) port (int): Port used to connect to etcd. srv_domain (str): Domain to search the SRV record for cluster autodiscovery. version_prefix (str): Url or version prefix in etcd url (default=/v2). read_timeout (int): max seconds to wait for a read. allow_redirect (bool): allow the client to connect to other nodes. protocol (str): Protocol used to connect to etcd. cert (mixed): If a string, the whole ssl client certificate; if a tuple, the cert and key file names. ca_cert (str): The ca certificate. If pressent it will enable validation. username (str): username for etcd authentication. password (str): password for etcd authentication. allow_reconnect (bool): allow the client to reconnect to another etcd server in the cluster in the case the default one does not respond. use_proxies (bool): we are using a list of proxies to which we connect, and don't want to connect to the original etcd cluster. expected_cluster_id (str): If a string, recorded as the expected UUID of the cluster (rather than learning it from the first request), reads will raise EtcdClusterIdChanged if they receive a response with a different cluster ID. per_host_pool_size (int): specifies maximum number of connections to pool by host. By default this will use up to 10 connections. """ # If a DNS record is provided, use it to get the hosts list if srv_domain is not None: try: host = self._discover(srv_domain) except Exception as e: _log.error("Could not discover the etcd hosts from %s: %s", srv_domain, e) self._protocol = protocol def uri(protocol, host, port): return '%s://%s:%d' % (protocol, host, port) if not isinstance(host, tuple): self._machines_cache = [] self._base_uri = uri(self._protocol, host, port) else: if not allow_reconnect: _log.error("List of hosts incompatible with allow_reconnect.") raise etcd.EtcdException("A list of hosts to connect to was given, but reconnection not allowed?") self._machines_cache = [uri(self._protocol, *conn) for conn in host] self._base_uri = self._machines_cache.pop(0) self.expected_cluster_id = expected_cluster_id self.version_prefix = version_prefix self._read_timeout = read_timeout self._allow_redirect = allow_redirect self._use_proxies = use_proxies self._allow_reconnect = allow_reconnect # SSL Client certificate support kw = { 'maxsize': per_host_pool_size } if self._read_timeout > 0: kw['timeout'] = self._read_timeout if protocol == 'https': # If we don't allow TLSv1, clients using older version of OpenSSL # (<1.0) won't be able to connect. _log.debug("HTTPS enabled.") kw['ssl_version'] = ssl.PROTOCOL_TLSv1 if cert: if isinstance(cert, tuple): # Key and cert are separate kw['cert_file'] = cert[0] kw['key_file'] = cert[1] else: # combined certificate kw['cert_file'] = cert if ca_cert: kw['ca_certs'] = ca_cert kw['cert_reqs'] = ssl.CERT_REQUIRED self.username = None self.password = None if username and password: self.username = username self.password = password elif username: _log.warning('Username provided without password, both are required for authentication') elif password: _log.warning('Password provided without username, both are required for authentication') self.http = urllib3.PoolManager(num_pools=10, **kw) _log.debug("New etcd client created for %s", self.base_uri) if self._allow_reconnect: # we need the set of servers in the cluster in order to try # reconnecting upon error. The cluster members will be # added to the hosts list you provided. If you are using # proxies, set all # # Beware though: if you input '127.0.0.1' as your host and # etcd advertises 'localhost', both will be in the # resulting list. # If we're connecting to the original cluster, we can # extend the list given to the client with what we get # from self.machines if not self._use_proxies: self._machines_cache = list(set(self._machines_cache) | set(self.machines)) if self._base_uri in self._machines_cache: self._machines_cache.remove(self._base_uri) _log.debug("Machines cache initialised to %s", self._machines_cache) def _discover(self, domain): srv_name = "_etcd._tcp.{}".format(domain) answers = dns.resolver.query(srv_name, 'SRV') hosts = [] for answer in answers: hosts.append( (answer.target.to_text(omit_final_dot=True), answer.port)) _log.debug("Found %s", hosts) if not len(hosts): raise ValueError("The SRV record is present but no host were found") return tuple(hosts) def __del__(self): """Clean up open connections""" if self.http is not None: try: self.http.clear() except ReferenceError: # this may hit an already-cleared weakref pass @property def base_uri(self): """URI used by the client to connect to etcd.""" return self._base_uri @property def host(self): """Node to connect etcd.""" return urlparse(self._base_uri).netloc.split(':')[0] @property def port(self): """Port to connect etcd.""" return int(urlparse(self._base_uri).netloc.split(':')[1]) @property def protocol(self): """Protocol used to connect etcd.""" return self._protocol @property def read_timeout(self): """Max seconds to wait for a read.""" return self._read_timeout @property def allow_redirect(self): """Allow the client to connect to other nodes.""" return self._allow_redirect @property def machines(self): """ Members of the cluster. Returns: list. str with all the nodes in the cluster. >>> print client.machines ['http://127.0.0.1:4001', 'http://127.0.0.1:4002'] """ # We can't use api_execute here, or it causes a logical loop try: uri = self._base_uri + self.version_prefix + '/machines' response = self.http.request( self._MGET, uri, headers=self._get_headers(), timeout=self.read_timeout, redirect=self.allow_redirect) machines = [ node.strip() for node in self._handle_server_response(response).data.decode('utf-8').split(',') ] _log.debug("Retrieved list of machines: %s", machines) return machines except (urllib3.exceptions.HTTPError, HTTPException, socket.error) as e: # We can't get the list of machines, if one server is in the # machines cache, try on it _log.error("Failed to get list of machines from %s%s: %r", self._base_uri, self.version_prefix, e) if self._machines_cache: self._base_uri = self._machines_cache.pop(0) _log.info("Retrying on %s", self._base_uri) # Call myself return self.machines else: raise etcd.EtcdException("Could not get the list of servers, " "maybe you provided the wrong " "host(s) to connect to?") @property def members(self): """ A more structured view of peers in the cluster. Note that while we have an internal DS called _members, accessing the public property will call etcd. """ # Empty the members list self._members = {} try: data = self.api_execute(self.version_prefix + '/members', self._MGET).data.decode('utf-8') res = json.loads(data) for member in res['members']: self._members[member['id']] = member return self._members except: raise etcd.EtcdException("Could not get the members list, maybe the cluster has gone away?") @property def leader(self): """ Returns: dict. the leader of the cluster. >>> print client.leader {"id":"ce2a822cea30bfca","name":"default","peerURLs":["http://localhost:2380","http://localhost:7001"],"clientURLs":["http://127.0.0.1:4001"]} """ try: leader = json.loads( self.api_execute(self.version_prefix + '/stats/self', self._MGET).data.decode('utf-8')) return self.members[leader['leaderInfo']['leader']] except Exception as e: raise etcd.EtcdException("Cannot get leader data: %s" % e) @property def stats(self): """ Returns: dict. the stats of the local server """ return self._stats() @property def leader_stats(self): """ Returns: dict. the stats of the leader """ return self._stats('leader') @property def store_stats(self): """ Returns: dict. the stats of the kv store """ return self._stats('store') def _stats(self, what='self'): """ Internal method to access the stats endpoints""" data = self.api_execute(self.version_prefix + '/stats/' + what, self._MGET).data.decode('utf-8') try: return json.loads(data) except (TypeError,ValueError): raise etcd.EtcdException("Cannot parse json data in the response") @property def key_endpoint(self): """ REST key endpoint. """ return self.version_prefix + '/keys' def __contains__(self, key): """ Check if a key is available in the cluster. >>> print 'key' in client True """ try: self.get(key) return True except etcd.EtcdKeyNotFound: return False def _sanitize_key(self, key): if not key.startswith('/'): key = "/{}".format(key) return key def write(self, key, value, ttl=None, dir=False, append=False, **kwdargs): """ Writes the value for a key, possibly doing atomit Compare-and-Swap Args: key (str): Key. value (object): value to set ttl (int): Time in seconds of expiration (optional). dir (bool): Set to true if we are writing a directory; default is false. append (bool): If true, it will post to append the new value to the dir, creating a sequential key. Defaults to false. Other parameters modifying the write method are accepted: prevValue (str): compare key to this value, and swap only if corresponding (optional). prevIndex (int): modify key only if actual modifiedIndex matches the provided one (optional). prevExist (bool): If false, only create key; if true, only update key. Returns: client.EtcdResult >>> print client.write('/key', 'newValue', ttl=60, prevExist=False).value 'newValue' """ _log.debug("Writing %s to key %s ttl=%s dir=%s append=%s", value, key, ttl, dir, append) key = self._sanitize_key(key) params = {} if value is not None: params['value'] = value if ttl is not None: params['ttl'] = ttl if dir: if value: raise etcd.EtcdException( 'Cannot create a directory with a value') params['dir'] = "true" for (k, v) in kwdargs.items(): if k in self._comparison_conditions: if type(v) == bool: params[k] = v and "true" or "false" else: params[k] = v method = append and self._MPOST or self._MPUT if '_endpoint' in kwdargs: path = kwdargs['_endpoint'] + key else: path = self.key_endpoint + key response = self.api_execute(path, method, params=params) return self._result_from_response(response) def update(self, obj): """ Updates the value for a key atomically. Typical usage would be: c = etcd.Client() o = c.read("/somekey") o.value += 1 c.update(o) Args: obj (etcd.EtcdResult): The object that needs updating. """ _log.debug("Updating %s to %s.", obj.key, obj.value) kwdargs = { 'dir': obj.dir, 'ttl': obj.ttl, 'prevExist': True } if not obj.dir: # prevIndex on a dir causes a 'not a file' error. d'oh! kwdargs['prevIndex'] = obj.modifiedIndex return self.write(obj.key, obj.value, **kwdargs) def read(self, key, **kwdargs): """ Returns the value of the key 'key'. Args: key (str): Key. Recognized kwd args recursive (bool): If you should fetch recursively a dir wait (bool): If we should wait and return next time the key is changed waitIndex (int): The index to fetch results from. sorted (bool): Sort the output keys (alphanumerically) timeout (int): max seconds to wait for a read. Returns: client.EtcdResult (or an array of client.EtcdResult if a subtree is queried) Raises: KeyValue: If the key doesn't exists. urllib3.exceptions.TimeoutError: If timeout is reached. >>> print client.get('/key').value 'value' """ _log.debug("Issuing read for key %s with args %s", key, kwdargs) key = self._sanitize_key(key) params = {} for (k, v) in kwdargs.items(): if k in self._read_options: if type(v) == bool: params[k] = v and "true" or "false" elif v is not None: params[k] = v timeout = kwdargs.get('timeout', None) response = self.api_execute( self.key_endpoint + key, self._MGET, params=params, timeout=timeout) return self._result_from_response(response) def delete(self, key, recursive=None, dir=None, **kwdargs): """ Removed a key from etcd. Args: key (str): Key. recursive (bool): if we want to recursively delete a directory, set it to true dir (bool): if we want to delete a directory, set it to true prevValue (str): compare key to this value, and swap only if corresponding (optional). prevIndex (int): modify key only if actual modifiedIndex matches the provided one (optional). Returns: client.EtcdResult Raises: KeyValue: If the key doesn't exists. >>> print client.delete('/key').key '/key' """ _log.debug("Deleting %s recursive=%s dir=%s extra args=%s", key, recursive, dir, kwdargs) key = self._sanitize_key(key) kwds = {} if recursive is not None: kwds['recursive'] = recursive and "true" or "false" if dir is not None: kwds['dir'] = dir and "true" or "false" for k in self._del_conditions: if k in kwdargs: kwds[k] = kwdargs[k] _log.debug("Calculated params = %s", kwds) response = self.api_execute( self.key_endpoint + key, self._MDELETE, params=kwds) return self._result_from_response(response) def pop(self, key, recursive=None, dir=None, **kwdargs): """ Remove specified key from etcd and return the corresponding value. Args: key (str): Key. recursive (bool): if we want to recursively delete a directory, set it to true dir (bool): if we want to delete a directory, set it to true prevValue (str): compare key to this value, and swap only if corresponding (optional). prevIndex (int): modify key only if actual modifiedIndex matches the provided one (optional). Returns: client.EtcdResult Raises: KeyValue: If the key doesn't exists. >>> print client.pop('/key').value 'value' """ return self.delete(key=key, recursive=recursive, dir=dir, **kwdargs)._prev_node # Higher-level methods on top of the basic primitives def test_and_set(self, key, value, prev_value, ttl=None): """ Atomic test & set operation. It will check if the value of 'key' is 'prev_value', if the the check is correct will change the value for 'key' to 'value' if the the check is false an exception will be raised. Args: key (str): Key. value (object): value to set prev_value (object): previous value. ttl (int): Time in seconds of expiration (optional). Returns: client.EtcdResult Raises: ValueError: When the 'prev_value' is not the current value. >>> print client.test_and_set('/key', 'new', 'old', ttl=60).value 'new' """ return self.write(key, value, prevValue=prev_value, ttl=ttl) def set(self, key, value, ttl=None): """ Compatibility: sets the value of the key 'key' to the value 'value' Args: key (str): Key. value (object): value to set ttl (int): Time in seconds of expiration (optional). Returns: client.EtcdResult Raises: etcd.EtcdException: when something weird goes wrong. """ return self.write(key, value, ttl=ttl) def get(self, key): """ Returns the value of the key 'key'. Args: key (str): Key. Returns: client.EtcdResult Raises: KeyError: If the key doesn't exists. >>> print client.get('/key').value 'value' """ return self.read(key) def watch(self, key, index=None, timeout=None, recursive=None): """ Blocks until a new event has been received, starting at index 'index' Args: key (str): Key. index (int): Index to start from. timeout (int): max seconds to wait for a read. Returns: client.EtcdResult Raises: KeyValue: If the key doesn't exists. urllib3.exceptions.TimeoutError: If timeout is reached. >>> print client.watch('/key').value 'value' """ _log.debug("About to wait on key %s, index %s", key, index) if index: return self.read(key, wait=True, waitIndex=index, timeout=timeout, recursive=recursive) else: return self.read(key, wait=True, timeout=timeout, recursive=recursive) def eternal_watch(self, key, index=None, recursive=None): """ Generator that will yield changes from a key. Note that this method will block forever until an event is generated. Args: key (str): Key to subcribe to. index (int): Index from where the changes will be received. Yields: client.EtcdResult >>> for event in client.eternal_watch('/subcription_key'): ... print event.value ... value1 value2 """ local_index = index while True: response = self.watch(key, index=local_index, timeout=0, recursive=recursive) local_index = response.modifiedIndex + 1 yield response def get_lock(self, *args, **kwargs): raise NotImplementedError('Lock primitives were removed from etcd 2.0') @property def election(self): raise NotImplementedError('Election primitives were removed from etcd 2.0') def _result_from_response(self, response): """ Creates an EtcdResult from json dictionary """ raw_response = response.data try: res = json.loads(raw_response.decode('utf-8')) except (TypeError, ValueError, UnicodeError) as e: raise etcd.EtcdException( 'Server response was not valid JSON: %r' % e) try: r = etcd.EtcdResult(**res) if response.status == 201: r.newKey = True r.parse_headers(response) return r except Exception as e: raise etcd.EtcdException( 'Unable to decode server response: %r' % e) def _next_server(self, cause=None): """ Selects the next server in the list, refreshes the server list. """ _log.debug("Selection next machine in cache. Available machines: %s", self._machines_cache) try: mach = self._machines_cache.pop() except IndexError: _log.error("Machines cache is empty, no machines to try.") raise etcd.EtcdConnectionFailed('No more machines in the cluster', cause=cause) else: _log.info("Selected new etcd server %s", mach) return mach def _wrap_request(payload): @wraps(payload) def wrapper(self, path, method, params=None, timeout=None): some_request_failed = False response = False if timeout is None: timeout = self.read_timeout if timeout == 0: timeout = None if not path.startswith('/'): raise ValueError('Path does not start with /') while not response: try: response = payload(self, path, method, params=params, timeout=timeout) # Check the cluster ID hasn't changed under us. We use # preload_content=False above so we can read the headers # before we wait for the content of a watch. self._check_cluster_id(response) # Now force the data to be preloaded in order to trigger any # IO-related errors in this method rather than when we try to # access it later. _ = response.data # urllib3 doesn't wrap all httplib exceptions and earlier versions # don't wrap socket errors either. except (urllib3.exceptions.HTTPError, HTTPException, socket.error) as e: if (isinstance(params, dict) and params.get("wait") == "true" and isinstance(e, urllib3.exceptions.ReadTimeoutError)): _log.debug("Watch timed out.") raise etcd.EtcdWatchTimedOut( "Watch timed out: %r" % e, cause=e ) _log.error("Request to server %s failed: %r", self._base_uri, e) if self._allow_reconnect: _log.info("Reconnection allowed, looking for another " "server.") # _next_server() raises EtcdException if there are no # machines left to try, breaking out of the loop. self._base_uri = self._next_server(cause=e) some_request_failed = True # if exception is raised on _ = response.data # the condition for while loop will be False # but we should retry response = False else: _log.debug("Reconnection disabled, giving up.") raise etcd.EtcdConnectionFailed( "Connection to etcd failed due to %r" % e, cause=e ) except etcd.EtcdClusterIdChanged as e: _log.warning(e) raise except: _log.exception("Unexpected request failure, re-raising.") raise if some_request_failed: if not self._use_proxies: # The cluster may have changed since last invocation self._machines_cache = self.machines self._machines_cache.remove(self._base_uri) return self._handle_server_response(response) return wrapper @_wrap_request def api_execute(self, path, method, params=None, timeout=None): """ Executes the query. """ url = self._base_uri + path if (method == self._MGET) or (method == self._MDELETE): return self.http.request( method, url, timeout=timeout, fields=params, redirect=self.allow_redirect, headers=self._get_headers(), preload_content=False) elif (method == self._MPUT) or (method == self._MPOST): return self.http.request_encode_body( method, url, fields=params, timeout=timeout, encode_multipart=False, redirect=self.allow_redirect, headers=self._get_headers(), preload_content=False) else: raise etcd.EtcdException( 'HTTP method {} not supported'.format(method)) @_wrap_request def api_execute_json(self, path, method, params=None, timeout=None): url = self._base_uri + path json_payload = json.dumps(params) headers = self._get_headers() headers['Content-Type'] = 'application/json' return self.http.urlopen(method, url, body=json_payload, timeout=timeout, redirect=self.allow_redirect, headers=headers, preload_content=False) def _check_cluster_id(self, response): cluster_id = response.getheader("x-etcd-cluster-id") if not cluster_id: _log.warning("etcd response did not contain a cluster ID") return id_changed = (self.expected_cluster_id and cluster_id != self.expected_cluster_id) # Update the ID so we only raise the exception once. old_expected_cluster_id = self.expected_cluster_id self.expected_cluster_id = cluster_id if id_changed: # Defensive: clear the pool so that we connect afresh next # time. self.http.clear() raise etcd.EtcdClusterIdChanged( 'The UUID of the cluster changed from {} to ' '{}.'.format(old_expected_cluster_id, cluster_id)) def _handle_server_response(self, response): """ Handles the server response """ if response.status in [200, 201]: return response else: resp = response.data.decode('utf-8') # throw the appropriate exception try: r = json.loads(resp) r['status'] = response.status except (TypeError, ValueError): # Bad JSON, make a response locally. r = {"message": "Bad response", "cause": str(resp)} etcd.EtcdError.handle(r) def _get_headers(self): if self.username and self.password: credentials = ':'.join((self.username, self.password)) return urllib3.make_headers(basic_auth=credentials) return {} python-etcd-0.4.3/src/etcd/lock.py000066400000000000000000000140211263362545500167750ustar00rootroot00000000000000import logging import etcd import uuid _log = logging.getLogger(__name__) class Lock(object): """ Locking recipe for etcd, inspired by the kazoo recipe for zookeeper """ def __init__(self, client, lock_name): self.client = client self.name = lock_name # props to Netflix Curator for this trick. It is possible for our # create request to succeed on the server, but for a failure to # prevent us from getting back the full path name. We prefix our # lock name with a uuid and can check for its presence on retry. self._uuid = uuid.uuid4().hex self.path = "/_locks/{}".format(lock_name) self.is_taken = False self._sequence = None _log.debug("Initiating lock for %s with uuid %s", self.path, self._uuid) @property def uuid(self): """ The unique id of the lock """ return self._uuid @uuid.setter def set_uuid(self, value): old_uuid = self._uuid self._uuid = value if not self._find_lock(): _log.warn("The hand-set uuid was not found, refusing") self._uuid = old_uuid raise ValueError("Inexistent UUID") @property def is_acquired(self): """ tells us if the lock is acquired """ if not self.is_taken: _log.debug("Lock not taken") return False try: self.client.read(self.lock_key) return True except etcd.EtcdKeyNotFound: _log.warn("Lock was supposedly taken, but we cannot find it") self.is_taken = False return False def acquire(self, blocking=True, lock_ttl=3600, timeout=None): """ Acquire the lock. :param blocking Block until the lock is obtained, or timeout is reached :param lock_ttl The duration of the lock we acquired, set to None for eternal locks :param timeout The time to wait before giving up on getting a lock """ # First of all try to write, if our lock is not present. if not self._find_lock(): _log.debug("Lock not found, writing it to %s", self.path) res = self.client.write(self.path, self.uuid, ttl=lock_ttl, append=True) self._set_sequence(res.key) _log.debug("Lock key %s written, sequence is %s", res.key, self._sequence) elif lock_ttl: # Renew our lock if already here! self.client.write(self.lock_key, self.uuid, ttl=lock_ttl) # now get the owner of the lock, and the next lowest sequence return self._acquired(blocking=blocking, timeout=timeout) def release(self): """ Release the lock """ if not self._sequence: self._find_lock() try: _log.debug("Releasing existing lock %s", self.lock_key) self.client.delete(self.lock_key) except etcd.EtcdKeyNotFound: _log.info("Lock %s not found, nothing to release", self.lock_key) pass finally: self.is_taken = False def __enter__(self): """ You can use the lock as a contextmanager """ self.acquire(blocking=True, lock_ttl=0) def __exit__(self, type, value, traceback): self.release() def _acquired(self, blocking=True, timeout=0): locker, nearest = self._get_locker() self.is_taken = False if self.lock_key == locker: _log.debug("Lock acquired!") # We own the lock, yay! self.is_taken = True return True else: self.is_taken = False if not blocking: return False # Let's look for the lock watch_key = nearest _log.debug("Lock not acquired, now watching %s", watch_key) t = max(0, timeout) while True: try: r = self.client.watch(watch_key, timeout=t) _log.debug("Detected variation for %s: %s", r.key, r.action) return self._acquired(blocking=True, timeout=timeout) except etcd.EtcdKeyNotFound: _log.debug("Key %s not present anymore, moving on", watch_key) return self._acquired(blocking=True, timeout=timeout) except etcd.EtcdException: # TODO: log something... pass @property def lock_key(self): if not self._sequence: raise ValueError("No sequence present.") return self.path + '/' + str(self._sequence) def _set_sequence(self, key): self._sequence = key.replace(self.path, '').lstrip('/') def _find_lock(self): if self._sequence: try: res = self.client.read(self.lock_key) self._uuid = res.value return True except etcd.EtcdKeyNotFound: return False elif self._uuid: try: for r in self.client.read(self.path, recursive=True).leaves: if r.value == self._uuid: self._set_sequence(r.key) return True except etcd.EtcdKeyNotFound: pass return False def _get_locker(self): results = [res for res in self.client.read(self.path, recursive=True).leaves] if not self._sequence: self._find_lock() l = sorted([r.key for r in results]) _log.debug("Lock keys found: %s", l) try: i = l.index(self.lock_key) if i == 0: _log.debug("No key before our one, we are the locker") return (l[0], None) else: _log.debug("Locker: %s, key to watch: %s", l[0], l[i-1]) return (l[0], l[i-1]) except ValueError: # Something very wrong is going on, most probably # our lock has expired raise etcd.EtcdLockExpired(u"Lock not found") python-etcd-0.4.3/src/etcd/tests/000077500000000000000000000000001263362545500166375ustar00rootroot00000000000000python-etcd-0.4.3/src/etcd/tests/__init__.py000066400000000000000000000000231263362545500207430ustar00rootroot00000000000000from . import unit python-etcd-0.4.3/src/etcd/tests/integration/000077500000000000000000000000001263362545500211625ustar00rootroot00000000000000python-etcd-0.4.3/src/etcd/tests/integration/__init__.py000066400000000000000000000000001263362545500232610ustar00rootroot00000000000000python-etcd-0.4.3/src/etcd/tests/integration/helpers.py000066400000000000000000000146371263362545500232110ustar00rootroot00000000000000import shutil import subprocess import tempfile import logging import time import hashlib import uuid from OpenSSL import crypto class EtcdProcessHelper(object): def __init__( self, base_directory, proc_name='etcd', port_range_start=4001, internal_port_range_start=7001, cluster=False, tls=False ): self.base_directory = base_directory self.proc_name = proc_name self.port_range_start = port_range_start self.internal_port_range_start = internal_port_range_start self.processes = {} self.cluster = cluster self.schema = 'http://' if tls: self.schema = 'https://' def run(self, number=1, proc_args=[]): if number > 1: initial_cluster = ",".join([ "test-node-{}={}127.0.0.1:{}".format(slot, 'http://', self.internal_port_range_start + slot) for slot in range(0, number)]) proc_args.extend([ '-initial-cluster', initial_cluster, '-initial-cluster-state', 'new' ]) else: proc_args.extend([ '-initial-cluster', 'test-node-0=http://127.0.0.1:{}'.format(self.internal_port_range_start), '-initial-cluster-state', 'new' ]) for i in range(0, number): self.add_one(i, proc_args) def stop(self): log = logging.getLogger() for key in [k for k in self.processes.keys()]: self.kill_one(key) def add_one(self, slot, proc_args=None): log = logging.getLogger() directory = tempfile.mkdtemp( dir=self.base_directory, prefix='python-etcd.%d-' % slot) log.debug('Created directory %s' % directory) client = '%s127.0.0.1:%d' % (self.schema, self.port_range_start + slot) peer = '%s127.0.0.1:%d' % ('http://', self.internal_port_range_start + slot) daemon_args = [ self.proc_name, '-data-dir', directory, '-name', 'test-node-%d' % slot, '-initial-advertise-peer-urls', peer, '-listen-peer-urls', peer, '-advertise-client-urls', client, '-listen-client-urls', client ] if proc_args: daemon_args.extend(proc_args) daemon = subprocess.Popen(daemon_args) log.debug('Started %d' % daemon.pid) log.debug('Params: %s' % daemon_args) time.sleep(2) self.processes[slot] = (directory, daemon) def kill_one(self, slot): log = logging.getLogger() data_dir, process = self.processes.pop(slot) process.kill() time.sleep(2) log.debug('Killed etcd pid:%d', process.pid) shutil.rmtree(data_dir) log.debug('Removed directory %s' % data_dir) class TestingCA(object): @classmethod def create_test_ca_certificate(cls, cert_path, key_path, cn=None): k = crypto.PKey() k.generate_key(crypto.TYPE_RSA, 4096) cert = crypto.X509() if not cn: serial = uuid.uuid4().int else: md5_hash = hashlib.md5() md5_hash.update(cn.encode('utf-8')) serial = int(md5_hash.hexdigest(), 36) cert.get_subject().CN = cn cert.get_subject().C = "ES" cert.get_subject().ST = "State" cert.get_subject().L = "City" cert.get_subject().O = "Organization" cert.get_subject().OU = "Organizational Unit" cert.set_serial_number(serial) cert.gmtime_adj_notBefore(0) cert.gmtime_adj_notAfter(315360000) cert.set_issuer(cert.get_subject()) cert.set_pubkey(k) cert.add_extensions([ crypto.X509Extension("basicConstraints".encode('ascii'), False, "CA:TRUE".encode('ascii')), crypto.X509Extension("keyUsage".encode('ascii'), False, "keyCertSign, cRLSign".encode('ascii')), crypto.X509Extension("subjectKeyIdentifier".encode('ascii'), False, "hash".encode('ascii'), subject=cert), ]) cert.add_extensions([ crypto.X509Extension( "authorityKeyIdentifier".encode('ascii'), False, "keyid:always".encode('ascii'), issuer=cert) ]) cert.sign(k, 'sha1') with open(cert_path, 'w') as f: f.write(crypto.dump_certificate(crypto.FILETYPE_PEM, cert) .decode('utf-8')) with open(key_path, 'w') as f: f.write(crypto.dump_privatekey(crypto.FILETYPE_PEM, k) .decode('utf-8')) return cert, k @classmethod def create_test_certificate(cls, ca, ca_key, cert_path, key_path, cn=None): k = crypto.PKey() k.generate_key(crypto.TYPE_RSA, 4096) cert = crypto.X509() if not cn: serial = uuid.uuid4().int else: md5_hash = hashlib.md5() md5_hash.update(cn.encode('utf-8')) serial = int(md5_hash.hexdigest(), 36) cert.get_subject().CN = cn cert.get_subject().C = "ES" cert.get_subject().ST = "State" cert.get_subject().L = "City" cert.get_subject().O = "Organization" cert.get_subject().OU = "Organizational Unit" cert.add_extensions([ crypto.X509Extension( "keyUsage".encode('ascii'), False, "nonRepudiation,digitalSignature,keyEncipherment".encode('ascii')), crypto.X509Extension( "extendedKeyUsage".encode('ascii'), False, "clientAuth,serverAuth".encode('ascii')), crypto.X509Extension( "subjectAltName".encode('ascii'), False, "IP: 127.0.0.1".encode('ascii')), ]) cert.gmtime_adj_notBefore(0) cert.gmtime_adj_notAfter(315360000) cert.set_issuer(ca.get_subject()) cert.set_pubkey(k) cert.set_serial_number(serial) cert.sign(ca_key, 'sha1') with open(cert_path, 'w') as f: f.write(crypto.dump_certificate(crypto.FILETYPE_PEM, cert) .decode('utf-8')) with open(key_path, 'w') as f: f.write(crypto.dump_privatekey(crypto.FILETYPE_PEM, k) .decode('utf-8')) python-etcd-0.4.3/src/etcd/tests/integration/test_simple.py000066400000000000000000000311141263362545500240640ustar00rootroot00000000000000import os import time import shutil import logging import unittest import multiprocessing import tempfile import urllib3 import etcd from . import helpers from nose.tools import nottest log = logging.getLogger() class EtcdIntegrationTest(unittest.TestCase): cl_size = 3 @classmethod def setUpClass(cls): program = cls._get_exe() cls.directory = tempfile.mkdtemp(prefix='python-etcd') cls.processHelper = helpers.EtcdProcessHelper( cls.directory, proc_name=program, port_range_start=6001, internal_port_range_start=8001) cls.processHelper.run(number=cls.cl_size) cls.client = etcd.Client(port=6001) @classmethod def tearDownClass(cls): cls.processHelper.stop() shutil.rmtree(cls.directory) @classmethod def _is_exe(cls, fpath): return os.path.isfile(fpath) and os.access(fpath, os.X_OK) @classmethod def _get_exe(cls): PROGRAM = 'etcd' program_path = None for path in os.environ["PATH"].split(os.pathsep): path = path.strip('"') exe_file = os.path.join(path, PROGRAM) if cls._is_exe(exe_file): program_path = exe_file break if not program_path: raise Exception('etcd not in path!!') return program_path class TestSimple(EtcdIntegrationTest): def test_machines(self): """ INTEGRATION: retrieve machines """ self.assertEquals(self.client.machines[0], 'http://127.0.0.1:6001') def test_leader(self): """ INTEGRATION: retrieve leader """ self.assertEquals(self.client.leader['clientURLs'], ['http://127.0.0.1:6001']) def test_get_set_delete(self): """ INTEGRATION: set a new value """ try: get_result = self.client.get('/test_set') assert False except etcd.EtcdKeyNotFound as e: pass self.assertFalse('/test_set' in self.client) set_result = self.client.set('/test_set', 'test-key') self.assertEquals('set', set_result.action.lower()) self.assertEquals('/test_set', set_result.key) self.assertEquals('test-key', set_result.value) self.assertTrue('/test_set' in self.client) get_result = self.client.get('/test_set') self.assertEquals('get', get_result.action.lower()) self.assertEquals('/test_set', get_result.key) self.assertEquals('test-key', get_result.value) delete_result = self.client.delete('/test_set') self.assertEquals('delete', delete_result.action.lower()) self.assertEquals('/test_set', delete_result.key) self.assertFalse('/test_set' in self.client) try: get_result = self.client.get('/test_set') assert False except etcd.EtcdKeyNotFound as e: pass def test_update(self): """INTEGRATION: update a value""" self.client.set('/foo', 3) c = self.client.get('/foo') c.value = int(c.value) + 3 self.client.update(c) newres = self.client.get('/foo') self.assertEquals(newres.value, u'6') self.assertRaises(ValueError, self.client.update, c) def test_retrieve_subkeys(self): """ INTEGRATION: retrieve multiple subkeys """ set_result = self.client.write('/subtree/test_set', 'test-key1') set_result = self.client.write('/subtree/test_set1', 'test-key2') set_result = self.client.write('/subtree/test_set2', 'test-key3') get_result = self.client.read('/subtree', recursive=True) result = [subkey.value for subkey in get_result.leaves] self.assertEquals(['test-key1', 'test-key2', 'test-key3'].sort(), result.sort()) def test_directory_ttl_update(self): """ INTEGRATION: should be able to update a dir TTL """ self.client.write('/dir', None, dir=True, ttl=30) res = self.client.write('/dir', None, dir=True, ttl=31, prevExist=True) self.assertEquals(res.ttl, 31) res = self.client.get('/dir') res.ttl = 120 new_res = self.client.update(res) self.assertEquals(new_res.ttl, 120) class TestErrors(EtcdIntegrationTest): def test_is_not_a_file(self): """ INTEGRATION: try to write value to an existing directory """ self.client.set('/directory/test-key', 'test-value') self.assertRaises(etcd.EtcdNotFile, self.client.set, '/directory', 'test-value') def test_test_and_set(self): """ INTEGRATION: try test_and_set operation """ set_result = self.client.set('/test-key', 'old-test-value') set_result = self.client.test_and_set( '/test-key', 'test-value', 'old-test-value') self.assertRaises(ValueError, self.client.test_and_set, '/test-key', 'new-value', 'old-test-value') def test_creating_already_existing_directory(self): """ INTEGRATION: creating an already existing directory without `prevExist=True` should fail """ self.client.write('/mydir', None, dir=True) self.assertRaises(etcd.EtcdNotFile, self.client.write, '/mydir', None, dir=True) self.assertRaises(etcd.EtcdAlreadyExist, self.client.write, '/mydir', None, dir=True, prevExist=False) class TestClusterFunctions(EtcdIntegrationTest): @classmethod def setUpClass(cls): program = cls._get_exe() cls.directory = tempfile.mkdtemp(prefix='python-etcd') cls.processHelper = helpers.EtcdProcessHelper( cls.directory, proc_name=program, port_range_start=6001, internal_port_range_start=8001, cluster=True) def test_reconnect(self): """ INTEGRATION: get key after the server we're connected fails. """ self.processHelper.stop() self.processHelper.run(number=3) self.client = etcd.Client(port=6001, allow_reconnect=True) set_result = self.client.set('/test_set', 'test-key1') get_result = self.client.get('/test_set') self.assertEquals('test-key1', get_result.value) self.processHelper.kill_one(0) get_result = self.client.get('/test_set') self.assertEquals('test-key1', get_result.value) def test_reconnect_with_several_hosts_passed(self): """ INTEGRATION: receive several hosts at connection setup. """ self.processHelper.stop() self.processHelper.run(number=3) self.client = etcd.Client( host=( ('127.0.0.1', 6004), ('127.0.0.1', 6001)), allow_reconnect=True) set_result = self.client.set('/test_set', 'test-key1') get_result = self.client.get('/test_set') self.assertEquals('test-key1', get_result.value) self.processHelper.kill_one(0) get_result = self.client.get('/test_set') self.assertEquals('test-key1', get_result.value) def test_reconnect_not_allowed(self): """ INTEGRATION: fail on server kill if not allow_reconnect """ self.processHelper.stop() self.processHelper.run(number=3) self.client = etcd.Client(port=6001, allow_reconnect=False) self.processHelper.kill_one(0) self.assertRaises(etcd.EtcdConnectionFailed, self.client.get, '/test_set') def test_reconnet_fails(self): """ INTEGRATION: fails to reconnect if no available machines """ self.processHelper.stop() # Start with three instances (0, 1, 2) self.processHelper.run(number=3) # Connect to instance 0 self.client = etcd.Client(port=6001, allow_reconnect=True) set_result = self.client.set('/test_set', 'test-key1') get_result = self.client.get('/test_set') self.assertEquals('test-key1', get_result.value) self.processHelper.kill_one(2) self.processHelper.kill_one(1) self.processHelper.kill_one(0) self.assertRaises(etcd.EtcdException, self.client.get, '/test_set') class TestWatch(EtcdIntegrationTest): def test_watch(self): """ INTEGRATION: Receive a watch event from other process """ set_result = self.client.set('/test-key', 'test-value') queue = multiprocessing.Queue() def change_value(key, newValue): c = etcd.Client(port=6001) c.set(key, newValue) def watch_value(key, queue): c = etcd.Client(port=6001) queue.put(c.watch(key).value) changer = multiprocessing.Process( target=change_value, args=('/test-key', 'new-test-value',)) watcher = multiprocessing.Process( target=watch_value, args=('/test-key', queue)) watcher.start() time.sleep(1) changer.start() value = queue.get(timeout=2) watcher.join(timeout=5) changer.join(timeout=5) assert value == 'new-test-value' def test_watch_indexed(self): """ INTEGRATION: Receive a watch event from other process, indexed """ set_result = self.client.set('/test-key', 'test-value') set_result = self.client.set('/test-key', 'test-value0') original_index = int(set_result.modifiedIndex) set_result = self.client.set('/test-key', 'test-value1') set_result = self.client.set('/test-key', 'test-value2') queue = multiprocessing.Queue() def change_value(key, newValue): c = etcd.Client(port=6001) c.set(key, newValue) c.get(key) def watch_value(key, index, queue): c = etcd.Client(port=6001) for i in range(0, 3): queue.put(c.watch(key, index=index + i).value) proc = multiprocessing.Process( target=change_value, args=('/test-key', 'test-value3',)) watcher = multiprocessing.Process( target=watch_value, args=('/test-key', original_index, queue)) watcher.start() time.sleep(0.5) proc.start() for i in range(0, 3): value = queue.get() log.debug("index: %d: %s" % (i, value)) self.assertEquals('test-value%d' % i, value) watcher.join(timeout=5) proc.join(timeout=5) def test_watch_generator(self): """ INTEGRATION: Receive a watch event from other process (gen) """ set_result = self.client.set('/test-key', 'test-value') queue = multiprocessing.Queue() def change_value(key): time.sleep(0.5) c = etcd.Client(port=6001) for i in range(0, 3): c.set(key, 'test-value%d' % i) c.get(key) def watch_value(key, queue): c = etcd.Client(port=6001) for i in range(0, 3): event = next(c.eternal_watch(key)).value queue.put(event) changer = multiprocessing.Process( target=change_value, args=('/test-key',)) watcher = multiprocessing.Process( target=watch_value, args=('/test-key', queue)) watcher.start() changer.start() values = ['test-value0', 'test-value1', 'test-value2'] for i in range(0, 1): value = queue.get() log.debug("index: %d: %s" % (i, value)) self.assertTrue(value in values) watcher.join(timeout=5) changer.join(timeout=5) def test_watch_indexed_generator(self): """ INTEGRATION: Receive a watch event from other process, ixd, (2) """ set_result = self.client.set('/test-key', 'test-value') set_result = self.client.set('/test-key', 'test-value0') original_index = int(set_result.modifiedIndex) set_result = self.client.set('/test-key', 'test-value1') set_result = self.client.set('/test-key', 'test-value2') queue = multiprocessing.Queue() def change_value(key, newValue): c = etcd.Client(port=6001) c.set(key, newValue) def watch_value(key, index, queue): c = etcd.Client(port=6001) iterevents = c.eternal_watch(key, index=index) for i in range(0, 3): queue.put(next(iterevents).value) proc = multiprocessing.Process( target=change_value, args=('/test-key', 'test-value3',)) watcher = multiprocessing.Process( target=watch_value, args=('/test-key', original_index, queue)) watcher.start() time.sleep(0.5) proc.start() for i in range(0, 3): value = queue.get() log.debug("index: %d: %s" % (i, value)) self.assertEquals('test-value%d' % i, value) watcher.join(timeout=5) proc.join(timeout=5) python-etcd-0.4.3/src/etcd/tests/integration/test_ssl.py000066400000000000000000000145201263362545500233760ustar00rootroot00000000000000import os import time import shutil import logging import unittest import multiprocessing import tempfile import urllib3 import etcd from . import helpers from . import test_simple from nose.tools import nottest log = logging.getLogger() class TestEncryptedAccess(test_simple.EtcdIntegrationTest): @classmethod def setUpClass(cls): program = cls._get_exe() cls.directory = tempfile.mkdtemp(prefix='python-etcd') cls.ca_cert_path = os.path.join(cls.directory, 'ca.crt') ca_key_path = os.path.join(cls.directory, 'ca.key') cls.ca2_cert_path = os.path.join(cls.directory, 'ca2.crt') ca2_key_path = os.path.join(cls.directory, 'ca2.key') server_cert_path = os.path.join(cls.directory, 'server.crt') server_key_path = os.path.join(cls.directory, 'server.key') ca, ca_key = helpers.TestingCA.create_test_ca_certificate( cls.ca_cert_path, ca_key_path, 'TESTCA') ca2, ca2_key = helpers.TestingCA.create_test_ca_certificate( cls.ca2_cert_path, ca2_key_path, 'TESTCA2') helpers.TestingCA.create_test_certificate( ca, ca_key, server_cert_path, server_key_path, '127.0.0.1') cls.processHelper = helpers.EtcdProcessHelper( cls.directory, proc_name=program, port_range_start=6001, internal_port_range_start=8001, tls=True ) cls.processHelper.run(number=3, proc_args=[ '-cert-file=%s' % server_cert_path, '-key-file=%s' % server_key_path ]) def test_get_set_unauthenticated(self): """ INTEGRATION: set/get a new value unauthenticated (http->https) """ client = etcd.Client(port=6001) # Since python 3 raises a MaxRetryError here, this gets caught in # different code blocks in python 2 and python 3, thus messages are # different. Python 3 does the right thing(TM), for the record self.assertRaises( etcd.EtcdException, client.set, '/test_set', 'test-key') self.assertRaises(etcd.EtcdException, client.get, '/test_set') @nottest def test_get_set_unauthenticated_missing_ca(self): """ INTEGRATION: try unauthenticated w/out validation (https->https)""" # This doesn't work for now and will need further inspection client = etcd.Client(protocol='https', port=6001) set_result = client.set('/test_set', 'test-key') get_result = client.get('/test_set') def test_get_set_unauthenticated_with_ca(self): """ INTEGRATION: try unauthenticated with validation (https->https)""" client = etcd.Client( protocol='https', port=6001, ca_cert=self.ca2_cert_path) self.assertRaises(etcd.EtcdConnectionFailed, client.set, '/test-set', 'test-key') self.assertRaises(etcd.EtcdConnectionFailed, client.get, '/test-set') def test_get_set_authenticated(self): """ INTEGRATION: set/get a new value authenticated """ client = etcd.Client( port=6001, protocol='https', ca_cert=self.ca_cert_path) set_result = client.set('/test_set', 'test-key') get_result = client.get('/test_set') class TestClientAuthenticatedAccess(test_simple.EtcdIntegrationTest): @classmethod def setUpClass(cls): program = cls._get_exe() cls.directory = tempfile.mkdtemp(prefix='python-etcd') cls.ca_cert_path = os.path.join(cls.directory, 'ca.crt') ca_key_path = os.path.join(cls.directory, 'ca.key') server_cert_path = os.path.join(cls.directory, 'server.crt') server_key_path = os.path.join(cls.directory, 'server.key') cls.client_cert_path = os.path.join(cls.directory, 'client.crt') cls.client_key_path = os.path.join(cls.directory, 'client.key') cls.client_all_cert = os.path.join(cls.directory, 'client-all.crt') ca, ca_key = helpers.TestingCA.create_test_ca_certificate( cls.ca_cert_path, ca_key_path) helpers.TestingCA.create_test_certificate( ca, ca_key, server_cert_path, server_key_path, '127.0.0.1') helpers.TestingCA.create_test_certificate( ca, ca_key, cls.client_cert_path, cls.client_key_path) cls.processHelper = helpers.EtcdProcessHelper( cls.directory, proc_name=program, port_range_start=6001, internal_port_range_start=8001, tls=True ) with open(cls.client_all_cert, 'w') as f: with open(cls.client_key_path, 'r') as g: f.write(g.read()) with open(cls.client_cert_path, 'r') as g: f.write(g.read()) cls.processHelper.run(number=3, proc_args=[ '-cert-file=%s' % server_cert_path, '-key-file=%s' % server_key_path, '-ca-file=%s' % cls.ca_cert_path, ]) def test_get_set_unauthenticated(self): """ INTEGRATION: set/get a new value unauthenticated (http->https) """ client = etcd.Client(port=6001) # See above for the reason of this change self.assertRaises( etcd.EtcdException, client.set, '/test_set', 'test-key') self.assertRaises(etcd.EtcdException, client.get, '/test_set') @nottest def test_get_set_authenticated(self): """ INTEGRATION: connecting to server with mutual auth """ # This gives an unexplicable ssl error, as connecting to the same # Etcd cluster where this fails with the exact same code this # doesn't fail client = etcd.Client( port=6001, protocol='https', cert=self.client_all_cert, ca_cert=self.ca_cert_path ) set_result = client.set('/test_set', 'test-key') self.assertEquals(u'set', set_result.action.lower()) self.assertEquals(u'/test_set', set_result.key) self.assertEquals(u'test-key', set_result.value) get_result = client.get('/test_set') self.assertEquals('get', get_result.action.lower()) self.assertEquals('/test_set', get_result.key) self.assertEquals('test-key', get_result.value) python-etcd-0.4.3/src/etcd/tests/test_auth.py000066400000000000000000000122671263362545500212210ustar00rootroot00000000000000from etcd.tests.integration.test_simple import EtcdIntegrationTest from etcd import auth import etcd class TestEtcdAuthBase(EtcdIntegrationTest): cl_size = 1 def setUp(self): # Sets up the root user, toggles auth u = auth.EtcdUser(self.client, 'root') u.password = 'testpass' u.write() self.client = etcd.Client(port=6001, username='root', password='testpass') self.unauth_client = etcd.Client(port=6001) a = auth.Auth(self.client) a.active = True def tearDown(self): u = auth.EtcdUser(self.client, 'test_user') r = auth.EtcdRole(self.client, 'test_role') try: u.delete() except: pass try: r.delete() except: pass a = auth.Auth(self.client) a.active = False class EtcdUserTest(TestEtcdAuthBase): def test_names(self): u = auth.EtcdUser(self.client, 'test_user') self.assertEquals(u.names, ['root']) def test_read(self): u = auth.EtcdUser(self.client, 'root') # Reading an existing user succeeds try: u.read() except Exception: self.fail("reading the root user raised an exception") # roles for said user are fetched self.assertEquals(u.roles, set(['root'])) # The user is correctly rendered out self.assertEquals(u._to_net(), [{'user': 'root', 'password': None, 'roles': ['root']}]) # An inexistent user raises the appropriate exception u = auth.EtcdUser(self.client, 'user.does.not.exist') self.assertRaises(etcd.EtcdKeyNotFound, u.read) # Reading with an unauthenticated client raises an exception u = auth.EtcdUser(self.unauth_client, 'root') self.assertRaises(etcd.EtcdInsufficientPermissions, u.read) # Generic errors are caught c = etcd.Client(port=9999) u = auth.EtcdUser(c, 'root') self.assertRaises(etcd.EtcdException, u.read) def test_write_and_delete(self): # Create an user u = auth.EtcdUser(self.client, 'test_user') u.roles.add('guest') u.roles.add('root') # directly from my suitcase u.password = '123456' try: u.write() except: self.fail("creating a user doesn't work") # Password gets wiped self.assertEquals(u.password, None) u.read() # Verify we can log in as this user and access the auth (it has the # root role) cl = etcd.Client(port=6001, username='test_user', password='123456') ul = auth.EtcdUser(cl, 'root') try: ul.read() except etcd.EtcdInsufficientPermissions: self.fail("Reading auth with the new user is not possible") self.assertEquals(u.name, "test_user") self.assertEquals(u.roles, set(['guest', 'root'])) # set roles as a list, it works! u.roles = ['guest', 'test_group'] try: u.write() except: self.fail("updating a user you previously created fails") u.read() self.assertIn('test_group', u.roles) # Unauthorized access is properly handled ua = auth.EtcdUser(self.unauth_client, 'test_user') self.assertRaises(etcd.EtcdInsufficientPermissions, ua.write) # now let's test deletion du = auth.EtcdUser(self.client, 'user.does.not.exist') self.assertRaises(etcd.EtcdKeyNotFound, du.delete) # Delete test_user u.delete() self.assertRaises(etcd.EtcdKeyNotFound, u.read) # Permissions are properly handled self.assertRaises(etcd.EtcdInsufficientPermissions, ua.delete) class EtcdRoleTest(TestEtcdAuthBase): def test_names(self): r = auth.EtcdRole(self.client, 'guest') self.assertListEqual(r.names, [u'guest', u'root']) def test_read(self): r = auth.EtcdRole(self.client, 'guest') try: r.read() except: self.fail('Reading an existing role failed') self.assertEquals(r.acls, {'*': 'RW'}) # We can actually skip most other read tests as they are common # with EtcdUser def test_write_and_delete(self): r = auth.EtcdRole(self.client, 'test_role') r.acls = {'*': 'R', '/test/*': 'RW'} try: r.write() except: self.fail("Writing a simple groups should not fail") r1 = auth.EtcdRole(self.client, 'test_role') r1.read() self.assertEquals(r1.acls, r.acls) r.revoke('/test/*', 'W') r.write() r1.read() self.assertEquals(r1.acls, {'*': 'R', '/test/*': 'R'}) r.grant('/pub/*', 'RW') r.write() r1.read() self.assertEquals(r1.acls['/pub/*'], 'RW') # All other exceptions are tested by the user tests r1.name = None self.assertRaises(etcd.EtcdException, r1.write) # ditto for delete try: r.delete() except: self.fail("A normal delete should not fail") self.assertRaises(etcd.EtcdKeyNotFound, r.read) python-etcd-0.4.3/src/etcd/tests/unit/000077500000000000000000000000001263362545500176165ustar00rootroot00000000000000python-etcd-0.4.3/src/etcd/tests/unit/__init__.py000066400000000000000000000016141263362545500217310ustar00rootroot00000000000000import etcd import unittest import urllib3 import json try: import mock except ImportError: from unittest import mock class TestClientApiBase(unittest.TestCase): def setUp(self): self.client = etcd.Client() def _prepare_response(self, s, d, cluster_id=None): if isinstance(d, dict): data = json.dumps(d).encode('utf-8') else: data = d.encode('utf-8') r = mock.create_autospec(urllib3.response.HTTPResponse)() r.status = s r.data = data r.getheader.return_value = cluster_id or "abcd1234" return r def _mock_api(self, status, d, cluster_id=None): resp = self._prepare_response(status, d, cluster_id=cluster_id) self.client.api_execute = mock.MagicMock(return_value=resp) def _mock_exception(self, exc, msg): self.client.api_execute = mock.Mock(side_effect=exc(msg)) python-etcd-0.4.3/src/etcd/tests/unit/test_client.py000066400000000000000000000123211263362545500225040ustar00rootroot00000000000000import unittest import etcd import dns.name import dns.rdtypes.IN.SRV import dns.resolver try: import mock except ImportError: from unittest import mock class TestClient(unittest.TestCase): def test_instantiate(self): """ client can be instantiated""" client = etcd.Client() assert client is not None def test_default_host(self): """ default host is 127.0.0.1""" client = etcd.Client() assert client.host == "127.0.0.1" def test_default_port(self): """ default port is 4001""" client = etcd.Client() assert client.port == 4001 def test_default_prefix(self): client = etcd.Client() assert client.version_prefix == '/v2' def test_default_protocol(self): """ default protocol is http""" client = etcd.Client() assert client.protocol == 'http' def test_default_read_timeout(self): """ default read_timeout is 60""" client = etcd.Client() assert client.read_timeout == 60 def test_default_allow_redirect(self): """ default allow_redirect is True""" client = etcd.Client() assert client.allow_redirect def test_default_username(self): """ default username is None""" client = etcd.Client() assert client.username is None def test_default_password(self): """ default username is None""" client = etcd.Client() assert client.password is None def test_set_host(self): """ can change host """ client = etcd.Client(host='192.168.1.1') assert client.host == '192.168.1.1' def test_set_port(self): """ can change port """ client = etcd.Client(port=4002) assert client.port == 4002 def test_set_prefix(self): client = etcd.Client(version_prefix='/etcd') assert client.version_prefix == '/etcd' def test_set_protocol(self): """ can change protocol """ client = etcd.Client(protocol='https') assert client.protocol == 'https' def test_set_read_timeout(self): """ can set read_timeout """ client = etcd.Client(read_timeout=45) assert client.read_timeout == 45 def test_set_allow_redirect(self): """ can change allow_redirect """ client = etcd.Client(allow_redirect=False) assert not client.allow_redirect def test_default_base_uri(self): """ default uri is http://127.0.0.1:4001 """ client = etcd.Client() assert client.base_uri == 'http://127.0.0.1:4001' def test_set_base_uri(self): """ can change base uri """ client = etcd.Client( host='192.168.1.1', port=4003, protocol='https') assert client.base_uri == 'https://192.168.1.1:4003' def test_set_use_proxies(self): """ can set the use_proxies flag """ client = etcd.Client(use_proxies = True) assert client._use_proxies def test_set_username_only(self): client = etcd.Client(username='username') assert client.username is None def test_set_password_only(self): client = etcd.Client(password='password') assert client.password is None def test_set_username_password(self): client = etcd.Client(username='username', password='password') assert client.username == 'username' assert client.password == 'password' def test_get_headers_with_auth(self): client = etcd.Client(username='username', password='password') assert client._get_headers() == { 'authorization': 'Basic dXNlcm5hbWU6cGFzc3dvcmQ=' } def test_get_headers_without_auth(self): client = etcd.Client() assert client._get_headers() == {} def test_allow_reconnect(self): """ Fails if allow_reconnect is false and a list of hosts is given""" with self.assertRaises(etcd.EtcdException): etcd.Client( host=(('localhost', 4001), ('localhost', 4002)), ) # This doesn't raise an exception client = etcd.Client( host=(('localhost', 4001), ('localhost', 4002)), allow_reconnect=True, use_proxies=True, ) def test_discover(self): """Tests discovery.""" answers = [] for i in range(1,3): r = mock.create_autospec(dns.rdtypes.IN.SRV.SRV) r.port = 2379 try: method = dns.name.from_unicode except AttributeError: method = dns.name.from_text r.target = method(u'etcd{}.example.com'.format(i)) answers.append(r) dns.resolver.query = mock.create_autospec(dns.resolver.query, return_value=answers) self.machines = etcd.Client.machines etcd.Client.machines = mock.create_autospec(etcd.Client.machines, return_value=[u'https://etcd2.example.com:2379']) c = etcd.Client(srv_domain="example.com", allow_reconnect=True, protocol="https") etcd.Client.machines = self.machines self.assertEquals(c.host, u'etcd1.example.com') self.assertEquals(c.port, 2379) self.assertEquals(c._machines_cache, [u'https://etcd2.example.com:2379']) python-etcd-0.4.3/src/etcd/tests/unit/test_lock.py000066400000000000000000000145101263362545500221600ustar00rootroot00000000000000import etcd try: import mock except ImportError: from unittest import mock from etcd.tests.unit import TestClientApiBase class TestClientLock(TestClientApiBase): def recursive_read(self): nodes = [ {"key": "/_locks/test_lock/1", "value": "2qwwwq", "modifiedIndex":33,"createdIndex":33}, {"key": "/_locks/test_lock/34", "value": self.locker.uuid, "modifiedIndex":34,"createdIndex":34}, ] d = { "action": "get", "node": {"dir": True, "nodes": [{"key":"/_locks/test_lock", "dir": True, "nodes": nodes}]} } self._mock_api(200, d) def setUp(self): super(TestClientLock, self).setUp() self.locker = etcd.Lock(self.client, 'test_lock') def test_initialization(self): """ Verify the lock gets initialized correctly """ self.assertEquals(self.locker.name, u'test_lock') self.assertEquals(self.locker.path, u'/_locks/test_lock') self.assertEquals(self.locker.is_taken, False) def test_acquire(self): """ Acquiring a precedingly inexistent lock works. """ l = etcd.Lock(self.client, 'test_lock') l._find_lock = mock.MagicMock(spec=l._find_lock, return_value=False) l._acquired = mock.MagicMock(spec=l._acquired, return_value=True) # Mock the write d = { u'action': u'set', u'node': { u'modifiedIndex': 190, u'key': u'/_locks/test_lock/1', u'value': l.uuid } } self._mock_api(200, d) self.assertEquals(l.acquire(), True) self.assertEquals(l._sequence, '1') def test_is_acquired(self): """ Test is_acquired """ self.locker._sequence = '1' d = { u'action': u'get', u'node': { u'modifiedIndex': 190, u'key': u'/_locks/test_lock/1', u'value': self.locker.uuid } } self._mock_api(200, d) self.locker.is_taken = True self.assertEquals(self.locker.is_acquired, True) def test_is_not_acquired(self): """ Test is_acquired failures """ self.locker._sequence = '2' self.locker.is_taken = False self.assertEquals(self.locker.is_acquired, False) self.locker.is_taken = True self._mock_exception(etcd.EtcdKeyNotFound, self.locker.lock_key) self.assertEquals(self.locker.is_acquired, False) self.assertEquals(self.locker.is_taken, False) def test_acquired(self): """ Test the acquiring primitives """ self.locker._sequence = '4' retval = ('/_locks/test_lock/4', None) self.locker._get_locker = mock.MagicMock( spec=self.locker._get_locker, return_value=retval) self.assertTrue(self.locker._acquired()) self.assertTrue(self.locker.is_taken) retval = ('/_locks/test_lock/1', '/_locks/test_lock/4') self.locker._get_locker = mock.MagicMock(return_value=retval) self.assertFalse(self.locker._acquired(blocking=False)) self.assertFalse(self.locker.is_taken) d = { u'action': u'delete', u'node': { u'modifiedIndex': 190, u'key': u'/_locks/test_lock/1', u'value': self.locker.uuid } } self._mock_api(200, d) returns = [('/_locks/test_lock/1', '/_locks/test_lock/4'), ('/_locks/test_lock/4', None)] def side_effect(): return returns.pop() self.locker._get_locker = mock.MagicMock( spec=self.locker._get_locker, side_effect=side_effect) self.assertTrue(self.locker._acquired()) def test_acquired_no_timeout(self): self.locker._sequence = 4 returns = [('/_locks/test_lock/4', None), ('/_locks/test_lock/1', '/_locks/test_lock/4')] def side_effect(): return returns.pop() d = { u'action': u'get', u'node': { u'modifiedIndex': 190, u'key': u'/_locks/test_lock/4', u'value': self.locker.uuid } } self._mock_api(200, d) self.locker._get_locker = mock.create_autospec( self.locker._get_locker, side_effect=side_effect) self.assertTrue(self.locker._acquired()) def test_lock_key(self): """ Test responses from the lock_key property """ with self.assertRaises(ValueError): self.locker.lock_key self.locker._sequence = '5' self.assertEquals(u'/_locks/test_lock/5',self.locker.lock_key) def test_set_sequence(self): self.locker._set_sequence('/_locks/test_lock/10') self.assertEquals('10', self.locker._sequence) def test_find_lock(self): d = { u'action': u'get', u'node': { u'modifiedIndex': 190, u'key': u'/_locks/test_lock/1', u'value': self.locker.uuid } } self._mock_api(200, d) self.locker._sequence = '1' self.assertTrue(self.locker._find_lock()) # Now let's pretend the lock is not there self._mock_exception(etcd.EtcdKeyNotFound, self.locker.lock_key) self.assertFalse(self.locker._find_lock()) self.locker._sequence = None self.recursive_read() self.assertTrue(self.locker._find_lock()) self.assertEquals(self.locker._sequence, '34') def test_get_locker(self): self.recursive_read() self.assertEquals((u'/_locks/test_lock/1', u'/_locks/test_lock/1'), self.locker._get_locker()) with self.assertRaises(etcd.EtcdLockExpired): self.locker._sequence = '35' self.locker._get_locker() def test_release(self): d = { u'action': u'delete', u'node': { u'modifiedIndex': 190, u'key': u'/_locks/test_lock/1', u'value': self.locker.uuid } } self._mock_api(200, d) self.locker._sequence = 1 self.locker.is_taken = True self.locker.release() self.assertFalse(self.locker.is_taken) python-etcd-0.4.3/src/etcd/tests/unit/test_old_request.py000066400000000000000000000345201263362545500235610ustar00rootroot00000000000000import etcd import unittest try: import mock except ImportError: from unittest import mock from etcd import EtcdException class FakeHTTPResponse(object): def __init__(self, status, data='', headers=None): self.status = status self.data = data.encode('utf-8') self.headers = headers or { "x-etcd-cluster-id": "abdef12345", } def getheaders(self): return self.headers def getheader(self, header): return self.headers[header] class TestClientRequest(unittest.TestCase): def test_set(self): """ Can set a value """ client = etcd.Client() client.api_execute = mock.Mock( return_value=FakeHTTPResponse(201, '{"action":"SET",' '"node": {' '"key":"/testkey",' '"value":"test",' '"newKey":true,' '"expiration":"2013-09-14T00:56:59.316195568+02:00",' '"ttl":19,"modifiedIndex":183}}') ) result = client.set('/testkey', 'test', ttl=19) self.assertEquals( etcd.EtcdResult( **{u'action': u'SET', 'node': { u'expiration': u'2013-09-14T00:56:59.316195568+02:00', u'modifiedIndex': 183, u'key': u'/testkey', u'newKey': True, u'ttl': 19, u'value': u'test'}}), result) def test_test_and_set(self): """ Can test and set a value """ client = etcd.Client() client.api_execute = mock.Mock( return_value=FakeHTTPResponse(200, '{"action":"SET",' '"node": {' '"key":"/testkey",' '"prevValue":"test",' '"value":"newvalue",' '"expiration":"2013-09-14T02:09:44.24390976+02:00",' '"ttl":49,"modifiedIndex":203}}') ) result = client.test_and_set('/testkey', 'newvalue', 'test', ttl=19) self.assertEquals( etcd.EtcdResult( **{u'action': u'SET', u'node': { u'expiration': u'2013-09-14T02:09:44.24390976+02:00', u'modifiedIndex': 203, u'key': u'/testkey', u'prevValue': u'test', u'ttl': 49, u'value': u'newvalue'} }), result) def test_test_and_test_failure(self): """ Exception will be raised if prevValue != value in test_set """ client = etcd.Client() client.api_execute = mock.Mock( side_effect=ValueError( 'The given PrevValue is not equal' ' to the value of the key : TestAndSet: 1!=3')) try: result = client.test_and_set( '/testkey', 'newvalue', 'test', ttl=19) except ValueError as e: #from ipdb import set_trace; set_trace() self.assertEquals( 'The given PrevValue is not equal' ' to the value of the key : TestAndSet: 1!=3', str(e)) def test_delete(self): """ Can delete a value """ client = etcd.Client() client.api_execute = mock.Mock( return_value=FakeHTTPResponse(200, '{"action":"DELETE",' '"node": {' '"key":"/testkey",' '"prevValue":"test",' '"expiration":"2013-09-14T01:06:35.5242587+02:00",' '"modifiedIndex":189}}') ) result = client.delete('/testkey') self.assertEquals(etcd.EtcdResult( **{u'action': u'DELETE', u'node': { u'expiration': u'2013-09-14T01:06:35.5242587+02:00', u'modifiedIndex': 189, u'key': u'/testkey', u'prevValue': u'test'} }), result) def test_get(self): """ Can get a value """ client = etcd.Client() client.api_execute = mock.Mock( return_value=FakeHTTPResponse(200, '{"action":"GET",' '"node": {' '"key":"/testkey",' '"value":"test",' '"modifiedIndex":190}}') ) result = client.get('/testkey') self.assertEquals(etcd.EtcdResult( **{u'action': u'GET', u'node': { u'modifiedIndex': 190, u'key': u'/testkey', u'value': u'test'} }), result) def test_get_multi(self): """Can get multiple values""" pass def test_get_subdirs(self): """ Can understand dirs in results """ pass def test_not_in(self): """ Can check if key is not in client """ client = etcd.Client() client.get = mock.Mock(side_effect=etcd.EtcdKeyNotFound()) result = '/testkey' not in client self.assertEquals(True, result) def test_in(self): """ Can check if key is in client """ client = etcd.Client() client.api_execute = mock.Mock( return_value=FakeHTTPResponse(200, '{"action":"GET",' '"node": {' '"key":"/testkey",' '"value":"test",' '"modifiedIndex":190}}') ) result = '/testkey' in client self.assertEquals(True, result) def test_simple_watch(self): """ Can watch values """ client = etcd.Client() client.api_execute = mock.Mock( return_value=FakeHTTPResponse(200, '{"action":"SET",' '"node": {' '"key":"/testkey",' '"value":"test",' '"newKey":true,' '"expiration":"2013-09-14T01:35:07.623681365+02:00",' '"ttl":19,' '"modifiedIndex":192}}') ) result = client.watch('/testkey') self.assertEquals( etcd.EtcdResult( **{u'action': u'SET', u'node': { u'expiration': u'2013-09-14T01:35:07.623681365+02:00', u'modifiedIndex': 192, u'key': u'/testkey', u'newKey': True, u'ttl': 19, u'value': u'test'} }), result) def test_index_watch(self): """ Can watch values from index """ client = etcd.Client() client.api_execute = mock.Mock( return_value=FakeHTTPResponse(200, '{"action":"SET",' '"node": {' '"key":"/testkey",' '"value":"test",' '"newKey":true,' '"expiration":"2013-09-14T01:35:07.623681365+02:00",' '"ttl":19,' '"modifiedIndex":180}}') ) result = client.watch('/testkey', index=180) self.assertEquals( etcd.EtcdResult( **{u'action': u'SET', u'node': { u'expiration': u'2013-09-14T01:35:07.623681365+02:00', u'modifiedIndex': 180, u'key': u'/testkey', u'newKey': True, u'ttl': 19, u'value': u'test'} }), result) class TestEventGenerator(object): def check_watch(self, result): assert etcd.EtcdResult( **{u'action': u'SET', u'node': { u'expiration': u'2013-09-14T01:35:07.623681365+02:00', u'modifiedIndex': 180, u'key': u'/testkey', u'newKey': True, u'ttl': 19, u'value': u'test'} }) == result def test_eternal_watch(self): """ Can watch values from generator """ client = etcd.Client() client.api_execute = mock.Mock( return_value=FakeHTTPResponse(200, '{"action":"SET",' '"node": {' '"key":"/testkey",' '"value":"test",' '"newKey":true,' '"expiration":"2013-09-14T01:35:07.623681365+02:00",' '"ttl":19,' '"modifiedIndex":180}}') ) for result in range(1, 5): result = next(client.eternal_watch('/testkey', index=180)) yield self.check_watch, result class TestClientApiExecutor(unittest.TestCase): def test_get(self): """ http get request """ client = etcd.Client() response = FakeHTTPResponse(status=200, data='arbitrary json data') client.http.request = mock.Mock(return_value=response) result = client.api_execute('/v1/keys/testkey', client._MGET) self.assertEquals('arbitrary json data'.encode('utf-8'), result.data) def test_delete(self): """ http delete request """ client = etcd.Client() response = FakeHTTPResponse(status=200, data='arbitrary json data') client.http.request = mock.Mock(return_value=response) result = client.api_execute('/v1/keys/testkey', client._MDELETE) self.assertEquals('arbitrary json data'.encode('utf-8'), result.data) def test_get_error(self): """ http get error request 101""" client = etcd.Client() response = FakeHTTPResponse(status=400, data='{"message": "message",' ' "cause": "cause",' ' "errorCode": 100}') client.http.request = mock.Mock(return_value=response) try: client.api_execute('/v2/keys/testkey', client._MGET) assert False except etcd.EtcdKeyNotFound as e: self.assertEquals(str(e), 'message : cause') def test_put(self): """ http put request """ client = etcd.Client() response = FakeHTTPResponse(status=200, data='arbitrary json data') client.http.request_encode_body = mock.Mock(return_value=response) result = client.api_execute('/v2/keys/testkey', client._MPUT) self.assertEquals('arbitrary json data'.encode('utf-8'), result.data) def test_test_and_set_error(self): """ http post error request 101 """ client = etcd.Client() response = FakeHTTPResponse( status=400, data='{"message": "message", "cause": "cause", "errorCode": 101}') client.http.request_encode_body = mock.Mock(return_value=response) payload = {'value': 'value', 'prevValue': 'oldValue', 'ttl': '60'} try: client.api_execute('/v2/keys/testkey', client._MPUT, payload) self.fail() except ValueError as e: self.assertEquals('message : cause', str(e)) def test_set_not_file_error(self): """ http post error request 102 """ client = etcd.Client() response = FakeHTTPResponse( status=400, data='{"message": "message", "cause": "cause", "errorCode": 102}') client.http.request_encode_body = mock.Mock(return_value=response) payload = {'value': 'value', 'prevValue': 'oldValue', 'ttl': '60'} try: client.api_execute('/v2/keys/testkey', client._MPUT, payload) self.fail() except etcd.EtcdNotFile as e: self.assertEquals('message : cause', str(e)) def test_get_error_unknown(self): """ http get error request unknown """ client = etcd.Client() response = FakeHTTPResponse(status=400, data='{"message": "message",' ' "cause": "cause",' ' "errorCode": 42}') client.http.request = mock.Mock(return_value=response) try: client.api_execute('/v2/keys/testkey', client._MGET) self.fail() except etcd.EtcdException as e: self.assertEqual(str(e), "message : cause") def test_get_error_request_invalid(self): """ http get error request invalid """ client = etcd.Client() response = FakeHTTPResponse(status=400, data='{)*garbage') client.http.request = mock.Mock(return_value=response) try: client.api_execute('/v2/keys/testkey', client._MGET) self.fail() except etcd.EtcdException as e: self.assertEqual(str(e), "Bad response : {)*garbage") def test_get_error_invalid(self): """ http get error request invalid """ client = etcd.Client() response = FakeHTTPResponse(status=400, data='{){){)*garbage*') client.http.request = mock.Mock(return_value=response) self.assertRaises(etcd.EtcdException, client.api_execute, '/v2/keys/testkey', client._MGET) python-etcd-0.4.3/src/etcd/tests/unit/test_request.py000066400000000000000000000420071263362545500227220ustar00rootroot00000000000000import socket import urllib3 import etcd from etcd.tests.unit import TestClientApiBase try: import mock except ImportError: from unittest import mock class TestClientApiInternals(TestClientApiBase): def test_read_default_timeout(self): """ Read timeout set to the default """ d = { u'action': u'get', u'node': { u'modifiedIndex': 190, u'key': u'/testkey', u'value': u'test' } } self._mock_api(200, d) res = self.client.read('/testkey') self.assertEqual(self.client.api_execute.call_args[1]['timeout'], None) def test_read_custom_timeout(self): """ Read timeout set to the supplied value """ d = { u'action': u'get', u'node': { u'modifiedIndex': 190, u'key': u'/testkey', u'value': u'test' } } self._mock_api(200, d) self.client.read('/testkey', timeout=15) self.assertEqual(self.client.api_execute.call_args[1]['timeout'], 15) def test_read_no_timeout(self): """ Read timeout disabled """ d = { u'action': u'get', u'node': { u'modifiedIndex': 190, u'key': u'/testkey', u'value': u'test' } } self._mock_api(200, d) self.client.read('/testkey', timeout=0) self.assertEqual(self.client.api_execute.call_args[1]['timeout'], 0) def test_write_no_params(self): """ Calling `write` without a value argument will omit the `value` from the API call params """ d = { u'action': u'set', u'node': { u'createdIndex': 17, u'dir': True, u'key': u'/newdir', u'modifiedIndex': 17 } } self._mock_api(200, d) self.client.write('/newdir', None, dir=True) self.assertEquals(self.client.api_execute.call_args, (('/v2/keys/newdir', 'PUT'), dict(params={'dir': 'true'}))) class TestClientApiInterface(TestClientApiBase): """ All tests defined in this class are executed also in TestClientRequest. If a test should be run only in this class, please override the method there. """ @mock.patch('urllib3.request.RequestMethods.request') def test_machines(self, mocker): """ Can request machines """ data = ['http://127.0.0.1:4001', 'http://127.0.0.1:4002', 'http://127.0.0.1:4003'] d = ','.join(data) mocker.return_value = self._prepare_response(200, d) self.assertEquals(data, self.client.machines) @mock.patch('etcd.Client.machines', new_callable=mock.PropertyMock) def test_use_proxies(self, mocker): """Do not overwrite the machines cache when using proxies""" mocker.return_value = ['https://10.0.0.2:4001', 'https://10.0.0.3:4001', 'https://10.0.0.4:4001'] c = etcd.Client( host=(('localhost', 4001), ('localproxy', 4001)), protocol='https', allow_reconnect=True, use_proxies=True ) self.assertEquals(c._machines_cache, ['https://localproxy:4001']) self.assertEquals(c._base_uri, 'https://localhost:4001') self.assertNotIn(c.base_uri, c._machines_cache) c = etcd.Client( host=(('localhost', 4001), ('10.0.0.2', 4001)), protocol='https', allow_reconnect=True, use_proxies=False ) self.assertIn('https://10.0.0.3:4001', c._machines_cache) self.assertNotIn(c.base_uri, c._machines_cache) def test_members(self): """ Can request machines """ data = { "members": [ { "id": "ce2a822cea30bfca", "name": "default", "peerURLs": ["http://localhost:2380", "http://localhost:7001"], "clientURLs": ["http://127.0.0.1:4001"] } ] } self._mock_api(200, data) self.assertEquals(self.client.members["ce2a822cea30bfca"]["id"], "ce2a822cea30bfca") def test_self_stats(self): """ Request for stats """ data = { "id": "eca0338f4ea31566", "leaderInfo": { "leader": "8a69d5f6b7814500", "startTime": "2014-10-24T13:15:51.186620747-07:00", "uptime": "10m59.322358947s" }, "name": "node3", "recvAppendRequestCnt": 5944, "recvBandwidthRate": 570.6254930219969, "recvPkgRate": 9.00892789741075, "sendAppendRequestCnt": 0, "startTime": "2014-10-24T13:15:50.072007085-07:00", "state": "StateFollower" } self._mock_api(200,data) self.assertEquals(self.client.stats['name'], "node3") def test_leader_stats(self): """ Request for leader stats """ data = {"leader": "924e2e83e93f2560", "followers": {}} self._mock_api(200,data) self.assertEquals(self.client.leader_stats['leader'], "924e2e83e93f2560") @mock.patch('etcd.Client.members', new_callable=mock.PropertyMock) def test_leader(self, mocker): """ Can request the leader """ members = {"ce2a822cea30bfca": {"id": "ce2a822cea30bfca", "name": "default"}} mocker.return_value = members self._mock_api(200, {"leaderInfo":{"leader": "ce2a822cea30bfca", "followers": {}}}) self.assertEquals(self.client.leader, members["ce2a822cea30bfca"]) def test_set_plain(self): """ Can set a value """ d = {u'action': u'set', u'node': { u'expiration': u'2013-09-14T00:56:59.316195568+02:00', u'modifiedIndex': 183, u'key': u'/testkey', u'ttl': 19, u'value': u'test' } } self._mock_api(200, d) res = self.client.write('/testkey', 'test') self.assertEquals(res, etcd.EtcdResult(**d)) def test_update(self): """Can update a result.""" d = {u'action': u'set', u'node': { u'expiration': u'2013-09-14T00:56:59.316195568+02:00', u'modifiedIndex': 6, u'key': u'/testkey', u'ttl': 19, u'value': u'test' } } self._mock_api(200,d) res = self.client.get('/testkey') res.value = 'ciao' d['node']['value'] = 'ciao' self._mock_api(200,d) newres = self.client.update(res) self.assertEquals(newres.value, 'ciao') def test_newkey(self): """ Can set a new value """ d = { u'action': u'set', u'node': { u'expiration': u'2013-09-14T00:56:59.316195568+02:00', u'modifiedIndex': 183, u'key': u'/testkey', u'ttl': 19, u'value': u'test' } } self._mock_api(201, d) res = self.client.write('/testkey', 'test') d['node']['newKey'] = True self.assertEquals(res, etcd.EtcdResult(**d)) def test_not_found_response(self): """ Can handle server not found response """ self._mock_api(404, 'Not found') self.assertRaises(etcd.EtcdException, self.client.read, '/somebadkey') def test_compare_and_swap(self): """ Can set compare-and-swap a value """ d = {u'action': u'compareAndSwap', u'node': { u'expiration': u'2013-09-14T00:56:59.316195568+02:00', u'modifiedIndex': 183, u'key': u'/testkey', u'ttl': 19, u'value': u'test' } } self._mock_api(200, d) res = self.client.write('/testkey', 'test', prevValue='test_old') self.assertEquals(res, etcd.EtcdResult(**d)) def test_compare_and_swap_failure(self): """ Exception will be raised if prevValue != value in test_set """ self._mock_exception(ValueError, 'Test Failed : [ 1!=3 ]') self.assertRaises( ValueError, self.client.write, '/testKey', 'test', prevValue='oldbog' ) def test_set_append(self): """ Can append a new key """ d = { u'action': u'create', u'node': { u'createdIndex': 190, u'modifiedIndex': 190, u'key': u'/testdir/190', u'value': u'test' } } self._mock_api(201, d) res = self.client.write('/testdir', 'test') self.assertEquals(res.createdIndex, 190) def test_set_dir_with_value(self): """ Creating a directory with a value raises an error. """ self.assertRaises(etcd.EtcdException, self.client.write, '/bar', 'testvalye', dir=True) def test_delete(self): """ Can delete a value """ d = { u'action': u'delete', u'node': { u'key': u'/testkey', "modifiedIndex": 3, "createdIndex": 2 } } self._mock_api(200, d) res = self.client.delete('/testKey') self.assertEquals(res, etcd.EtcdResult(**d)) def test_pop(self): """ Can pop a value """ d = { u'action': u'delete', u'node': { u'key': u'/testkey', u'modifiedIndex': 3, u'createdIndex': 2 }, u'prevNode': {u'newKey': False, u'createdIndex': None, u'modifiedIndex': 190, u'value': u'test', u'expiration': None, u'key': u'/testkey', u'ttl': None, u'dir': False} } self._mock_api(200, d) res = self.client.pop(d['node']['key']) self.assertEquals({attr: getattr(res, attr) for attr in dir(res) if attr in etcd.EtcdResult._node_props}, d['prevNode']) self.assertEqual(res.value, d['prevNode']['value']) def test_read(self): """ Can get a value """ d = { u'action': u'get', u'node': { u'modifiedIndex': 190, u'key': u'/testkey', u'value': u'test' } } self._mock_api(200, d) res = self.client.read('/testKey') self.assertEquals(res, etcd.EtcdResult(**d)) def test_get_dir(self): """Can get values in dirs""" d = { u'action': u'get', u'node': { u'modifiedIndex': 190, u'key': u'/testkey', u'dir': True, u'nodes': [ { u'key': u'/testDir/testKey', u'modifiedIndex': 150, u'value': 'test' }, { u'key': u'/testDir/testKey2', u'modifiedIndex': 190, u'value': 'test2' } ] } } self._mock_api(200, d) res = self.client.read('/testDir', recursive=True) self.assertEquals(res, etcd.EtcdResult(**d)) def test_not_in(self): """ Can check if key is not in client """ self._mock_exception(etcd.EtcdKeyNotFound, 'Key not Found : /testKey') self.assertTrue('/testey' not in self.client) def test_in(self): """ Can check if key is not in client """ d = { u'action': u'get', u'node': { u'modifiedIndex': 190, u'key': u'/testkey', u'value': u'test' } } self._mock_api(200, d) self.assertTrue('/testey' in self.client) def test_watch(self): """ Can watch a key """ d = { u'action': u'get', u'node': { u'modifiedIndex': 190, u'key': u'/testkey', u'value': u'test' } } self._mock_api(200, d) res = self.client.read('/testkey', wait=True) self.assertEquals(res, etcd.EtcdResult(**d)) def test_watch_index(self): """ Can watch a key starting from the given Index """ d = { u'action': u'get', u'node': { u'modifiedIndex': 170, u'key': u'/testkey', u'value': u'testold' } } self._mock_api(200, d) res = self.client.read('/testkey', wait=True, waitIndex=True) self.assertEquals(res, etcd.EtcdResult(**d)) class TestClientRequest(TestClientApiInterface): def setUp(self): self.client = etcd.Client(expected_cluster_id="abcdef1234") def _mock_api(self, status, d, cluster_id=None): resp = self._prepare_response(status, d) resp.getheader.return_value = cluster_id or "abcdef1234" self.client.http.request_encode_body = mock.MagicMock( return_value=resp) self.client.http.request = mock.MagicMock(return_value=resp) def _mock_error(self, error_code, msg, cause, method='PUT', fields=None, cluster_id=None): resp = self._prepare_response( 500, {'errorCode': error_code, 'message': msg, 'cause': cause} ) resp.getheader.return_value = cluster_id or "abcdef1234" self.client.http.request_encode_body = mock.create_autospec( self.client.http.request_encode_body, return_value=resp ) self.client.http.request = mock.create_autospec( self.client.http.request, return_value=resp ) def test_compare_and_swap_failure(self): """ Exception will be raised if prevValue != value in test_set """ self._mock_error(200, 'Test Failed', '[ 1!=3 ]', fields={'prevValue': 'oldbog'}) self.assertRaises( ValueError, self.client.write, '/testKey', 'test', prevValue='oldbog' ) def test_watch_timeout(self): """ Exception will be raised if prevValue != value in test_set """ self.client.http.request = mock.create_autospec( self.client.http.request, side_effect=urllib3.exceptions.ReadTimeoutError(self.client.http, "foo", "Read timed out") ) self.assertRaises( etcd.EtcdWatchTimedOut, self.client.watch, '/testKey', ) def test_path_without_trailing_slash(self): """ Exception will be raised if a path without a trailing slash is used """ self.assertRaises(ValueError, self.client.api_execute, 'testpath/bar', self.client._MPUT) def test_api_method_not_supported(self): """ Exception will be raised if an unsupported HTTP method is used """ self.assertRaises(etcd.EtcdException, self.client.api_execute, '/testpath/bar', 'TRACE') def test_read_cluster_id_changed(self): """ Read timeout set to the default """ d = { u'action': u'set', u'node': { u'expiration': u'2013-09-14T00:56:59.316195568+02:00', u'modifiedIndex': 6, u'key': u'/testkey', u'ttl': 19, u'value': u'test', } } self._mock_api(200, d, cluster_id="notabcd1234") self.assertRaises(etcd.EtcdClusterIdChanged, self.client.read, '/testkey') self.client.read("/testkey") def test_read_connection_error(self): self.client.http.request = mock.create_autospec( self.client.http.request, side_effect=socket.error() ) self.assertRaises(etcd.EtcdConnectionFailed, self.client.read, '/something') # Direct GET request self.assertRaises(etcd.EtcdConnectionFailed, self.client.api_execute, '/a', 'GET') def test_not_in(self): pass def test_in(self): pass def test_update_fails(self): """ Non-atomic updates fail """ d = { u'action': u'set', u'node': { u'expiration': u'2013-09-14T00:56:59.316195568+02:00', u'modifiedIndex': 6, u'key': u'/testkey', u'ttl': 19, u'value': u'test' } } res = etcd.EtcdResult(**d) error = { "errorCode": 101, "message": "Compare failed", "cause": "[ != bar] [7 != 6]", "index": 6} self._mock_api(412, error) res.value = 'bar' self.assertRaises(ValueError, self.client.update, res) python-etcd-0.4.3/src/etcd/tests/unit/test_result.py000066400000000000000000000111271263362545500225470ustar00rootroot00000000000000import etcd import unittest import json import urllib3 try: import mock except ImportError: from unittest import mock class TestEtcdResult(unittest.TestCase): def test_get_subtree_1_level(self): """ Test get_subtree() for a read with tree 1 level deep. """ response = {"node": { 'key': "/test", 'value': "hello", 'expiration': None, 'ttl': None, 'modifiedIndex': 5, 'createdIndex': 1, 'newKey': False, 'dir': False, }} result = etcd.EtcdResult(**response) self.assertEqual(result.key, response["node"]["key"]) self.assertEqual(result.value, response["node"]["value"]) # Get subtree returns itself, whether or not leaves_only subtree = list(result.get_subtree(leaves_only=True)) self.assertListEqual([result], subtree) subtree = list(result.get_subtree(leaves_only=False)) self.assertListEqual([result], subtree) def test_get_subtree_2_level(self): """ Test get_subtree() for a read with tree 2 levels deep. """ leaf0 = { 'key': "/test/leaf0", 'value': "hello1", 'expiration': None, 'ttl': None, 'modifiedIndex': 5, 'createdIndex': 1, 'newKey': False, 'dir': False, } leaf1 = { 'key': "/test/leaf1", 'value': "hello2", 'expiration': None, 'ttl': None, 'modifiedIndex': 6, 'createdIndex': 2, 'newKey': False, 'dir': False, } testnode = {"node": { 'key': "/test/", 'expiration': None, 'ttl': None, 'modifiedIndex': 6, 'createdIndex': 2, 'newKey': False, 'dir': True, 'nodes': [leaf0, leaf1] }} result = etcd.EtcdResult(**testnode) self.assertEqual(result.key, "/test/") self.assertTrue(result.dir) # Get subtree returns just two leaves for leaves only. subtree = list(result.get_subtree(leaves_only=True)) self.assertEqual(subtree[0].key, "/test/leaf0") self.assertEqual(subtree[1].key, "/test/leaf1") self.assertEqual(len(subtree), 2) # Get subtree returns leaves and directory. subtree = list(result.get_subtree(leaves_only=False)) self.assertEqual(subtree[0].key, "/test/") self.assertEqual(subtree[1].key, "/test/leaf0") self.assertEqual(subtree[2].key, "/test/leaf1") self.assertEqual(len(subtree), 3) def test_get_subtree_3_level(self): """ Test get_subtree() for a read with tree 3 levels deep. """ leaf0 = { 'key': "/test/mid0/leaf0", 'value': "hello1", } leaf1 = { 'key': "/test/mid0/leaf1", 'value': "hello2", } leaf2 = { 'key': "/test/mid1/leaf2", 'value': "hello1", } leaf3 = { 'key': "/test/mid1/leaf3", 'value': "hello2", } mid0 = { 'key': "/test/mid0/", 'dir': True, 'nodes': [leaf0, leaf1] } mid1 = { 'key': "/test/mid1/", 'dir': True, 'nodes': [leaf2, leaf3] } testnode = {"node": { 'key': "/test/", 'dir': True, 'nodes': [mid0, mid1] }} result = etcd.EtcdResult(**testnode) self.assertEqual(result.key, "/test/") self.assertTrue(result.dir) # Get subtree returns just two leaves for leaves only. subtree = list(result.get_subtree(leaves_only=True)) self.assertEqual(subtree[0].key, "/test/mid0/leaf0") self.assertEqual(subtree[1].key, "/test/mid0/leaf1") self.assertEqual(subtree[2].key, "/test/mid1/leaf2") self.assertEqual(subtree[3].key, "/test/mid1/leaf3") self.assertEqual(len(subtree), 4) # Get subtree returns leaves and directory. subtree = list(result.get_subtree(leaves_only=False)) self.assertEqual(subtree[0].key, "/test/") self.assertEqual(subtree[1].key, "/test/mid0/") self.assertEqual(subtree[2].key, "/test/mid0/leaf0") self.assertEqual(subtree[3].key, "/test/mid0/leaf1") self.assertEqual(subtree[4].key, "/test/mid1/") self.assertEqual(subtree[5].key, "/test/mid1/leaf2") self.assertEqual(subtree[6].key, "/test/mid1/leaf3") self.assertEqual(len(subtree), 7)