mockupdb-1.3.0/0000755000076700000240000000000013242634066015031 5ustar emptysquarestaff00000000000000mockupdb-1.3.0/AUTHORS.rst0000644000076700000240000000024312474506007016706 0ustar emptysquarestaff00000000000000======= Credits ======= Development Lead ---------------- * A. Jesse Jiryu Davis Contributors ------------ None yet. Why not be the first? mockupdb-1.3.0/CHANGELOG.rst0000644000076700000240000000340113242634017017044 0ustar emptysquarestaff00000000000000.. :changelog: Changelog ========= 1.3.0 (2018-02-19) ------------------ Support Windows. Log a traceback if a bad client request causes an assert. Fix SSL. Make errors less likely on shutdown. Enable testing on Travis and Appveyor. Fix doctests and interactive server for modern MongoDB protocol. 1.2.1 (2017-12-06) ------------------ Set minWireVersion to 0, not to 2. I had been wrong about MongoDB 3.6's wire version range: it's actually 0 to 6. MockupDB now reports the same wire version range as MongoDB 3.6 by default. 1.2.0 (2017-09-22) ------------------ Update for MongoDB 3.6: report minWireVersion 2 and maxWireVersion 6 by default. 1.1.3 (2017-04-23) ------------------ Avoid rare RuntimeError in close(), if a client thread shuts down a socket as MockupDB iterates its list of sockets. 1.1.2 (2016-08-23) ------------------ Properly detect closed sockets so ``MockupDB.stop()`` doesn't take 10 seconds per connection. Thanks to Sean Purcell. 1.1.1 (2016-08-01) ------------------ Don't use "client" as a keyword arg for ``Request``, it conflicts with the actual "client" field in drivers' new handshake protocol. 1.1.0 (2016-02-11) ------------------ Add cursor_id property to OpGetMore, and ssl parameter to interactive_server. 1.0.3 (2015-09-12) ------------------ ``MockupDB(auto_ismaster=True)`` had just responded ``{"ok": 1}``, but this isn't enough to convince PyMongo 3 it's talking to a valid standalone, so auto-respond ``{"ok": 1, "ismaster": True}``. 1.0.2 (2015-09-11) ------------------ Restore Request.assert_matches method, used in pymongo-mockup-tests. 1.0.1 (2015-09-11) ------------------ Allow co-installation with PyMongo. 1.0.0 (2015-09-10) ------------------ First release. 0.1.0 (2015-02-25) ------------------ Development begun. mockupdb-1.3.0/CONTRIBUTING.rst0000644000076700000240000000605312574434213017475 0ustar emptysquarestaff00000000000000============ Contributing ============ Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given. You can contribute in many ways: Types of Contributions ---------------------- Report Bugs ~~~~~~~~~~~ Report bugs at https://github.com/ajdavis/mongo-mockup-db/issues. If you are reporting a bug, please include: * Your operating system name and version. * Any details about your local setup that might be helpful in troubleshooting. * Detailed steps to reproduce the bug. Fix Bugs ~~~~~~~~ Look through the GitHub issues for bugs. Anything tagged with "bug" is open to whoever wants to implement it. Implement Features ~~~~~~~~~~~~~~~~~~ Look through the GitHub issues for features. Anything tagged with "feature" is open to whoever wants to implement it. Write Documentation ~~~~~~~~~~~~~~~~~~~ MockupDB could always use more documentation, whether as part of the official MockupDB docs, in docstrings, or even on the web in blog posts, articles, and such. Submit Feedback ~~~~~~~~~~~~~~~ The best way to send feedback is to file an issue at https://github.com/ajdavis/mongo-mockup-db/issues. If you are proposing a feature: * Explain in detail how it would work. * Keep the scope as narrow as possible, to make it easier to implement. * Remember that this is a volunteer-driven project, and that contributions are welcome :) Get Started! ------------ Ready to contribute? Here's how to set up MockupDB for local development. 1. Fork the `mongo-mockup-db` repo on GitHub. 2. Clone your fork locally:: $ git clone git@github.com:your_name_here/mongo-mockup-db.git 3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:: $ mkvirtualenv mongo-mockup-db $ cd mongo-mockup-db/ $ python setup.py develop 4. Create a branch for local development:: $ git checkout -b name-of-your-bugfix-or-feature Now you can make your changes locally. 5. When you're done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox:: $ flake8 mockupdb tests $ python setup.py test $ tox To get flake8 and tox, just pip install them into your virtualenv. 6. Commit your changes and push your branch to GitHub:: $ git add . $ git commit -m "Your detailed description of your changes." $ git push origin name-of-your-bugfix-or-feature 7. Submit a pull request through the GitHub website. Pull Request Guidelines ----------------------- Before you submit a pull request, check that it meets these guidelines: 1. The pull request should include tests. 2. If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring, and add the feature to the list in README.rst. 3. The pull request should work for Python 2.6, 2.7, 3.3, and 3.4. Check that tests pass in all versions with `tox`. Tips ---- To run a subset of tests:: $ python setup.py test -s tests.test_mockupdb mockupdb-1.3.0/docs/0000755000076700000240000000000013242634066015761 5ustar emptysquarestaff00000000000000mockupdb-1.3.0/docs/authors.rst0000644000076700000240000000003412474506007020174 0ustar emptysquarestaff00000000000000.. include:: ../AUTHORS.rst mockupdb-1.3.0/docs/changelog.rst0000644000076700000240000000003612474512511020435 0ustar emptysquarestaff00000000000000.. include:: ../CHANGELOG.rst mockupdb-1.3.0/docs/conf.py0000755000076700000240000002142213242633741017263 0ustar emptysquarestaff00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # # mongo-mockup-db documentation build configuration file, created by # sphinx-quickstart on Tue Jul 9 22:26:36 2013. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys import os # If extensions (or modules to document with autodoc) are in another # directory, add these directories to sys.path here. If the directory is # relative to the documentation root, use os.path.abspath to make it # absolute, like shown here. #sys.path.insert(0, os.path.abspath('.')) # Get the project root dir, which is the parent dir of this cwd = os.getcwd() project_root = os.path.dirname(cwd) # Insert the project root dir as the first element in the PYTHONPATH. # This lets us ensure that the source package is imported, and that its # version is used. sys.path.insert(0, project_root) import mockupdb # -- General configuration --------------------------------------------- # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.coverage', 'sphinx.ext.todo', 'sphinx.ext.intersphinx', ] intersphinx_mapping = { 'python': ('https://docs.python.org/3/', None), 'pymongo': ('http://api.mongodb.com/python/current/', None), } primary_domain = 'py' default_role = 'py:obj' doctest_global_setup = """ try: from collections import OrderedDict except: from ordereddict import OrderedDict # Python 2.6, "pip install ordereddict" from mockupdb import * """ # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = 'MockupDB' copyright = '2015, MongoDB, Inc.' # The version info for the project you're documenting, acts as replacement # for |version| and |release|, also used in various other places throughout # the built documents. # # The short X.Y version. version = mockupdb.__version__ # The full version, including alpha/beta/rc tags. release = mockupdb.__version__ # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to # some non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build'] # The reST default role (used for this markup: `text`) to use for all # documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built # documents. #keep_warnings = False # -- Options for HTML output ------------------------------------------- # Theme gratefully vendored from CPython source. html_theme = "pydoctheme" html_theme_path = ["."] html_theme_options = {'collapsiblesidebar': True} # Theme options are theme-specific and customize the look and feel of a # theme further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as # html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the # top of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon # of the docs. This file should be a Windows icon file (.ico) being # 16x16 or 32x32 pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) # here, relative to this directory. They are copied after the builtin # static files, so a file named "default.css" will overwrite the builtin # "default.css". html_static_path = ['_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page # bottom, using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names # to template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. # Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. # Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages # will contain a tag referring to it. The value of this option # must be the base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'mockupdbdoc' # -- Options for LaTeX output ------------------------------------------ latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass # [howto/manual]). latex_documents = [ ('index', 'mockupdb.tex', 'MockupDB Documentation', 'A. Jesse Jiryu Davis', 'manual'), ] # The name of an image file (relative to this directory) to place at # the top of the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings # are parts, not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output ------------------------------------ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'mockupdb', 'MockupDB Documentation', ['A. Jesse Jiryu Davis'], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ---------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ('index', 'mockupdb', 'MockupDB Documentation', 'A. Jesse Jiryu Davis', 'mockupdb', ('Mock server for testing MongoDB clients and creating MongoDB Wire Protocol' ' servers.'), 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' # If true, do not generate a @detailmenu in the "Top" node's menu. #texinfo_no_detailmenu = False mockupdb-1.3.0/docs/contributing.rst0000644000076700000240000000004112474506007021214 0ustar emptysquarestaff00000000000000.. include:: ../CONTRIBUTING.rst mockupdb-1.3.0/docs/index.rst0000644000076700000240000000145213242633741017623 0ustar emptysquarestaff00000000000000.. mongo-mockup-db documentation master file, created by sphinx-quickstart on Tue Jul 9 22:26:36 2013. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. MockupDB ======== .. image:: _static/mask.jpg Mock server for testing MongoDB clients and creating `MongoDB Wire Protocol`_ servers. The :doc:`tutorial` is the primary documentation. Contents: .. toctree:: :maxdepth: 1 installation tutorial reference contributing authors changelog Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` Image Credit: `gnuckx `_ .. _MongoDB Wire Protocol: http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wire-protocol/ mockupdb-1.3.0/docs/installation.rst0000644000076700000240000000016013003675262021207 0ustar emptysquarestaff00000000000000============ Installation ============ Install MockupDB with pip: $ python -m pip install mongo-mockup-db mockupdb-1.3.0/docs/make.bat0000644000076700000240000001451512474506007017373 0ustar emptysquarestaff00000000000000@ECHO OFF REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set BUILDDIR=_build set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . set I18NSPHINXOPTS=%SPHINXOPTS% . if NOT "%PAPER%" == "" ( set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% ) if "%1" == "" goto help if "%1" == "help" ( :help echo.Please use `make ^` where ^ is one of echo. html to make standalone HTML files echo. dirhtml to make HTML files named index.html in directories echo. singlehtml to make a single large HTML file echo. pickle to make pickle files echo. json to make JSON files echo. htmlhelp to make HTML files and a HTML help project echo. qthelp to make HTML files and a qthelp project echo. devhelp to make HTML files and a Devhelp project echo. epub to make an epub echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter echo. text to make text files echo. man to make manual pages echo. texinfo to make Texinfo files echo. gettext to make PO message catalogs echo. changes to make an overview over all changed/added/deprecated items echo. xml to make Docutils-native XML files echo. pseudoxml to make pseudoxml-XML files for display purposes echo. linkcheck to check all external links for integrity echo. doctest to run all doctests embedded in the documentation if enabled goto end ) if "%1" == "clean" ( for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i del /q /s %BUILDDIR%\* goto end ) %SPHINXBUILD% 2> nul if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx echo.installed, then set the SPHINXBUILD environment variable to point echo.to the full path of the 'sphinx-build' executable. Alternatively you echo.may add the Sphinx directory to PATH. echo. echo.If you don't have Sphinx installed, grab it from echo.http://sphinx-doc.org/ exit /b 1 ) if "%1" == "html" ( %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/html. goto end ) if "%1" == "dirhtml" ( %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. goto end ) if "%1" == "singlehtml" ( %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. goto end ) if "%1" == "pickle" ( %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the pickle files. goto end ) if "%1" == "json" ( %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the JSON files. goto end ) if "%1" == "htmlhelp" ( %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run HTML Help Workshop with the ^ .hhp project file in %BUILDDIR%/htmlhelp. goto end ) if "%1" == "qthelp" ( %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run "qcollectiongenerator" with the ^ .qhcp project file in %BUILDDIR%/qthelp, like this: echo.^> qcollectiongenerator %BUILDDIR%\qthelp\mongo-mockup-db.qhcp echo.To view the help file: echo.^> assistant -collectionFile %BUILDDIR%\qthelp\mongo-mockup-db.ghc goto end ) if "%1" == "devhelp" ( %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp if errorlevel 1 exit /b 1 echo. echo.Build finished. goto end ) if "%1" == "epub" ( %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub if errorlevel 1 exit /b 1 echo. echo.Build finished. The epub file is in %BUILDDIR%/epub. goto end ) if "%1" == "latex" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex if errorlevel 1 exit /b 1 echo. echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdf" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf cd %BUILDDIR%/.. echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdfja" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf-ja cd %BUILDDIR%/.. echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "text" ( %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text if errorlevel 1 exit /b 1 echo. echo.Build finished. The text files are in %BUILDDIR%/text. goto end ) if "%1" == "man" ( %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man if errorlevel 1 exit /b 1 echo. echo.Build finished. The manual pages are in %BUILDDIR%/man. goto end ) if "%1" == "texinfo" ( %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo if errorlevel 1 exit /b 1 echo. echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. goto end ) if "%1" == "gettext" ( %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale if errorlevel 1 exit /b 1 echo. echo.Build finished. The message catalogs are in %BUILDDIR%/locale. goto end ) if "%1" == "changes" ( %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes if errorlevel 1 exit /b 1 echo. echo.The overview file is in %BUILDDIR%/changes. goto end ) if "%1" == "linkcheck" ( %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck if errorlevel 1 exit /b 1 echo. echo.Link check complete; look for any errors in the above output ^ or in %BUILDDIR%/linkcheck/output.txt. goto end ) if "%1" == "doctest" ( %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest if errorlevel 1 exit /b 1 echo. echo.Testing of doctests in the sources finished, look at the ^ results in %BUILDDIR%/doctest/output.txt. goto end ) if "%1" == "xml" ( %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml if errorlevel 1 exit /b 1 echo. echo.Build finished. The XML files are in %BUILDDIR%/xml. goto end ) if "%1" == "pseudoxml" ( %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml if errorlevel 1 exit /b 1 echo. echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. goto end ) :end mockupdb-1.3.0/docs/Makefile0000644000076700000240000001521612474506007017425 0ustar emptysquarestaff00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # User-friendly check for sphinx-build ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) endif # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " xml to make Docutils-native XML files" @echo " pseudoxml to make pseudoxml-XML files for display purposes" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/mongo-mockup-db.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/mongo-mockup-db.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/mongo-mockup-db" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/mongo-mockup-db" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." latexpdfja: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through platex and dvipdfmx..." $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." xml: $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml @echo @echo "Build finished. The XML files are in $(BUILDDIR)/xml." pseudoxml: $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml @echo @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." mockupdb-1.3.0/docs/reference.rst0000644000076700000240000000010312475516614020450 0ustar emptysquarestaff00000000000000API Reference ============= .. automodule:: mockupdb :members: mockupdb-1.3.0/docs/tutorial.rst0000644000076700000240000004051613242633741020363 0ustar emptysquarestaff00000000000000======== Tutorial ======== .. currentmodule:: mockupdb This tutorial is the primary documentation for the MockupDB project. I assume some familiarity with PyMongo_ and the `MongoDB Wire Protocol`_. .. contents:: Introduction ------------ Begin by running a :class:`.MockupDB` and connecting to it with PyMongo's `~pymongo.mongo_client.MongoClient`: >>> from mockupdb import * >>> server = MockupDB() >>> port = server.run() # Returns the TCP port number it listens on. >>> from pymongo import MongoClient >>> client = MongoClient(server.uri) When the client connects it calls the "ismaster" command, then blocks until the server responds. MockupDB receives the "ismaster" command but does not respond until you tell it to: >>> request = server.receives() >>> request.command_name 'ismaster' We respond: >>> request.replies({'ok': 1, 'maxWireVersion': 6}) True The `~MockupDB.receives` call blocks until it receives a request from the client. Responding to each "ismaster" call is tiresome, so tell the client to send the default response to all ismaster calls: >>> responder = server.autoresponds('ismaster', maxWireVersion=6) >>> client.admin.command('ismaster') == {'ok': 1, 'maxWireVersion': 6} True A call to `~MockupDB.receives` now blocks waiting for some request that does *not* match "ismaster". (Notice that `~Request.replies` returns True. This makes more advanced uses of `~MockupDB.autoresponds` easier, see the reference document.) Reply To Legacy Writes ---------------------- Send an unacknowledged OP_INSERT: >>> from pymongo.write_concern import WriteConcern >>> w0 = WriteConcern(w=0) >>> collection = client.db.coll.with_options(write_concern=w0) >>> collection.insert_one({'_id': 1}) # doctest: +ELLIPSIS >>> server.receives() OpInsert({"_id": 1}, namespace="db.coll") Reply To Write Commands ----------------------- If PyMongo sends an unacknowledged OP_INSERT it does not block waiting for you to call `~Request.replies`. However, for acknowledge operations it does block. Use `~test.utils.go` to defer PyMongo to a background thread so you can respond from the main thread: >>> collection = client.db.coll >>> from mockupdb import go >>> # Default write concern is acknowledged. >>> future = go(collection.insert_one, {'_id': 1}) Pass a method and its arguments to the `go` function, the same as to `functools.partial`. It launches `~pymongo.collection.Collection.insert_one` on a thread and returns a handle to its future outcome. Meanwhile, wait for the client's request to arrive on the main thread: >>> cmd = server.receives() >>> cmd Command({"insert": "coll", "ordered": true, "documents": [{"_id": 1}]}, namespace="db") (Note how MockupDB renders requests and replies as JSON, not Python. The chief differences are that "true" and "false" are lower-case, and the order of keys and values is faithfully shown, even in Python versions with unordered dicts.) Respond thus: >>> cmd.ok() True The server's response unblocks the client, so its future contains the return value of `~pymongo.collection.Collection.insert_one`, which is an `~pymongo.results.InsertOneResult`: >>> write_result = future() >>> write_result # doctest: +ELLIPSIS >>> write_result.inserted_id 1 If you don't need the future's return value, you can express this more tersely with `going`: >>> with going(collection.insert_one, {'_id': 1}): ... server.receives().ok() True Simulate a command error: >>> future = go(collection.insert_one, {'_id': 1}) >>> server.receives(insert='coll').command_err(11000, 'eek!') True >>> future() Traceback (most recent call last): ... DuplicateKeyError: eek! Or a network error: >>> future = go(collection.insert_one, {'_id': 1}) >>> server.receives(insert='coll').hangup() True >>> future() Traceback (most recent call last): ... AutoReconnect: connection closed Pattern-Match Requests ---------------------- MockupDB's pattern-matching is useful for testing: you can tell the server to verify any aspect of the expected client request. Pass a pattern to `~.MockupDB.receives` to test that the next request matches the pattern: >>> future = go(client.db.command, 'commandFoo') >>> request = server.receives('commandBar') # doctest: +NORMALIZE_WHITESPACE Traceback (most recent call last): ... AssertionError: expected to receive Command({"commandBar": 1}), got Command({"commandFoo": 1}) Even if the pattern does not match, the request is still popped from the queue. If you do not know what order you need to accept requests, you can make a little loop: >>> import traceback >>> def loop(): ... try: ... while server.running: ... # Match queries most restrictive first. ... if server.got(Command('find', 'coll', filter={'a': {'$gt': 1}})): ... server.reply(cursor={'id': 0, 'firstBatch':[{'a': 2}]}) ... elif server.got('break'): ... server.ok() ... break ... elif server.got(Command('find', 'coll')): ... server.reply( ... cursor={'id': 0, 'firstBatch':[{'a': 1}, {'a': 2}]}) ... else: ... server.command_err(errmsg='unrecognized request') ... except: ... traceback.print_exc() ... raise ... >>> future = go(loop) >>> >>> list(client.db.coll.find()) [{'a': 1}, {'a': 2}] >>> list(client.db.coll.find({'a': {'$gt': 1}})) [{'a': 2}] >>> client.db.command('break') {'ok': 1} >>> future() You can even implement the "shutdown" command: >>> def loop(): ... try: ... while server.running: ... if server.got('shutdown'): ... server.stop() # Hangs up. ... else: ... server.command_err('unrecognized request') ... except: ... traceback.print_exc() ... raise ... >>> future = go(loop) >>> client.db.command('shutdown') Traceback (most recent call last): ... AutoReconnect: connection closed >>> future() >>> server.running False >>> client.close() To show off a difficult test that MockupDB makes easy, assert that PyMongo sends a ``writeConcern`` argument if you specify ``w=1``: >>> server = MockupDB() >>> responder = server.autoresponds('ismaster', maxWireVersion=6) >>> port = server.run() >>> >>> # Specify w=1. This is distinct from the default write concern. >>> client = MongoClient(server.uri, w=1) >>> collection = client.db.coll >>> future = go(collection.insert_one, {'_id': 4}) >>> server.receives({'writeConcern': {'w': 1}}).sends() True >>> client.close() ... but not by default: >>> # Accept the default write concern. >>> client = MongoClient(server.uri) >>> collection = client.db.coll >>> future = go(collection.insert_one, {'_id': 5}) >>> assert 'writeConcern' not in server.receives() >>> client.close() .. _message spec: Message Specs ------------- We've seen some examples of ways to specify messages to send, and examples of ways to assert that a reply matches an expected pattern. Both are "message specs", a flexible syntax for describing wire protocol messages. Matching a request '''''''''''''''''' One of MockupDB's most useful features for testing your application is that it can assert that your application's requests match a particular pattern: >>> client = MongoClient(server.uri) >>> future = go(client.db.collection.insert, {'_id': 1}) >>> # Assert the command name is "insert" and its parameter is "collection". >>> request = server.receives(Command('insert', 'collection')) >>> request.ok() True >>> assert future() If the request did not match, MockupDB would raise an `AssertionError`. The arguments to `Command` above are an example of a message spec. The pattern-matching rules are implemented in `Matcher`. Here are some more examples. The empty matcher matches anything: >>> Matcher().matches({'a': 1}) True >>> Matcher().matches({'a': 1}, {'a': 1}) True >>> Matcher().matches('ismaster') True A matcher's document matches if its key-value pairs are a subset of the request's: >>> Matcher({'a': 1}).matches({'a': 1}) True >>> Matcher({'a': 2}).matches({'a': 1}) False >>> Matcher({'a': 1}).matches({'a': 1, 'b': 1}) True Prohibit a field: >>> Matcher({'field': absent}) Matcher(Request({"field": {"absent": 1}})) >>> Matcher({'field': absent}).matches({'field': 1}) False >>> Matcher({'field': absent}).matches({'otherField': 1}) True Order matters if you use an OrderedDict: >>> doc0 = OrderedDict([('a', 1), ('b', 1)]) >>> doc1 = OrderedDict([('b', 1), ('a', 1)]) >>> Matcher(doc0).matches(doc0) True >>> Matcher(doc0).matches(doc1) False The matcher must have the same number of documents as the request: >>> Matcher().matches() True >>> Matcher([]).matches([]) True >>> Matcher({'a': 2}).matches({'a': 1}, {'a': 1}) False By default, it matches any opcode: >>> m = Matcher() >>> m.matches(OpQuery) True >>> m.matches(OpInsert) True You can specify what request opcode to match: >>> m = Matcher(OpQuery) >>> m.matches(OpInsert, {'_id': 1}) False >>> m.matches(OpQuery, {'_id': 1}) True Commands are queries on some database's "database.$cmd" namespace. They are specially prohibited from matching regular queries: >>> Matcher(OpQuery).matches(Command) False >>> Matcher(Command).matches(Command) True >>> Matcher(OpQuery).matches(OpQuery) True >>> Matcher(Command).matches(OpQuery) False The command name is matched case-insensitively: >>> Matcher(Command('ismaster')).matches(Command('IsMaster')) True You can match properties specific to certain opcodes: >>> m = Matcher(OpGetMore, num_to_return=3) >>> m.matches(OpGetMore()) False >>> m.matches(OpGetMore(num_to_return=2)) False >>> m.matches(OpGetMore(num_to_return=3)) True >>> m = Matcher(OpQuery(namespace='db.collection')) >>> m.matches(OpQuery) False >>> m.matches(OpQuery(namespace='db.collection')) True It matches any wire protocol header bits you specify: >>> m = Matcher(flags=QUERY_FLAGS['SlaveOkay']) >>> m.matches(OpQuery({'_id': 1})) False >>> m.matches(OpQuery({'_id': 1}, flags=QUERY_FLAGS['SlaveOkay'])) True If you match on flags, be careful to also match on opcode. For example, if you simply check that the flag in bit position 0 is set: >>> m = Matcher(flags=INSERT_FLAGS['ContinueOnError']) ... you will match any request with that flag: >>> m.matches(OpDelete, flags=DELETE_FLAGS['SingleRemove']) True So specify the opcode, too: >>> m = Matcher(OpInsert, flags=INSERT_FLAGS['ContinueOnError']) >>> m.matches(OpDelete, flags=DELETE_FLAGS['SingleRemove']) False Sending a reply ''''''''''''''' The default reply is ``{'ok': 1}``: .. code-block:: pycon3 >>> request = server.receives() >>> request.ok() # Send {'ok': 1}. You can send additional information with the `~Request.ok` method: .. code-block:: pycon3 >>> request.ok(field='value') # Send {'ok': 1, 'field': 'value'}. Simulate a server error with `~Request.command_err`: .. code-block:: pycon3 >>> request.command_err(code=11000, errmsg='Duplicate key', field='value') All methods for sending replies parse their arguments with the `make_reply` internal function. The function interprets its first argument as the "ok" field value if it is a number, otherwise interprets it as the first field of the reply document and assumes the value is 1: >>> import mockupdb >>> mockupdb.make_reply() OpReply() >>> mockupdb.make_reply(0) OpReply({"ok": 0}) >>> mockupdb.make_reply("foo") OpReply({"foo": 1}) You can pass a dict or OrderedDict of fields instead of using keyword arguments. This is best for fieldnames that are not valid Python identifiers: >>> mockupdb.make_reply(OrderedDict([('ok', 0), ('$err', 'bad')])) OpReply({"ok": 0, "$err": "bad"}) You can customize the OP_REPLY header flags with the "flags" keyword argument: >>> r = mockupdb.make_reply(OrderedDict([('ok', 0), ('$err', 'bad')]), ... flags=REPLY_FLAGS['QueryFailure']) >>> repr(r) 'OpReply({"ok": 0, "$err": "bad"}, flags=QueryFailure)' The above logic, which simulates a query error in MongoDB before 3.2, is provided conveniently in `~Request.fail()`. This protocol is obsolete in MongoDB 3.2+, which uses commands for all operations. Although these examples call `make_reply` explicitly, this is only to illustrate how replies are specified. Your code will pass these arguments to a `Request` method like `~Request.replies`. Wait For A Request Impatiently ------------------------------ If your test waits for PyMongo to send a request but receives none, it times out after 10 seconds by default. This way MockupDB ensures that even failing tests all take finite time. To abbreviate the wait, pass a timeout in seconds to `~MockupDB.receives`: >>> try: ... server.receives(timeout=0.1) ... except AssertionError as err: ... print("Error: %s" % err) Error: expected to receive Request(), got nothing Test Cursor Behavior -------------------- Test what happens when a query fails: >>> cursor = collection.find().batch_size(1) >>> future = go(next, cursor) >>> server.receives(Command('find', 'coll')).fail() True >>> future() Traceback (most recent call last): ... OperationFailure: database error: MockupDB query failure You can simulate normal querying, too: >>> cursor = collection.find().batch_size(2) >>> future = go(list, cursor) >>> documents = [{'_id': 1}, {'x': 2}, {'foo': 'bar'}, {'beauty': True}] >>> request = server.receives(Command('find', 'coll')) >>> n = request['batchSize'] >>> request.replies(cursor={'id': 123, 'firstBatch': documents[:n]}) True >>> while True: ... getmore = server.receives(Command('getMore', 123)) ... n = getmore['batchSize'] ... if documents: ... cursor_id = 123 ... else: ... cursor_id = 0 ... getmore.ok(cursor={'id': cursor_id, 'nextBatch': documents[:n]}) ... print('returned %d' % len(documents[:n])) ... del documents[:n] ... if cursor_id == 0: ... break True returned 2 True returned 2 True returned 0 The loop receives three getMore commands and replies three times (``True`` is printed each time we call ``getmore.ok``), sending a cursor id of 0 on the last iteration to tell PyMongo that the cursor is finished. The cursor receives all documents: >>> future() [{'_id': 1}, {'x': 2}, {'_id': 1}, {'x': 2}, {'foo': 'bar'}, {'beauty': True}] But this is just a parlor trick. Let us test something serious. Test Server Discovery And Monitoring ------------------------------------ To test PyMongo's server monitor, make the server a secondary: >>> hosts = [server.address_string] >>> secondary_reply = OpReply({ ... 'ismaster': False, ... 'secondary': True, ... 'setName': 'rs', ... 'hosts': hosts, ... 'maxWireVersion': 6}) >>> responder = server.autoresponds('ismaster', secondary_reply) Connect to the replica set: >>> client = MongoClient(server.uri, replicaSet='rs') >>> from mockupdb import wait_until >>> wait_until(lambda: server.address in client.secondaries, ... 'discover secondary') True Add a primary to the host list: >>> primary = MockupDB() >>> port = primary.run() >>> hosts.append(primary.address_string) >>> primary_reply = OpReply({ ... 'ismaster': True, ... 'secondary': False, ... 'setName': 'rs', ... 'hosts': hosts, ... 'maxWireVersion': 6}) >>> responder = primary.autoresponds('ismaster', primary_reply) Client discovers it quickly if there's a pending operation: >>> with going(client.db.command, 'buildinfo'): ... wait_until(lambda: primary.address == client.primary, ... 'discovery primary') ... primary.pop('buildinfo').ok() True True .. _PyMongo: https://pypi.python.org/pypi/pymongo/ .. _MongoDB Wire Protocol: http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wire-protocol/ .. _serverStatus: http://docs.mongodb.org/manual/reference/server-status/ .. _collect: https://docs.python.org/2/library/gc.html#gc.collect mockupdb-1.3.0/LICENSE0000644000076700000240000002613512474510641016043 0ustar emptysquarestaff00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. mockupdb-1.3.0/MANIFEST.in0000644000076700000240000000042312574437536016600 0ustar emptysquarestaff00000000000000include AUTHORS.rst include CONTRIBUTING.rst include CHANGELOG.rst include LICENSE include README.rst recursive-include mockupdb * recursive-include tests * recursive-exclude * __pycache__ recursive-exclude * *.py[co] recursive-include docs *.rst conf.py Makefile make.bat mockupdb-1.3.0/mockupdb/0000755000076700000240000000000013242634066016635 5ustar emptysquarestaff00000000000000mockupdb-1.3.0/mockupdb/__init__.py0000755000076700000240000015526213242634017020760 0ustar emptysquarestaff00000000000000# -*- coding: utf-8 -*- # Copyright 2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Simulate a MongoDB server, for use in unittests.""" from __future__ import print_function __author__ = 'A. Jesse Jiryu Davis' __email__ = 'jesse@mongodb.com' __version__ = '1.3.0' import atexit import collections import contextlib import errno import functools import inspect import os import random import select import ssl as _ssl import socket import struct import traceback import threading import time import weakref import sys from codecs import utf_8_decode as _utf_8_decode try: from queue import Queue, Empty except ImportError: from Queue import Queue, Empty try: from collections import OrderedDict except: from ordereddict import OrderedDict # Python 2.6, "pip install ordereddict" try: from io import StringIO except ImportError: from cStringIO import StringIO # Pure-Python bson lib vendored in from PyMongo 3.0.3. from mockupdb import _bson import mockupdb._bson.codec_options as _codec_options import mockupdb._bson.json_util as _json_util CODEC_OPTIONS = _codec_options.CodecOptions(document_class=OrderedDict) PY3 = sys.version_info[0] == 3 if PY3: string_type = str text_type = str def reraise(exctype, value, trace=None): raise exctype(str(value)).with_traceback(trace) else: string_type = basestring text_type = unicode # "raise x, y, z" raises SyntaxError in Python 3. exec("""def reraise(exctype, value, trace=None): raise exctype, str(value), trace """) __all__ = [ 'MockupDB', 'go', 'going', 'Future', 'wait_until', 'interactive_server', 'OP_REPLY', 'OP_UPDATE', 'OP_INSERT', 'OP_QUERY', 'OP_GET_MORE', 'OP_DELETE', 'OP_KILL_CURSORS', 'QUERY_FLAGS', 'UPDATE_FLAGS', 'INSERT_FLAGS', 'DELETE_FLAGS', 'REPLY_FLAGS', 'Request', 'Command', 'OpQuery', 'OpGetMore', 'OpKillCursors', 'OpInsert', 'OpUpdate', 'OpDelete', 'OpReply', 'Matcher', 'absent', ] def go(fn, *args, **kwargs): """Launch an operation on a thread and get a handle to its future result. >>> from time import sleep >>> def print_sleep_print(duration): ... sleep(duration) ... print('hello from background thread') ... sleep(duration) ... print('goodbye from background thread') ... return 'return value' ... >>> future = go(print_sleep_print, 0.1) >>> sleep(0.15) hello from background thread >>> print('main thread') main thread >>> result = future() goodbye from background thread >>> result 'return value' """ if not callable(fn): raise TypeError('go() requires a function, not %r' % (fn, )) result = [None] error = [] def target(): try: result[0] = fn(*args, **kwargs) except Exception: # Are we in interpreter shutdown? if sys: error.extend(sys.exc_info()) t = threading.Thread(target=target) t.daemon = True t.start() def get_result(timeout=10): t.join(timeout) if t.is_alive(): raise AssertionError('timed out waiting for %r' % fn) if error: reraise(*error) return result[0] return get_result @contextlib.contextmanager def going(fn, *args, **kwargs): """Launch a thread and wait for its result before exiting the code block. >>> with going(lambda: 'return value') as future: ... pass >>> future() # Won't block, the future is ready by now. 'return value' Or discard the result: >>> with going(lambda: "don't care"): ... pass If an exception is raised within the context, the result is lost: >>> with going(lambda: 'return value') as future: ... assert 1 == 0 Traceback (most recent call last): ... AssertionError """ future = go(fn, *args, **kwargs) try: yield future except: # We are raising an exception, just try to clean up the future. exc_info = sys.exc_info() try: # Shorter than normal timeout. future(timeout=1) except: log_message = ('\nerror in %s:\n' % format_call(inspect.currentframe())) sys.stderr.write(log_message) traceback.print_exc() # sys.stderr.write('exc in %s' % format_call(inspect.currentframe())) reraise(*exc_info) else: # Raise exception or discard result. future(timeout=10) class Future(object): def __init__(self): self._result = None self._event = threading.Event() def result(self, timeout=None): self._event.wait(timeout) # wait() always returns None in Python 2.6. if not self._event.is_set(): raise AssertionError('timed out waiting for Future') return self._result def set_result(self, result): if self._event.is_set(): raise RuntimeError("Future is already resolved") self._result = result self._event.set() def wait_until(predicate, success_description, timeout=10): """Wait up to 10 seconds (by default) for predicate to be true. E.g.: wait_until(lambda: client.primary == ('a', 1), 'connect to the primary') If the lambda-expression isn't true after 10 seconds, we raise AssertionError("Didn't ever connect to the primary"). Returns the predicate's first true value. """ start = time.time() while True: retval = predicate() if retval: return retval if time.time() - start > timeout: raise AssertionError("Didn't ever %s" % success_description) time.sleep(0.1) OP_REPLY = 1 OP_UPDATE = 2001 OP_INSERT = 2002 OP_QUERY = 2004 OP_GET_MORE = 2005 OP_DELETE = 2006 OP_KILL_CURSORS = 2007 QUERY_FLAGS = OrderedDict([ ('TailableCursor', 2), ('SlaveOkay', 4), ('OplogReplay', 8), ('NoTimeout', 16), ('AwaitData', 32), ('Exhaust', 64), ('Partial', 128)]) UPDATE_FLAGS = OrderedDict([ ('Upsert', 1), ('MultiUpdate', 2)]) INSERT_FLAGS = OrderedDict([ ('ContinueOnError', 1)]) DELETE_FLAGS = OrderedDict([ ('SingleRemove', 1)]) REPLY_FLAGS = OrderedDict([ ('CursorNotFound', 1), ('QueryFailure', 2)]) _UNPACK_INT = struct.Struct(">> {'_id': 0} in OpInsert({'_id': 0}) True >>> {'_id': 1} in OpInsert({'_id': 0}) False >>> {'_id': 1} in OpInsert([{'_id': 0}, {'_id': 1}]) True >>> {'_id': 1} == OpInsert([{'_id': 0}, {'_id': 1}])[1] True >>> 'field' in Command(field=1) True >>> 'field' in Command() False >>> 'field' in Command('ismaster') False >>> Command(ismaster=False)['ismaster'] is False True """ opcode = None is_command = None _non_matched_attrs = 'doc', 'docs' _flags_map = None def __init__(self, *args, **kwargs): self._flags = kwargs.pop('flags', None) self._namespace = kwargs.pop('namespace', None) self._client = kwargs.pop('_client', None) self._request_id = kwargs.pop('request_id', None) self._server = kwargs.pop('_server', None) self._verbose = self._server and self._server.verbose self._server_port = kwargs.pop('server_port', None) self._docs = make_docs(*args, **kwargs) if not all(isinstance(doc, collections.Mapping) for doc in self._docs): raise_args_err() @property def doc(self): """The request document, if there is exactly one. Use this for queries, commands, and legacy deletes. Legacy writes may have many documents, OP_GET_MORE and OP_KILL_CURSORS have none. """ assert len(self.docs) == 1, '%r has more than one document' % self return self.docs[0] @property def docs(self): """The request documents, if any.""" return self._docs @property def namespace(self): """The operation namespace or None.""" return self._namespace @property def flags(self): """The request flags or None.""" return self._flags @property def slave_ok(self): """True if the SlaveOkay wire protocol flag is set.""" return self._flags and bool( self._flags & QUERY_FLAGS['SlaveOkay']) slave_okay = slave_ok """Synonym for `.slave_ok`.""" @property def request_id(self): """The request id or None.""" return self._request_id @property def client_port(self): """Client connection's TCP port.""" return self._client.getpeername()[1] @property def server(self): """The `.MockupDB` server.""" return self._server def assert_matches(self, *args, **kwargs): """Assert this matches a :ref:`message spec `. Returns self. """ matcher = make_matcher(*args, **kwargs) if not matcher.matches(self): raise AssertionError('%r does not match %r' % (self, matcher)) return self def matches(self, *args, **kwargs): """True if this matches a :ref:`message spec `.""" return make_matcher(*args, **kwargs).matches(self) def replies(self, *args, **kwargs): """Send an `OpReply` to the client. The default reply to a command is ``{'ok': 1}``, otherwise the default is empty (no documents). Returns True so it is suitable as an `~MockupDB.autoresponds` handler. """ self._replies(*args, **kwargs) return True ok = send = sends = reply = replies """Synonym for `.replies`.""" def fail(self, err='MockupDB query failure', *args, **kwargs): """Reply to a query with the QueryFailure flag and an '$err' key. Returns True so it is suitable as an `~MockupDB.autoresponds` handler. """ kwargs.setdefault('flags', 0) kwargs['flags'] |= REPLY_FLAGS['QueryFailure'] kwargs['$err'] = err self.replies(*args, **kwargs) return True def command_err(self, code=1, errmsg='MockupDB command failure', *args, **kwargs): """Error reply to a command. Returns True so it is suitable as an `~MockupDB.autoresponds` handler. """ kwargs.setdefault('ok', 0) kwargs['code'] = code kwargs['errmsg'] = errmsg self.replies(*args, **kwargs) return True def hangup(self): """Close the connection. Returns True so it is suitable as an `~MockupDB.autoresponds` handler. """ if self._server: self._server._log('\t%d\thangup' % self.client_port) self._client.shutdown(socket.SHUT_RDWR) return True hangs_up = hangup """Synonym for `.hangup`.""" def _matches_docs(self, docs, other_docs): """Overridable method.""" for i, doc in enumerate(docs): other_doc = other_docs[i] for key, value in doc.items(): if value is absent: if key in other_doc: return False elif other_doc.get(key, None) != value: return False if isinstance(doc, (OrderedDict, _bson.SON)): if not isinstance(other_doc, (OrderedDict, _bson.SON)): raise TypeError( "Can't compare ordered and unordered document types:" " %r, %r" % (doc, other_doc)) keys = [key for key, value in doc.items() if value is not absent] if not seq_match(keys, list(other_doc.keys())): return False return True def _replies(self, *args, **kwargs): """Overridable method.""" reply_msg = make_reply(*args, **kwargs) if self._server: self._server._log('\t%d\t<-- %r' % (self.client_port, reply_msg)) reply_bytes = reply_msg.reply_bytes(self) self._client.sendall(reply_bytes) def __contains__(self, item): if item in self.docs: return True if len(self.docs) == 1 and isinstance(item, (string_type, text_type)): return item in self.doc return False def __getitem__(self, item): return self.doc[item] if len(self.docs) == 1 else self.docs[item] def __str__(self): return docs_repr(*self.docs) def __repr__(self): name = self.__class__.__name__ parts = [] if self.docs: parts.append(docs_repr(*self.docs)) if self._flags: if self._flags_map: parts.append('flags=%s' % ( '|'.join(name for name, value in self._flags_map.items() if self._flags & value))) else: parts.append('flags=%d' % self._flags) if self._namespace: parts.append('namespace="%s"' % self._namespace) return '%s(%s)' % (name, ', '.join(str(part) for part in parts)) class OpQuery(Request): """A query (besides a command) the client executes on the server. >>> OpQuery({'i': {'$gt': 2}}, fields={'j': False}) OpQuery({"i": {"$gt": 2}}, fields={"j": false}) """ opcode = OP_QUERY is_command = False _flags_map = QUERY_FLAGS @classmethod def unpack(cls, msg, client, server, request_id): """Parse message and return an `OpQuery` or `Command`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ flags, = _UNPACK_INT(msg[:4]) namespace, pos = _get_c_string(msg, 4) is_command = namespace.endswith('.$cmd') num_to_skip, = _UNPACK_INT(msg[pos:pos + 4]) pos += 4 num_to_return, = _UNPACK_INT(msg[pos:pos + 4]) pos += 4 docs = _bson.decode_all(msg[pos:], CODEC_OPTIONS) if is_command: assert len(docs) == 1 command_ns = namespace[:-len('.$cmd')] return Command(docs, namespace=command_ns, flags=flags, _client=client, request_id=request_id, _server=server) else: if len(docs) == 1: fields = None else: assert len(docs) == 2 fields = docs[1] return OpQuery(docs[0], fields=fields, namespace=namespace, flags=flags, num_to_skip=num_to_skip, num_to_return=num_to_return, _client=client, request_id=request_id, _server=server) def __init__(self, *args, **kwargs): fields = kwargs.pop('fields', None) if fields is not None and not isinstance(fields, collections.Mapping): raise_args_err() self._fields = fields self._num_to_skip = kwargs.pop('num_to_skip', None) self._num_to_return = kwargs.pop('num_to_return', None) super(OpQuery, self).__init__(*args, **kwargs) if not self._docs: self._docs = [{}] # Default query filter. elif len(self._docs) > 1: raise_args_err('OpQuery too many documents', ValueError) @property def num_to_skip(self): """Client query's numToSkip or None.""" return self._num_to_skip @property def num_to_return(self): """Client query's numToReturn or None.""" return self._num_to_return @property def fields(self): """Client query's fields selector or None.""" return self._fields def __repr__(self): rep = super(OpQuery, self).__repr__().rstrip(')') if self._fields: rep += ', fields=%s' % docs_repr(self._fields) if self._num_to_skip is not None: rep += ', numToSkip=%d' % self._num_to_skip if self._num_to_return is not None: rep += ', numToReturn=%d' % self._num_to_return return rep + ')' class Command(OpQuery): """A command the client executes on the server.""" is_command = True # Check command name case-insensitively. _non_matched_attrs = OpQuery._non_matched_attrs + ('command_name', ) @property def command_name(self): """The command name or None. >>> Command({'count': 'collection'}).command_name 'count' >>> Command('aggregate', 'collection', cursor=absent).command_name 'aggregate' """ if self.docs and self.docs[0]: return list(self.docs[0])[0] def _matches_docs(self, docs, other_docs): assert len(docs) == len(other_docs) == 1 doc, = docs other_doc, = other_docs items = list(doc.items()) other_items = list(other_doc.items()) # Compare command name case-insensitively. if items and other_items: if items[0][0].lower() != other_items[0][0].lower(): return False if items[0][1] != other_items[0][1]: return False return super(Command, self)._matches_docs( [OrderedDict(items[1:])], [OrderedDict(other_items[1:])]) def _replies(self, *args, **kwargs): reply = make_reply(*args, **kwargs) if not reply.docs: reply.docs = [{'ok': 1}] else: if len(reply.docs) > 1: raise ValueError('Command reply with multiple documents: %s' % (reply.docs, )) reply.doc.setdefault('ok', 1) super(Command, self)._replies(reply) def replies_to_gle(self, **kwargs): """Send a getlasterror response. Defaults to ``{ok: 1, err: null}``. Add or override values by passing keyword arguments. Returns True so it is suitable as an `~MockupDB.autoresponds` handler. """ kwargs.setdefault('err', None) return self.replies(**kwargs) class OpGetMore(Request): """An OP_GET_MORE the client executes on the server.""" @classmethod def unpack(cls, msg, client, server, request_id): """Parse message and return an `OpGetMore`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ flags, = _UNPACK_INT(msg[:4]) namespace, pos = _get_c_string(msg, 4) num_to_return, = _UNPACK_INT(msg[pos:pos + 4]) pos += 4 cursor_id, = _UNPACK_LONG(msg[pos:pos + 8]) return OpGetMore(namespace=namespace, flags=flags, _client=client, num_to_return=num_to_return, cursor_id=cursor_id, request_id=request_id, _server=server) def __init__(self, **kwargs): self._num_to_return = kwargs.pop('num_to_return', None) self._cursor_id = kwargs.pop('cursor_id', None) super(OpGetMore, self).__init__(**kwargs) @property def num_to_return(self): """The client message's numToReturn field.""" return self._num_to_return @property def cursor_id(self): """The client message's cursorId field.""" return self._cursor_id class OpKillCursors(Request): """An OP_KILL_CURSORS the client executes on the server.""" @classmethod def unpack(cls, msg, client, server, _): """Parse message and return an `OpKillCursors`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ # Leading 4 bytes are reserved. num_of_cursor_ids, = _UNPACK_INT(msg[4:8]) cursor_ids = [] pos = 8 for _ in range(num_of_cursor_ids): cursor_ids.append(_UNPACK_INT(msg[pos:pos+4])[0]) pos += 4 return OpKillCursors(_client=client, cursor_ids=cursor_ids, _server=server) def __init__(self, **kwargs): self._cursor_ids = kwargs.pop('cursor_ids', None) super(OpKillCursors, self).__init__(**kwargs) @property def cursor_ids(self): """List of cursor ids the client wants to kill.""" return self._cursor_ids def __repr__(self): return '%s(%s)' % (self.__class__.__name__, self._cursor_ids) class _LegacyWrite(Request): is_command = False class OpInsert(_LegacyWrite): """A legacy OP_INSERT the client executes on the server.""" opcode = OP_INSERT _flags_map = INSERT_FLAGS @classmethod def unpack(cls, msg, client, server, request_id): """Parse message and return an `OpInsert`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ flags, = _UNPACK_INT(msg[:4]) namespace, pos = _get_c_string(msg, 4) docs = _bson.decode_all(msg[pos:], CODEC_OPTIONS) return cls(*docs, namespace=namespace, flags=flags, _client=client, request_id=request_id, _server=server) class OpUpdate(_LegacyWrite): """A legacy OP_UPDATE the client executes on the server.""" opcode = OP_UPDATE _flags_map = UPDATE_FLAGS @classmethod def unpack(cls, msg, client, server, request_id): """Parse message and return an `OpUpdate`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ # First 4 bytes of OP_UPDATE are "reserved". namespace, pos = _get_c_string(msg, 4) flags, = _UNPACK_INT(msg[pos:pos + 4]) docs = _bson.decode_all(msg[pos + 4:], CODEC_OPTIONS) return cls(*docs, namespace=namespace, flags=flags, _client=client, request_id=request_id, _server=server) class OpDelete(_LegacyWrite): """A legacy OP_DELETE the client executes on the server.""" opcode = OP_DELETE _flags_map = DELETE_FLAGS @classmethod def unpack(cls, msg, client, server, request_id): """Parse message and return an `OpDelete`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ # First 4 bytes of OP_DELETE are "reserved". namespace, pos = _get_c_string(msg, 4) flags, = _UNPACK_INT(msg[pos:pos + 4]) docs = _bson.decode_all(msg[pos + 4:], CODEC_OPTIONS) return cls(*docs, namespace=namespace, flags=flags, _client=client, request_id=request_id, _server=server) class OpReply(object): """A reply from `MockupDB` to the client.""" def __init__(self, *args, **kwargs): self._flags = kwargs.pop('flags', 0) self._cursor_id = kwargs.pop('cursor_id', 0) self._starting_from = kwargs.pop('starting_from', 0) self._docs = make_docs(*args, **kwargs) @property def docs(self): """The reply documents, if any.""" return self._docs @docs.setter def docs(self, docs): self._docs = make_docs(docs) @property def doc(self): """Contents of reply. Useful for replies to commands; replies to other messages may have no documents or multiple documents. """ assert len(self._docs) == 1, '%s has more than one document' % self return self._docs[0] def update(self, *args, **kwargs): """Update the document. Same as ``dict().update()``. >>> reply = OpReply({'ismaster': True}) >>> reply.update(maxWireVersion=3) >>> reply.doc['maxWireVersion'] 3 >>> reply.update({'maxWriteBatchSize': 10, 'msg': 'isdbgrid'}) """ self.doc.update(*args, **kwargs) def reply_bytes(self, request): """Take a `Request` and return an OP_REPLY message as bytes.""" flags = struct.pack("`. Used by `~MockupDB.receives` to assert the client sent the expected request, and by `~MockupDB.got` to test if it did and return ``True`` or ``False``. Used by `.autoresponds` to match requests with autoresponses. """ def __init__(self, *args, **kwargs): self._kwargs = kwargs self._prototype = make_prototype_request(*args, **kwargs) def matches(self, *args, **kwargs): """Test if a request matches a :ref:`message spec `. Returns ``True`` or ``False``. """ request = make_prototype_request(*args, **kwargs) if self._prototype.opcode not in (None, request.opcode): return False if self._prototype.is_command not in (None, request.is_command): return False for name in dir(self._prototype): if name.startswith('_') or name in request._non_matched_attrs: # Ignore privates, and handle documents specially. continue prototype_value = getattr(self._prototype, name, None) if inspect.ismethod(prototype_value): continue actual_value = getattr(request, name, None) if prototype_value not in (None, actual_value): return False if len(self._prototype.docs) not in (0, len(request.docs)): return False return self._prototype._matches_docs(self._prototype.docs, request.docs) @property def prototype(self): """The prototype `.Request` used to match actual requests with.""" return self._prototype def __repr__(self): return '%s(%r)' % (self.__class__.__name__, self._prototype) def _synchronized(meth): """Call method while holding a lock.""" @functools.wraps(meth) def wrapper(self, *args, **kwargs): with self._lock: return meth(self, *args, **kwargs) return wrapper class _AutoResponder(object): def __init__(self, server, matcher, *args, **kwargs): self._server = server if inspect.isfunction(matcher) or inspect.ismethod(matcher): if args or kwargs: raise_args_err() self._matcher = Matcher() # Match anything. self._handler = matcher self._args = () self._kwargs = {} else: self._matcher = make_matcher(matcher) if args and callable(args[0]): self._handler = args[0] if args[1:] or kwargs: raise_args_err() self._args = () self._kwargs = {} else: self._handler = None self._args = args self._kwargs = kwargs def handle(self, request): if self._matcher.matches(request): if self._handler: return self._handler(request) else: # Command.replies() overrides Request.replies() with special # logic, which is why we saved args and kwargs until now to # pass it into request.replies, instead of making an OpReply # ourselves in __init__. request.replies(*self._args, **self._kwargs) return True def cancel(self): """Stop autoresponding.""" self._server.cancel_responder(self) def __repr__(self): return '_AutoResponder(%r, %r, %r)' % ( self._matcher, self._args, self._kwargs) _shutting_down = False _global_threads = weakref.WeakKeyDictionary() def _shut_down(threads): global _shutting_down _shutting_down = True for t in threads: try: t.join(10) except: pass atexit.register(_shut_down, _global_threads) class MockupDB(object): """A simulated mongod or mongos. Call `run` to start the server, and always `close` it to avoid exceptions during interpreter shutdown. See the tutorial for comprehensive examples. :Optional parameters: - `port`: listening port number. If not specified, choose some unused port and return the port number from `run`. - `verbose`: if ``True``, print requests and replies to stdout. - `request_timeout`: seconds to wait for the next client request, or else assert. Default 10 seconds. Pass int(1e6) to disable. - `auto_ismaster`: pass ``True`` to autorespond ``{'ok': 1}`` to ismaster requests, or pass a dict or `OpReply`. - `ssl`: pass ``True`` to require SSL. - `min_wire_version`: the minWireVersion to include in ismaster responses if `auto_ismaster` is True, default 0. - `max_wire_version`: the maxWireVersion to include in ismaster responses if `auto_ismaster` is True, default 6. """ def __init__(self, port=None, verbose=False, request_timeout=10, auto_ismaster=None, ssl=False, min_wire_version=0, max_wire_version=6): self._address = ('localhost', port) self._verbose = verbose self._label = None self._ssl = ssl self._request_timeout = request_timeout self._listening_sock = None self._accept_thread = None # Track sockets that we want to close in stop(). Keys are sockets, # values are None (this could be a WeakSet but it's new in Python 2.7). self._server_threads = weakref.WeakKeyDictionary() self._server_socks = weakref.WeakKeyDictionary() self._stopped = False self._request_q = _PeekableQueue() self._requests_count = 0 self._lock = threading.Lock() # List of (request_matcher, args, kwargs), where args and kwargs are # like those sent to request.reply(). self._autoresponders = [] if auto_ismaster is True: self.autoresponds(Command('ismaster'), {'ismaster': True, 'minWireVersion': min_wire_version, 'maxWireVersion': max_wire_version}) elif auto_ismaster: self.autoresponds(Command('ismaster'), auto_ismaster) @_synchronized def run(self): """Begin serving. Returns the bound port.""" self._listening_sock, self._address = bind_socket(self._address) if self._ssl: certfile = os.path.join(os.path.dirname(__file__), 'server.pem') self._listening_sock = _ssl.wrap_socket( self._listening_sock, certfile=certfile, server_side=True) self._accept_thread = threading.Thread(target=self._accept_loop) self._accept_thread.daemon = True self._accept_thread.start() return self.port @_synchronized def stop(self): """Stop serving. Always call this to clean up after yourself.""" self._stopped = True threads = [self._accept_thread] threads.extend(self._server_threads) self._listening_sock.close() for sock in list(self._server_socks): try: sock.shutdown(socket.SHUT_RDWR) except socket.error: pass try: sock.close() except socket.error: pass with self._unlock(): for thread in threads: thread.join(10) def receives(self, *args, **kwargs): """Pop the next `Request` and assert it matches. Returns None if the server is stopped. Pass a `Request` or request pattern to specify what client request to expect. See the tutorial for examples. Pass ``timeout`` as a keyword argument to override this server's ``request_timeout``. """ timeout = kwargs.pop('timeout', self._request_timeout) end = time.time() + timeout matcher = Matcher(*args, **kwargs) while not self._stopped: try: # Short timeout so we notice if the server is stopped. request = self._request_q.get(timeout=0.05) except Empty: if time.time() > end: raise AssertionError('expected to receive %r, got nothing' % matcher.prototype) else: if matcher.matches(request): return request else: raise AssertionError('expected to receive %r, got %r' % (matcher.prototype, request)) gets = pop = receive = receives """Synonym for `receives`.""" def got(self, *args, **kwargs): """Does `.request` match the given :ref:`message spec `? >>> s = MockupDB(auto_ismaster=True) >>> port = s.run() >>> s.got(timeout=0) # No request enqueued. False >>> from pymongo import MongoClient >>> client = MongoClient(s.uri) >>> future = go(client.db.command, 'foo') >>> s.got('foo') True >>> s.got(Command('foo', namespace='db')) True >>> s.got(Command('foo', key='value')) False >>> s.ok() >>> future() == {'ok': 1} True >>> s.stop() """ timeout = kwargs.pop('timeout', self._request_timeout) end = time.time() + timeout matcher = make_matcher(*args, **kwargs) while not self._stopped: try: # Short timeout so we notice if the server is stopped. request = self._request_q.peek(timeout=timeout) except Empty: if time.time() > end: return False else: return matcher.matches(request) wait = got """Synonym for `got`.""" def replies(self, *args, **kwargs): """Call `~Request.reply` on the currently enqueued request.""" self.pop().replies(*args, **kwargs) ok = send = sends = reply = replies """Synonym for `.replies`.""" def fail(self, *args, **kwargs): """Call `~Request.fail` on the currently enqueued request.""" self.pop().fail(*args, **kwargs) def command_err(self, *args, **kwargs): """Call `~Request.command_err` on the currently enqueued request.""" self.pop().command_err(*args, **kwargs) def hangup(self): """Call `~Request.hangup` on the currently enqueued request.""" self.pop().hangup() hangs_up = hangup """Synonym for `.hangup`.""" @_synchronized def autoresponds(self, matcher, *args, **kwargs): """Send a canned reply to all matching client requests. ``matcher`` is a `Matcher` or a command name, or an instance of `OpInsert`, `OpQuery`, etc. >>> s = MockupDB() >>> port = s.run() >>> >>> from pymongo import MongoClient >>> client = MongoClient(s.uri) >>> responder = s.autoresponds('ismaster', maxWireVersion=6) >>> client.admin.command('ismaster') == {'ok': 1, 'maxWireVersion': 6} True The remaining arguments are a :ref:`message spec `: >>> responder = s.autoresponds('bar', ok=0, errmsg='err') >>> client.db.command('bar') Traceback (most recent call last): ... OperationFailure: command SON([('bar', 1)]) on namespace db.$cmd failed: err >>> responder = s.autoresponds(Command('find', 'collection'), ... {'cursor': {'id': 0, 'firstBatch': [{'_id': 1}, {'_id': 2}]}}) >>> list(client.db.collection.find()) == [{'_id': 1}, {'_id': 2}] True >>> responder = s.autoresponds(Command('find', 'collection'), ... {'cursor': {'id': 0, 'firstBatch': [{'a': 1}, {'a': 2}]}}) >>> list(client.db.collection.find()) == [{'a': 1}, {'a': 2}] True Remove an autoresponder like: >>> responder.cancel() If the request currently at the head of the queue matches, it is popped and replied to. Future matching requests skip the queue. >>> future = go(client.db.command, 'baz') >>> responder = s.autoresponds('baz', {'key': 'value'}) >>> future() == {'ok': 1, 'key': 'value'} True Responders are applied in order, most recently added first, until one matches: >>> responder = s.autoresponds('baz') >>> client.db.command('baz') == {'ok': 1} True >>> responder.cancel() >>> # The previous responder takes over again. >>> client.db.command('baz') == {'ok': 1, 'key': 'value'} True You can pass a request handler in place of the message spec. Return True if you handled the request: >>> responder = s.autoresponds('baz', lambda r: r.ok(a=2)) The standard `Request.ok`, `~Request.replies`, `~Request.fail`, `~Request.hangup` and so on all return True to make them suitable as handler functions. >>> client.db.command('baz') == {'ok': 1, 'a': 2} True If the request is not handled, it is checked against the remaining responders, or enqueued if none match. You can pass the handler as the only argument so it receives *all* requests. For example you could log them, then return None to allow other handlers to run: >>> def logger(request): ... if not request.matches('ismaster'): ... print('logging: %r' % request) >>> responder = s.autoresponds(logger) >>> client.db.command('baz') == {'ok': 1, 'a': 2} logging: Command({"baz": 1}, flags=SlaveOkay, namespace="db") True The synonym `subscribe` better expresses your intent if your handler never returns True: >>> subscriber = s.subscribe(logger) .. doctest: :hide: >>> client.close() >>> s.stop() """ responder = _AutoResponder(self, matcher, *args, **kwargs) self._autoresponders.append(responder) try: request = self._request_q.peek(block=False) except Empty: pass else: if responder.handle(request): self._request_q.get_nowait() # Pop it. return responder subscribe = autoresponds """Synonym for `.autoresponds`.""" @_synchronized def cancel_responder(self, responder): """Cancel a responder that was registered with `autoresponds`.""" self._autoresponders.remove(responder) @property def address(self): """The listening (host, port).""" return self._address @property def address_string(self): """The listening "host:port".""" return '%s:%d' % self._address @property def host(self): """The listening hostname.""" return self._address[0] @property def port(self): """The listening port.""" return self._address[1] @property def uri(self): """Connection string to pass to `~pymongo.mongo_client.MongoClient`.""" assert self.host and self.port uri = 'mongodb://%s:%s' % self._address return uri + '/?ssl=true' if self._ssl else uri @property def verbose(self): """If verbose logging is turned on.""" return self._verbose @verbose.setter def verbose(self, value): if not isinstance(value, bool): raise TypeError('value must be True or False, not %r' % value) self._verbose = value @property def label(self): """Label for logging, or None.""" return self._label @label.setter def label(self, value): self._label = value @property def requests_count(self): """Number of requests this server has received. Includes autoresponded requests. """ return self._requests_count @property def request(self): """The currently enqueued `Request`, or None. .. warning:: This property is useful to check what the current request is, but the pattern ``server.request.replies()`` is dangerous: you must follow it with ``server.pop()`` or the current request remains enqueued. Better to reply with ``server.pop().replies()`` than ``server.request.replies()`` or any variation on it. """ return self.got() or None @property @_synchronized def running(self): """If this server is started and not stopped.""" return self._accept_thread and not self._stopped def _accept_loop(self): """Accept client connections and spawn a thread for each.""" self._listening_sock.setblocking(0) while not self._stopped and not _shutting_down: try: # Wait a short time to accept. if select.select([self._listening_sock.fileno()], [], [], 1): client, client_addr = self._listening_sock.accept() client.setblocking(True) self._log('connection from %s:%s' % client_addr) server_thread = threading.Thread( target=functools.partial( self._server_loop, client, client_addr)) # Store weakrefs to the thread and socket, so we can # dispose them in stop(). self._server_threads[server_thread] = None self._server_socks[client] = None server_thread.daemon = True server_thread.start() except socket.error as error: if error.errno not in ( errno.EAGAIN, errno.EBADF, errno.EWOULDBLOCK): raise except select.error as error: if error.args[0] == errno.EBADF: # Closed. break else: raise @_synchronized def _server_loop(self, client, client_addr): """Read requests from one client socket, 'client'.""" while not self._stopped and not _shutting_down: try: with self._unlock(): request = mock_server_receive_request(client, self) self._requests_count += 1 self._log('%d\t%r' % (request.client_port, request)) # Give most recently added responders precedence. for responder in reversed(self._autoresponders): if responder.handle(request): self._log('\t(autoresponse)') break else: self._request_q.put(request) except socket.error as error: if error.errno in (errno.ECONNRESET, errno.EBADF): # We hung up, or the client did. break raise except select.error as error: if error.args[0] == errno.EBADF: # Closed. break else: raise except AssertionError: traceback.print_exc() break self._log('disconnected: %s:%d' % client_addr) client.close() def _log(self, msg): if self._verbose: if self._label: msg = '%s:\t%s' % (self._label, msg) print(msg) @contextlib.contextmanager def _unlock(self): """Temporarily release the lock.""" self._lock.release() try: yield finally: self._lock.acquire() def __iter__(self): return self def next(self): request = self.receives() if request is None: # Server stopped. raise StopIteration() return request __next__ = next def __repr__(self): return 'MockupDB(%s, %s)' % self._address def bind_socket(address): """Takes (host, port) and returns (socket_object, (host, port)). If the passed-in port is None, bind an unused port and return it. """ host, port = address for res in set(socket.getaddrinfo(host, port, socket.AF_INET, socket.SOCK_STREAM, 0, socket.AI_PASSIVE)): family, socktype, proto, _, sock_addr = res sock = socket.socket(family, socktype, proto) if os.name != 'nt': sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) # Automatic port allocation with port=None. sock.bind(sock_addr) sock.listen(128) bound_port = sock.getsockname()[1] return sock, (host, bound_port) raise socket.error('could not bind socket') OPCODES = {OP_QUERY: OpQuery, OP_INSERT: OpInsert, OP_UPDATE: OpUpdate, OP_DELETE: OpDelete, OP_GET_MORE: OpGetMore, OP_KILL_CURSORS: OpKillCursors} def mock_server_receive_request(client, server): """Take a client socket and return a Request.""" header = mock_server_receive(client, 16) length = _UNPACK_INT(header[:4])[0] request_id = _UNPACK_INT(header[4:8])[0] opcode = _UNPACK_INT(header[12:])[0] msg_bytes = mock_server_receive(client, length - 16) if opcode not in OPCODES: raise NotImplementedError("Don't know how to unpack opcode %d yet" % opcode) return OPCODES[opcode].unpack(msg_bytes, client, server, request_id) def _errno_from_exception(exc): if hasattr(exc, 'errno'): return exc.errno elif exc.args: return exc.args[0] else: return None def mock_server_receive(sock, length): """Receive `length` bytes from a socket object.""" msg = b'' while length: chunk = sock.recv(length) if chunk == b'': raise socket.error(errno.ECONNRESET, 'closed') length -= len(chunk) msg += chunk return msg def make_docs(*args, **kwargs): """Make the documents for a `Request` or `OpReply`. Takes a variety of argument styles, returns a list of dicts. Used by `make_prototype_request` and `make_reply`, which are in turn used by `MockupDB.receives`, `Request.replies`, and so on. See examples in tutorial. """ err_msg = "Can't interpret args: " if not args and not kwargs: return [] if not args: # OpReply(ok=1, ismaster=True). return [kwargs] if isinstance(args[0], (int, float, bool)): # server.receives().ok(0, err='uh oh'). if args[1:]: raise_args_err(err_msg, ValueError) doc = OrderedDict({'ok': args[0]}) doc.update(kwargs) return [doc] if isinstance(args[0], (list, tuple)): # Send a batch: OpReply([{'a': 1}, {'a': 2}]). if not all(isinstance(doc, (OpReply, collections.Mapping)) for doc in args[0]): raise_args_err('each doc must be a dict:') if kwargs: raise_args_err(err_msg, ValueError) return list(args[0]) if isinstance(args[0], (string_type, text_type)): if args[2:]: raise_args_err(err_msg, ValueError) if len(args) == 2: # Command('aggregate', 'collection', {'cursor': {'batchSize': 1}}). doc = OrderedDict({args[0]: args[1]}) else: # OpReply('ismaster', me='a.com'). doc = OrderedDict({args[0]: 1}) doc.update(kwargs) return [doc] if kwargs: raise_args_err(err_msg, ValueError) # Send a batch as varargs: OpReply({'a': 1}, {'a': 2}). if not all(isinstance(doc, (OpReply, collections.Mapping)) for doc in args): raise_args_err('each doc must be a dict') return args def make_matcher(*args, **kwargs): """Make a Matcher from a :ref:`message spec `: >>> make_matcher() Matcher(Request()) >>> make_matcher({'ismaster': 1}, namespace='admin') Matcher(Request({"ismaster": 1}, namespace="admin")) >>> make_matcher({}, {'_id': 1}) Matcher(Request({}, {"_id": 1})) See more examples in the tutorial section for :ref:`Message Specs`. """ if args and isinstance(args[0], Matcher): if args[1:] or kwargs: raise_args_err("can't interpret args") return args[0] return Matcher(*args, **kwargs) def make_prototype_request(*args, **kwargs): """Make a prototype Request for a Matcher.""" if args and inspect.isclass(args[0]) and issubclass(args[0], Request): request_cls, arg_list = args[0], args[1:] return request_cls(*arg_list, **kwargs) if args and isinstance(args[0], Request): if args[1:] or kwargs: raise_args_err("can't interpret args") return args[0] # Match any opcode. return Request(*args, **kwargs) def make_reply(*args, **kwargs): # Error we might raise. if args and isinstance(args[0], OpReply): if args[1:] or kwargs: raise_args_err("can't interpret args") return args[0] return OpReply(*args, **kwargs) def unprefixed(bson_str): rep = unicode(repr(bson_str)) if rep.startswith(u'u"') or rep.startswith(u"u'"): return rep[1:] else: return rep def docs_repr(*args): """Stringify ordered dicts like a regular ones. Preserve order, remove 'u'-prefix on unicodes in Python 2: >>> print(docs_repr(OrderedDict([(u'_id', 2)]))) {"_id": 2} >>> print(docs_repr(OrderedDict([(u'_id', 2), (u'a', u'b')]), ... OrderedDict([(u'a', 1)]))) {"_id": 2, "a": "b"}, {"a": 1} >>> >>> import datetime >>> now = datetime.datetime.utcfromtimestamp(123456) >>> print(docs_repr(OrderedDict([(u'ts', now)]))) {"ts": {"$date": 123456000}} >>> >>> oid = _bson.ObjectId(b'123456781234567812345678') >>> print(docs_repr(OrderedDict([(u'oid', oid)]))) {"oid": {"$oid": "123456781234567812345678"}} """ sio = StringIO() for doc_idx, doc in enumerate(args): if doc_idx > 0: sio.write(u', ') sio.write(text_type(_json_util.dumps(doc))) return sio.getvalue() def seq_match(seq0, seq1): """True if seq0 is a subset of seq1 and their elements are in same order. >>> seq_match([], []) True >>> seq_match([1], [1]) True >>> seq_match([1, 1], [1]) False >>> seq_match([1], [1, 2]) True >>> seq_match([1, 1], [1, 1]) True >>> seq_match([3], [1, 2, 3]) True >>> seq_match([1, 3], [1, 2, 3]) True >>> seq_match([2, 1], [1, 2, 3]) False """ len_seq1 = len(seq1) if len_seq1 < len(seq0): return False seq1_idx = 0 for i, elem in enumerate(seq0): while seq1_idx < len_seq1: if seq1[seq1_idx] == elem: break seq1_idx += 1 if seq1_idx >= len_seq1 or seq1[seq1_idx] != elem: return False seq1_idx += 1 return True def format_call(frame): fn_name = inspect.getframeinfo(frame)[2] arg_info = inspect.getargvalues(frame) args = [repr(arg_info.locals[arg]) for arg in arg_info.args] varargs = [repr(x) for x in arg_info.locals[arg_info.varargs]] kwargs = [', '.join("%s=%r" % (key, value) for key, value in arg_info.locals[arg_info.keywords].items())] return '%s(%s)' % (fn_name, ', '.join(args + varargs + kwargs)) def raise_args_err(message='bad arguments', error_class=TypeError): """Throw an error with standard message, displaying function call. >>> def f(a, *args, **kwargs): ... raise_args_err() ... >>> f(1, 2, x='y') Traceback (most recent call last): ... TypeError: bad arguments: f(1, 2, x='y') """ frame = inspect.currentframe().f_back raise error_class(message + ': ' + format_call(frame)) def interactive_server(port=27017, verbose=True, all_ok=False, name='MockupDB', ssl=False): """A `MockupDB` that the mongo shell can connect to. Call `~.MockupDB.run` on the returned server, and clean it up with `~.MockupDB.stop`. If ``all_ok`` is True, replies {ok: 1} to anything unmatched by a specific responder. """ server = MockupDB(port=port, verbose=verbose, request_timeout=int(1e6), ssl=ssl, auto_ismaster=True) if all_ok: server.autoresponds({}) server.autoresponds('whatsmyuri', you='localhost:12345') server.autoresponds({'getLog': 'startupWarnings'}, log=['hello from %s!' % name]) server.autoresponds(Command('buildInfo'), version='MockupDB ' + __version__) server.autoresponds(Command('listCollections')) server.autoresponds('replSetGetStatus', ok=0) return server mockupdb-1.3.0/mockupdb/__main__.py0000644000076700000240000000312013242633741020722 0ustar emptysquarestaff00000000000000# -*- coding: utf-8 -*- # Copyright 2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Demonstrate a mocked MongoDB server.""" import time from mockupdb import interactive_server def main(): """Start an interactive `MockupDB`. Use like ``python -m mockupdb``. """ from optparse import OptionParser parser = OptionParser('Start mock MongoDB server') parser.add_option('-p', '--port', dest='port', default=27017, help='port on which mock mongod listens') parser.add_option('-q', '--quiet', action='store_false', dest='verbose', default=True, help="don't print messages to stdout") options, cmdline_args = parser.parse_args() if cmdline_args: parser.error('Unrecognized argument(s): %s' % ' '.join(cmdline_args)) server = interactive_server(port=options.port, verbose=options.verbose) try: server.run() print('Listening on port %d' % server.port) time.sleep(1e6) except KeyboardInterrupt: server.stop() if __name__ == '__main__': main() mockupdb-1.3.0/mockupdb/_bson/0000755000076700000240000000000013242634066017735 5ustar emptysquarestaff00000000000000mockupdb-1.3.0/mockupdb/_bson/__init__.py0000644000076700000240000007706412574401634022064 0ustar emptysquarestaff00000000000000# Copyright 2009-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """BSON (Binary JSON) encoding and decoding. """ import calendar import collections import datetime import itertools import re import struct import sys import uuid from codecs import (utf_8_decode as _utf_8_decode, utf_8_encode as _utf_8_encode) from mockupdb._bson.binary import (Binary, OLD_UUID_SUBTYPE, JAVA_LEGACY, CSHARP_LEGACY, UUIDLegacy) from mockupdb._bson.code import Code from mockupdb._bson.codec_options import CodecOptions, DEFAULT_CODEC_OPTIONS from mockupdb._bson.dbref import DBRef from mockupdb._bson.errors import (InvalidBSON, InvalidDocument, InvalidStringData) from mockupdb._bson.int64 import Int64 from mockupdb._bson.max_key import MaxKey from mockupdb._bson.min_key import MinKey from mockupdb._bson.objectid import ObjectId from mockupdb._bson.py3compat import (b, PY3, iteritems, text_type, string_type, reraise) from mockupdb._bson.regex import Regex from mockupdb._bson.son import SON, RE_TYPE from mockupdb._bson.timestamp import Timestamp from mockupdb._bson.tz_util import utc try: from mockupdb._bson import _cbson _USE_C = True except ImportError: _USE_C = False EPOCH_AWARE = datetime.datetime.fromtimestamp(0, utc) EPOCH_NAIVE = datetime.datetime.utcfromtimestamp(0) BSONNUM = b"\x01" # Floating point BSONSTR = b"\x02" # UTF-8 string BSONOBJ = b"\x03" # Embedded document BSONARR = b"\x04" # Array BSONBIN = b"\x05" # Binary BSONUND = b"\x06" # Undefined BSONOID = b"\x07" # ObjectId BSONBOO = b"\x08" # Boolean BSONDAT = b"\x09" # UTC Datetime BSONNUL = b"\x0A" # Null BSONRGX = b"\x0B" # Regex BSONREF = b"\x0C" # DBRef BSONCOD = b"\x0D" # Javascript code BSONSYM = b"\x0E" # Symbol BSONCWS = b"\x0F" # Javascript code with scope BSONINT = b"\x10" # 32bit int BSONTIM = b"\x11" # Timestamp BSONLON = b"\x12" # 64bit int BSONMIN = b"\xFF" # Min key BSONMAX = b"\x7F" # Max key _UNPACK_FLOAT = struct.Struct("= obj_end: raise InvalidBSON("invalid object length") obj = _elements_to_dict(data, position + 4, end, opts) position += obj_size if "$ref" in obj: return (DBRef(obj.pop("$ref"), obj.pop("$id", None), obj.pop("$db", None), obj), position) return obj, position def _get_array(data, position, obj_end, opts): """Decode a BSON array to python list.""" size = _UNPACK_INT(data[position:position + 4])[0] end = position + size - 1 if data[end:end + 1] != b"\x00": raise InvalidBSON("bad eoo") position += 4 end -= 1 result = [] # Avoid doing global and attibute lookups in the loop. append = result.append index = data.index getter = _ELEMENT_GETTER while position < end: element_type = data[position:position + 1] # Just skip the keys. position = index(b'\x00', position) + 1 value, position = getter[element_type](data, position, obj_end, opts) append(value) return result, position + 1 def _get_binary(data, position, dummy, opts): """Decode a BSON binary to bson.binary.Binary or python UUID.""" length, subtype = _UNPACK_LENGTH_SUBTYPE(data[position:position + 5]) position += 5 if subtype == 2: length2 = _UNPACK_INT(data[position:position + 4])[0] position += 4 if length2 != length - 4: raise InvalidBSON("invalid binary (st 2) - lengths don't match!") length = length2 end = position + length if subtype in (3, 4): # Java Legacy uuid_representation = opts.uuid_representation if uuid_representation == JAVA_LEGACY: java = data[position:end] value = uuid.UUID(bytes=java[0:8][::-1] + java[8:16][::-1]) # C# legacy elif uuid_representation == CSHARP_LEGACY: value = uuid.UUID(bytes_le=data[position:end]) # Python else: value = uuid.UUID(bytes=data[position:end]) return value, end # Python3 special case. Decode subtype 0 to 'bytes'. if PY3 and subtype == 0: value = data[position:end] else: value = Binary(data[position:end], subtype) return value, end def _get_oid(data, position, dummy0, dummy1): """Decode a BSON ObjectId to bson.objectid.ObjectId.""" end = position + 12 return ObjectId(data[position:end]), end def _get_boolean(data, position, dummy0, dummy1): """Decode a BSON true/false to python True/False.""" end = position + 1 return data[position:end] == b"\x01", end def _get_date(data, position, dummy, opts): """Decode a BSON datetime to python datetime.datetime.""" end = position + 8 millis = _UNPACK_LONG(data[position:end])[0] diff = ((millis % 1000) + 1000) % 1000 seconds = (millis - diff) / 1000 micros = diff * 1000 if opts.tz_aware: dt = EPOCH_AWARE + datetime.timedelta( seconds=seconds, microseconds=micros) if opts.tzinfo: dt = dt.astimezone(opts.tzinfo) else: dt = EPOCH_NAIVE + datetime.timedelta( seconds=seconds, microseconds=micros) return dt, end def _get_code(data, position, obj_end, opts): """Decode a BSON code to bson.code.Code.""" code, position = _get_string(data, position, obj_end, opts) return Code(code), position def _get_code_w_scope(data, position, obj_end, opts): """Decode a BSON code_w_scope to bson.code.Code.""" code, position = _get_string(data, position + 4, obj_end, opts) scope, position = _get_object(data, position, obj_end, opts) return Code(code, scope), position def _get_regex(data, position, dummy0, opts): """Decode a BSON regex to bson.regex.Regex or a python pattern object.""" pattern, position = _get_c_string(data, position, opts) bson_flags, position = _get_c_string(data, position, opts) bson_re = Regex(pattern, bson_flags) return bson_re, position def _get_ref(data, position, obj_end, opts): """Decode (deprecated) BSON DBPointer to bson.dbref.DBRef.""" collection, position = _get_string(data, position, obj_end, opts) oid, position = _get_oid(data, position, obj_end, opts) return DBRef(collection, oid), position def _get_timestamp(data, position, dummy0, dummy1): """Decode a BSON timestamp to bson.timestamp.Timestamp.""" end = position + 8 inc, timestamp = _UNPACK_TIMESTAMP(data[position:end]) return Timestamp(timestamp, inc), end def _get_int64(data, position, dummy0, dummy1): """Decode a BSON int64 to bson.int64.Int64.""" end = position + 8 return Int64(_UNPACK_LONG(data[position:end])[0]), end # Each decoder function's signature is: # - data: bytes # - position: int, beginning of object in 'data' to decode # - obj_end: int, end of object to decode in 'data' if variable-length type # - opts: a CodecOptions _ELEMENT_GETTER = { BSONNUM: _get_float, BSONSTR: _get_string, BSONOBJ: _get_object, BSONARR: _get_array, BSONBIN: _get_binary, BSONUND: lambda w, x, y, z: (None, x), # Deprecated undefined BSONOID: _get_oid, BSONBOO: _get_boolean, BSONDAT: _get_date, BSONNUL: lambda w, x, y, z: (None, x), BSONRGX: _get_regex, BSONREF: _get_ref, # Deprecated DBPointer BSONCOD: _get_code, BSONSYM: _get_string, # Deprecated symbol BSONCWS: _get_code_w_scope, BSONINT: _get_int, BSONTIM: _get_timestamp, BSONLON: _get_int64, BSONMIN: lambda w, x, y, z: (MinKey(), x), BSONMAX: lambda w, x, y, z: (MaxKey(), x)} def _element_to_dict(data, position, obj_end, opts): """Decode a single key, value pair.""" element_type = data[position:position + 1] position += 1 element_name, position = _get_c_string(data, position, opts) value, position = _ELEMENT_GETTER[element_type](data, position, obj_end, opts) return element_name, value, position def _elements_to_dict(data, position, obj_end, opts): """Decode a BSON document.""" result = opts.document_class() end = obj_end - 1 while position < end: (key, value, position) = _element_to_dict(data, position, obj_end, opts) result[key] = value return result def _bson_to_dict(data, opts): """Decode a BSON string to document_class.""" try: obj_size = _UNPACK_INT(data[:4])[0] except struct.error as exc: raise InvalidBSON(str(exc)) if obj_size != len(data): raise InvalidBSON("invalid object size") if data[obj_size - 1:obj_size] != b"\x00": raise InvalidBSON("bad eoo") try: return _elements_to_dict(data, 4, obj_size - 1, opts) except InvalidBSON: raise except Exception: # Change exception type to InvalidBSON but preserve traceback. _, exc_value, exc_tb = sys.exc_info() reraise(InvalidBSON, exc_value, exc_tb) if _USE_C: _bson_to_dict = _cbson._bson_to_dict _PACK_FLOAT = struct.Struct(">> import collections # From Python standard library. >>> import bson >>> from mockupdb._bson.codec_options import CodecOptions >>> data = bson.BSON.encode({'a': 1}) >>> decoded_doc = bson.BSON.decode(data) >>> options = CodecOptions(document_class=collections.OrderedDict) >>> decoded_doc = bson.BSON.decode(data, codec_options=options) >>> type(decoded_doc) :Parameters: - `codec_options` (optional): An instance of :class:`~bson.codec_options.CodecOptions`. .. versionchanged:: 3.0 Removed `compile_re` option: PyMongo now always represents BSON regular expressions as :class:`~bson.regex.Regex` objects. Use :meth:`~bson.regex.Regex.try_compile` to attempt to convert from a BSON regular expression to a Python regular expression object. Replaced `as_class`, `tz_aware`, and `uuid_subtype` options with `codec_options`. .. versionchanged:: 2.7 Added `compile_re` option. If set to False, PyMongo represented BSON regular expressions as :class:`~bson.regex.Regex` objects instead of attempting to compile BSON regular expressions as Python native regular expressions, thus preventing errors for some incompatible patterns, see `PYTHON-500`_. .. _PYTHON-500: https://jira.mongodb.org/browse/PYTHON-500 """ if not isinstance(codec_options, CodecOptions): raise _CODEC_OPTIONS_TYPE_ERROR return _bson_to_dict(self, codec_options) def has_c(): """Is the C extension installed? """ return _USE_C mockupdb-1.3.0/mockupdb/_bson/binary.py0000644000076700000240000001600512574371305021576 0ustar emptysquarestaff00000000000000# Copyright 2009-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from uuid import UUID from mockupdb._bson.py3compat import PY3 """Tools for representing BSON binary data. """ BINARY_SUBTYPE = 0 """BSON binary subtype for binary data. This is the default subtype for binary data. """ FUNCTION_SUBTYPE = 1 """BSON binary subtype for functions. """ OLD_BINARY_SUBTYPE = 2 """Old BSON binary subtype for binary data. This is the old default subtype, the current default is :data:`BINARY_SUBTYPE`. """ OLD_UUID_SUBTYPE = 3 """Old BSON binary subtype for a UUID. :class:`uuid.UUID` instances will automatically be encoded by :mod:`bson` using this subtype. .. versionadded:: 2.1 """ UUID_SUBTYPE = 4 """BSON binary subtype for a UUID. This is the new BSON binary subtype for UUIDs. The current default is :data:`OLD_UUID_SUBTYPE` but will change to this in a future release. .. versionchanged:: 2.1 Changed to subtype 4. """ STANDARD = UUID_SUBTYPE """The standard UUID representation. :class:`uuid.UUID` instances will automatically be encoded to and decoded from mockupdb._bson binary, using RFC-4122 byte order with binary subtype :data:`UUID_SUBTYPE`. .. versionadded:: 3.0 """ PYTHON_LEGACY = OLD_UUID_SUBTYPE """The Python legacy UUID representation. :class:`uuid.UUID` instances will automatically be encoded to and decoded from mockupdb._bson binary, using RFC-4122 byte order with binary subtype :data:`OLD_UUID_SUBTYPE`. .. versionadded:: 3.0 """ JAVA_LEGACY = 5 """The Java legacy UUID representation. :class:`uuid.UUID` instances will automatically be encoded to and decoded from mockupdb._bson binary, using the Java driver's legacy byte order with binary subtype :data:`OLD_UUID_SUBTYPE`. .. versionadded:: 2.3 """ CSHARP_LEGACY = 6 """The C#/.net legacy UUID representation. :class:`uuid.UUID` instances will automatically be encoded to and decoded from mockupdb._bson binary, using the C# driver's legacy byte order and binary subtype :data:`OLD_UUID_SUBTYPE`. .. versionadded:: 2.3 """ ALL_UUID_SUBTYPES = (OLD_UUID_SUBTYPE, UUID_SUBTYPE) ALL_UUID_REPRESENTATIONS = (STANDARD, PYTHON_LEGACY, JAVA_LEGACY, CSHARP_LEGACY) UUID_REPRESENTATION_NAMES = { PYTHON_LEGACY: 'PYTHON_LEGACY', STANDARD: 'STANDARD', JAVA_LEGACY: 'JAVA_LEGACY', CSHARP_LEGACY: 'CSHARP_LEGACY'} MD5_SUBTYPE = 5 """BSON binary subtype for an MD5 hash. """ USER_DEFINED_SUBTYPE = 128 """BSON binary subtype for any user defined structure. """ class Binary(bytes): """Representation of BSON binary data. This is necessary because we want to represent Python strings as the BSON string type. We need to wrap binary data so we can tell the difference between what should be considered binary data and what should be considered a string when we encode to BSON. Raises TypeError if `data` is not an instance of :class:`str` (:class:`bytes` in python 3) or `subtype` is not an instance of :class:`int`. Raises ValueError if `subtype` is not in [0, 256). .. note:: In python 3 instances of Binary with subtype 0 will be decoded directly to :class:`bytes`. :Parameters: - `data`: the binary data to represent - `subtype` (optional): the `binary subtype `_ to use """ _type_marker = 5 def __new__(cls, data, subtype=BINARY_SUBTYPE): if not isinstance(data, bytes): raise TypeError("data must be an instance of bytes") if not isinstance(subtype, int): raise TypeError("subtype must be an instance of int") if subtype >= 256 or subtype < 0: raise ValueError("subtype must be contained in [0, 256)") self = bytes.__new__(cls, data) self.__subtype = subtype return self @property def subtype(self): """Subtype of this binary data. """ return self.__subtype def __getnewargs__(self): # Work around http://bugs.python.org/issue7382 data = super(Binary, self).__getnewargs__()[0] if PY3 and not isinstance(data, bytes): data = data.encode('latin-1') return data, self.__subtype def __eq__(self, other): if isinstance(other, Binary): return ((self.__subtype, bytes(self)) == (other.subtype, bytes(other))) # We don't return NotImplemented here because if we did then # Binary("foo") == "foo" would return True, since Binary is a # subclass of str... return False def __hash__(self): return super(Binary, self).__hash__() ^ hash(self.__subtype) def __ne__(self, other): return not self == other def __repr__(self): return "Binary(%s, %s)" % (bytes.__repr__(self), self.__subtype) class UUIDLegacy(Binary): """UUID wrapper to support working with UUIDs stored as PYTHON_LEGACY. .. doctest:: >>> import uuid >>> from mockupdb._bson.binary import Binary, UUIDLegacy, STANDARD >>> from mockupdb._bson.codec_options import CodecOptions >>> my_uuid = uuid.uuid4() >>> coll = db.get_collection('test', ... CodecOptions(uuid_representation=STANDARD)) >>> coll.insert_one({'uuid': Binary(my_uuid.bytes, 3)}).inserted_id ObjectId('...') >>> coll.find({'uuid': my_uuid}).count() 0 >>> coll.find({'uuid': UUIDLegacy(my_uuid)}).count() 1 >>> coll.find({'uuid': UUIDLegacy(my_uuid)})[0]['uuid'] UUID('...') >>> >>> # Convert from subtype 3 to subtype 4 >>> doc = coll.find_one({'uuid': UUIDLegacy(my_uuid)}) >>> coll.replace_one({"_id": doc["_id"]}, doc).matched_count 1 >>> coll.find({'uuid': UUIDLegacy(my_uuid)}).count() 0 >>> coll.find({'uuid': {'$in': [UUIDLegacy(my_uuid), my_uuid]}}).count() 1 >>> coll.find_one({'uuid': my_uuid})['uuid'] UUID('...') Raises TypeError if `obj` is not an instance of :class:`~uuid.UUID`. :Parameters: - `obj`: An instance of :class:`~uuid.UUID`. """ def __new__(cls, obj): if not isinstance(obj, UUID): raise TypeError("obj must be an instance of uuid.UUID") self = Binary.__new__(cls, obj.bytes, OLD_UUID_SUBTYPE) self.__uuid = obj return self def __getnewargs__(self): # Support copy and deepcopy return (self.__uuid,) @property def uuid(self): """UUID instance wrapped by this UUIDLegacy instance. """ return self.__uuid def __repr__(self): return "UUIDLegacy('%s')" % self.__uuid mockupdb-1.3.0/mockupdb/_bson/code.py0000644000076700000240000000511512574371305021224 0ustar emptysquarestaff00000000000000# Copyright 2009-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Tools for representing JavaScript code in BSON. """ import collections from mockupdb._bson.py3compat import string_type class Code(str): """BSON's JavaScript code type. Raises :class:`TypeError` if `code` is not an instance of :class:`basestring` (:class:`str` in python 3) or `scope` is not ``None`` or an instance of :class:`dict`. Scope variables can be set by passing a dictionary as the `scope` argument or by using keyword arguments. If a variable is set as a keyword argument it will override any setting for that variable in the `scope` dictionary. :Parameters: - `code`: string containing JavaScript code to be evaluated - `scope` (optional): dictionary representing the scope in which `code` should be evaluated - a mapping from identifiers (as strings) to values - `**kwargs` (optional): scope variables can also be passed as keyword arguments """ _type_marker = 13 def __new__(cls, code, scope=None, **kwargs): if not isinstance(code, string_type): raise TypeError("code must be an " "instance of %s" % (string_type.__name__)) self = str.__new__(cls, code) try: self.__scope = code.scope except AttributeError: self.__scope = {} if scope is not None: if not isinstance(scope, collections.Mapping): raise TypeError("scope must be an instance of dict") self.__scope.update(scope) self.__scope.update(kwargs) return self @property def scope(self): """Scope dictionary for this instance. """ return self.__scope def __repr__(self): return "Code(%s, %r)" % (str.__repr__(self), self.__scope) def __eq__(self, other): if isinstance(other, Code): return (self.__scope, str(self)) == (other.__scope, str(other)) return False __hash__ = None def __ne__(self, other): return not self == other mockupdb-1.3.0/mockupdb/_bson/codec_options.py0000644000076700000240000001204412574421162023136 0ustar emptysquarestaff00000000000000# Copyright 2014-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Tools for specifying BSON codec options.""" import datetime from collections import MutableMapping, namedtuple from mockupdb._bson.py3compat import string_type from mockupdb._bson.binary import (ALL_UUID_REPRESENTATIONS, PYTHON_LEGACY, UUID_REPRESENTATION_NAMES) _options_base = namedtuple( 'CodecOptions', ('document_class', 'tz_aware', 'uuid_representation', 'unicode_decode_error_handler', 'tzinfo')) class CodecOptions(_options_base): """Encapsulates BSON options used in CRUD operations. :Parameters: - `document_class`: BSON documents returned in queries will be decoded to an instance of this class. Must be a subclass of :class:`~collections.MutableMapping`. Defaults to :class:`dict`. - `tz_aware`: If ``True``, BSON datetimes will be decoded to timezone aware instances of :class:`~datetime.datetime`. Otherwise they will be naive. Defaults to ``False``. - `uuid_representation`: The BSON representation to use when encoding and decoding instances of :class:`~uuid.UUID`. Defaults to :data:`~bson.binary.PYTHON_LEGACY`. - `unicode_decode_error_handler`: The error handler to use when decoding an invalid BSON string. Valid options include 'strict', 'replace', and 'ignore'. Defaults to 'strict'. .. warning:: Care must be taken when changing `unicode_decode_error_handler` from its default value ('strict'). The 'replace' and 'ignore' modes should not be used when documents retrieved from the server will be modified in the client application and stored back to the server. - `tzinfo`: A :class:`~datetime.tzinfo` subclass that specifies the timezone to/from which :class:`~datetime.datetime` objects should be encoded/decoded. """ def __new__(cls, document_class=dict, tz_aware=False, uuid_representation=PYTHON_LEGACY, unicode_decode_error_handler="strict", tzinfo=None): if not issubclass(document_class, MutableMapping): raise TypeError("document_class must be dict, bson.son.SON, or " "another subclass of collections.MutableMapping") if not isinstance(tz_aware, bool): raise TypeError("tz_aware must be True or False") if uuid_representation not in ALL_UUID_REPRESENTATIONS: raise ValueError("uuid_representation must be a value " "from mockupdb._bson.binary.ALL_UUID_REPRESENTATIONS") if not isinstance(unicode_decode_error_handler, (string_type, None)): raise ValueError("unicode_decode_error_handler must be a string " "or None") if tzinfo is not None: if not isinstance(tzinfo, datetime.tzinfo): raise TypeError( "tzinfo must be an instance of datetime.tzinfo") if not tz_aware: raise ValueError( "cannot specify tzinfo without also setting tz_aware=True") return tuple.__new__( cls, (document_class, tz_aware, uuid_representation, unicode_decode_error_handler, tzinfo)) def __repr__(self): document_class_repr = ( 'dict' if self.document_class is dict else repr(self.document_class)) uuid_rep_repr = UUID_REPRESENTATION_NAMES.get(self.uuid_representation, self.uuid_representation) return ( 'CodecOptions(document_class=%s, tz_aware=%r, uuid_representation=' '%s, unicode_decode_error_handler=%r, tzinfo=%r)' % (document_class_repr, self.tz_aware, uuid_rep_repr, self.unicode_decode_error_handler, self.tzinfo)) DEFAULT_CODEC_OPTIONS = CodecOptions() def _parse_codec_options(options): """Parse BSON codec options.""" return CodecOptions( document_class=options.get( 'document_class', DEFAULT_CODEC_OPTIONS.document_class), tz_aware=options.get( 'tz_aware', DEFAULT_CODEC_OPTIONS.tz_aware), uuid_representation=options.get( 'uuidrepresentation', DEFAULT_CODEC_OPTIONS.uuid_representation), unicode_decode_error_handler=options.get( 'unicode_decode_error_handler', DEFAULT_CODEC_OPTIONS.unicode_decode_error_handler), tzinfo=options.get('tzinfo', DEFAULT_CODEC_OPTIONS.tzinfo)) mockupdb-1.3.0/mockupdb/_bson/dbref.py0000644000076700000240000001122112574371305021367 0ustar emptysquarestaff00000000000000# Copyright 2009-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Tools for manipulating DBRefs (references to MongoDB documents).""" from copy import deepcopy from mockupdb._bson.py3compat import iteritems, string_type from mockupdb._bson.son import SON class DBRef(object): """A reference to a document stored in MongoDB. """ # DBRef isn't actually a BSON "type" so this number was arbitrarily chosen. _type_marker = 100 def __init__(self, collection, id, database=None, _extra={}, **kwargs): """Initialize a new :class:`DBRef`. Raises :class:`TypeError` if `collection` or `database` is not an instance of :class:`basestring` (:class:`str` in python 3). `database` is optional and allows references to documents to work across databases. Any additional keyword arguments will create additional fields in the resultant embedded document. :Parameters: - `collection`: name of the collection the document is stored in - `id`: the value of the document's ``"_id"`` field - `database` (optional): name of the database to reference - `**kwargs` (optional): additional keyword arguments will create additional, custom fields .. mongodoc:: dbrefs """ if not isinstance(collection, string_type): raise TypeError("collection must be an " "instance of %s" % string_type.__name__) if database is not None and not isinstance(database, string_type): raise TypeError("database must be an " "instance of %s" % string_type.__name__) self.__collection = collection self.__id = id self.__database = database kwargs.update(_extra) self.__kwargs = kwargs @property def collection(self): """Get the name of this DBRef's collection as unicode. """ return self.__collection @property def id(self): """Get this DBRef's _id. """ return self.__id @property def database(self): """Get the name of this DBRef's database. Returns None if this DBRef doesn't specify a database. """ return self.__database def __getattr__(self, key): try: return self.__kwargs[key] except KeyError: raise AttributeError(key) # Have to provide __setstate__ to avoid # infinite recursion since we override # __getattr__. def __setstate__(self, state): self.__dict__.update(state) def as_doc(self): """Get the SON document representation of this DBRef. Generally not needed by application developers """ doc = SON([("$ref", self.collection), ("$id", self.id)]) if self.database is not None: doc["$db"] = self.database doc.update(self.__kwargs) return doc def __repr__(self): extra = "".join([", %s=%r" % (k, v) for k, v in iteritems(self.__kwargs)]) if self.database is None: return "DBRef(%r, %r%s)" % (self.collection, self.id, extra) return "DBRef(%r, %r, %r%s)" % (self.collection, self.id, self.database, extra) def __eq__(self, other): if isinstance(other, DBRef): us = (self.__database, self.__collection, self.__id, self.__kwargs) them = (other.__database, other.__collection, other.__id, other.__kwargs) return us == them return NotImplemented def __ne__(self, other): return not self == other def __hash__(self): """Get a hash value for this :class:`DBRef`.""" return hash((self.__collection, self.__id, self.__database, tuple(sorted(self.__kwargs.items())))) def __deepcopy__(self, memo): """Support function for `copy.deepcopy()`.""" return DBRef(deepcopy(self.__collection, memo), deepcopy(self.__id, memo), deepcopy(self.__database, memo), deepcopy(self.__kwargs, memo)) mockupdb-1.3.0/mockupdb/_bson/errors.py0000644000076700000240000000220412547632703021624 0ustar emptysquarestaff00000000000000# Copyright 2009-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Exceptions raised by the BSON package.""" class BSONError(Exception): """Base class for all BSON exceptions. """ class InvalidBSON(BSONError): """Raised when trying to create a BSON object from invalid data. """ class InvalidStringData(BSONError): """Raised when trying to encode a string containing non-UTF8 data. """ class InvalidDocument(BSONError): """Raised when trying to create a BSON object from an invalid document. """ class InvalidId(BSONError): """Raised when trying to create an ObjectId from invalid data. """ mockupdb-1.3.0/mockupdb/_bson/int64.py0000644000076700000240000000205212574371305021253 0ustar emptysquarestaff00000000000000# Copyright 2014-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """A BSON wrapper for long (int in python3)""" from mockupdb._bson.py3compat import PY3 if PY3: long = int class Int64(long): """Representation of the BSON int64 type. This is necessary because every integral number is an :class:`int` in Python 3. Small integral numbers are encoded to BSON int32 by default, but Int64 numbers will always be encoded to BSON int64. :Parameters: - `value`: the numeric value to represent """ _type_marker = 18 mockupdb-1.3.0/mockupdb/_bson/json_util.py0000644000076700000240000002267412574371305022331 0ustar emptysquarestaff00000000000000# Copyright 2009-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Tools for using Python's :mod:`json` module with BSON documents. This module provides two helper methods `dumps` and `loads` that wrap the native :mod:`json` methods and provide explicit BSON conversion to and from json. This allows for specialized encoding and decoding of BSON documents into `Mongo Extended JSON `_'s *Strict* mode. This lets you encode / decode BSON documents to JSON even when they use special BSON types. Example usage (serialization): .. doctest:: >>> from mockupdb._bson import Binary, Code >>> from mockupdb._bson.json_util import dumps >>> dumps([{'foo': [1, 2]}, ... {'bar': {'hello': 'world'}}, ... {'code': Code("function x() { return 1; }")}, ... {'bin': Binary("\x01\x02\x03\x04")}]) '[{"foo": [1, 2]}, {"bar": {"hello": "world"}}, {"code": {"$code": "function x() { return 1; }", "$scope": {}}}, {"bin": {"$binary": "AQIDBA==", "$type": "00"}}]' Example usage (deserialization): .. doctest:: >>> from mockupdb._bson.json_util import loads >>> loads('[{"foo": [1, 2]}, {"bar": {"hello": "world"}}, {"code": {"$scope": {}, "$code": "function x() { return 1; }"}}, {"bin": {"$type": "00", "$binary": "AQIDBA=="}}]') [{u'foo': [1, 2]}, {u'bar': {u'hello': u'world'}}, {u'code': Code('function x() { return 1; }', {})}, {u'bin': Binary('...', 0)}] Alternatively, you can manually pass the `default` to :func:`json.dumps`. It won't handle :class:`~bson.binary.Binary` and :class:`~bson.code.Code` instances (as they are extended strings you can't provide custom defaults), but it will be faster as there is less recursion. .. versionchanged:: 2.8 The output format for :class:`~bson.timestamp.Timestamp` has changed from '{"t": , "i": }' to '{"$timestamp": {"t": , "i": }}'. This new format will be decoded to an instance of :class:`~bson.timestamp.Timestamp`. The old format will continue to be decoded to a python dict as before. Encoding to the old format is no longer supported as it was never correct and loses type information. Added support for $numberLong and $undefined - new in MongoDB 2.6 - and parsing $date in ISO-8601 format. .. versionchanged:: 2.7 Preserves order when rendering SON, Timestamp, Code, Binary, and DBRef instances. .. versionchanged:: 2.3 Added dumps and loads helpers to automatically handle conversion to and from json and supports :class:`~bson.binary.Binary` and :class:`~bson.code.Code` """ import base64 import calendar import collections import datetime import json import re import uuid from mockupdb._bson import EPOCH_AWARE, RE_TYPE, SON from mockupdb._bson.binary import Binary from mockupdb._bson.code import Code from mockupdb._bson.dbref import DBRef from mockupdb._bson.int64 import Int64 from mockupdb._bson.max_key import MaxKey from mockupdb._bson.min_key import MinKey from mockupdb._bson.objectid import ObjectId from mockupdb._bson.regex import Regex from mockupdb._bson.timestamp import Timestamp from mockupdb._bson.tz_util import utc from mockupdb._bson.py3compat import PY3, iteritems, string_type, text_type _RE_OPT_TABLE = { "i": re.I, "l": re.L, "m": re.M, "s": re.S, "u": re.U, "x": re.X, } def dumps(obj, *args, **kwargs): """Helper function that wraps :class:`json.dumps`. Recursive function that handles all BSON types including :class:`~bson.binary.Binary` and :class:`~bson.code.Code`. .. versionchanged:: 2.7 Preserves order when rendering SON, Timestamp, Code, Binary, and DBRef instances. """ return json.dumps(_json_convert(obj), *args, **kwargs) def loads(s, *args, **kwargs): """Helper function that wraps :class:`json.loads`. Automatically passes the object_hook for BSON type conversion. """ kwargs['object_hook'] = lambda dct: object_hook(dct) return json.loads(s, *args, **kwargs) def _json_convert(obj): """Recursive helper method that converts BSON types so they can be converted into json. """ if hasattr(obj, 'iteritems') or hasattr(obj, 'items'): # PY3 support return SON(((k, _json_convert(v)) for k, v in iteritems(obj))) elif hasattr(obj, '__iter__') and not isinstance(obj, (text_type, bytes)): return list((_json_convert(v) for v in obj)) try: return default(obj) except TypeError: return obj def object_hook(dct): if "$oid" in dct: return ObjectId(str(dct["$oid"])) if "$ref" in dct: return DBRef(dct["$ref"], dct["$id"], dct.get("$db", None)) if "$date" in dct: dtm = dct["$date"] # mongoexport 2.6 and newer if isinstance(dtm, string_type): aware = datetime.datetime.strptime( dtm[:23], "%Y-%m-%dT%H:%M:%S.%f").replace(tzinfo=utc) offset = dtm[23:] if not offset or offset == 'Z': # UTC return aware else: if len(offset) == 5: # Offset from mongoexport is in format (+|-)HHMM secs = (int(offset[1:3]) * 3600 + int(offset[3:]) * 60) elif ':' in offset and len(offset) == 6: # RFC-3339 format (+|-)HH:MM hours, minutes = offset[1:].split(':') secs = (int(hours) * 3600 + int(minutes) * 60) else: # Not RFC-3339 compliant or mongoexport output. raise ValueError("invalid format for offset") if offset[0] == "-": secs *= -1 return aware - datetime.timedelta(seconds=secs) # mongoexport 2.6 and newer, time before the epoch (SERVER-15275) elif isinstance(dtm, collections.Mapping): secs = float(dtm["$numberLong"]) / 1000.0 # mongoexport before 2.6 else: secs = float(dtm) / 1000.0 return EPOCH_AWARE + datetime.timedelta(seconds=secs) if "$regex" in dct: flags = 0 # PyMongo always adds $options but some other tools may not. for opt in dct.get("$options", ""): flags |= _RE_OPT_TABLE.get(opt, 0) return Regex(dct["$regex"], flags) if "$minKey" in dct: return MinKey() if "$maxKey" in dct: return MaxKey() if "$binary" in dct: if isinstance(dct["$type"], int): dct["$type"] = "%02x" % dct["$type"] subtype = int(dct["$type"], 16) if subtype >= 0xffffff80: # Handle mongoexport values subtype = int(dct["$type"][6:], 16) return Binary(base64.b64decode(dct["$binary"].encode()), subtype) if "$code" in dct: return Code(dct["$code"], dct.get("$scope")) if "$uuid" in dct: return uuid.UUID(dct["$uuid"]) if "$undefined" in dct: return None if "$numberLong" in dct: return Int64(dct["$numberLong"]) if "$timestamp" in dct: tsp = dct["$timestamp"] return Timestamp(tsp["t"], tsp["i"]) return dct def default(obj): # We preserve key order when rendering SON, DBRef, etc. as JSON by # returning a SON for those types instead of a dict. if isinstance(obj, ObjectId): return {"$oid": str(obj)} if isinstance(obj, DBRef): return _json_convert(obj.as_doc()) if isinstance(obj, datetime.datetime): # TODO share this code w/ bson.py? if obj.utcoffset() is not None: obj = obj - obj.utcoffset() millis = int(calendar.timegm(obj.timetuple()) * 1000 + obj.microsecond / 1000) return {"$date": millis} if isinstance(obj, (RE_TYPE, Regex)): flags = "" if obj.flags & re.IGNORECASE: flags += "i" if obj.flags & re.LOCALE: flags += "l" if obj.flags & re.MULTILINE: flags += "m" if obj.flags & re.DOTALL: flags += "s" if obj.flags & re.UNICODE: flags += "u" if obj.flags & re.VERBOSE: flags += "x" if isinstance(obj.pattern, text_type): pattern = obj.pattern else: pattern = obj.pattern.decode('utf-8') return SON([("$regex", pattern), ("$options", flags)]) if isinstance(obj, MinKey): return {"$minKey": 1} if isinstance(obj, MaxKey): return {"$maxKey": 1} if isinstance(obj, Timestamp): return {"$timestamp": SON([("t", obj.time), ("i", obj.inc)])} if isinstance(obj, Code): return SON([('$code', str(obj)), ('$scope', obj.scope)]) if isinstance(obj, Binary): return SON([ ('$binary', base64.b64encode(obj).decode()), ('$type', "%02x" % obj.subtype)]) if PY3 and isinstance(obj, bytes): return SON([ ('$binary', base64.b64encode(obj).decode()), ('$type', "00")]) if isinstance(obj, uuid.UUID): return {"$uuid": obj.hex} raise TypeError("%r is not JSON serializable" % obj) mockupdb-1.3.0/mockupdb/_bson/max_key.py0000644000076700000240000000245012573362134021745 0ustar emptysquarestaff00000000000000# Copyright 2010-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Representation for the MongoDB internal MaxKey type. """ class MaxKey(object): """MongoDB internal MaxKey type. .. versionchanged:: 2.7 ``MaxKey`` now implements comparison operators. """ _type_marker = 127 def __eq__(self, other): return isinstance(other, MaxKey) def __hash__(self): return hash(self._type_marker) def __ne__(self, other): return not self == other def __le__(self, other): return isinstance(other, MaxKey) def __lt__(self, dummy): return False def __ge__(self, dummy): return True def __gt__(self, other): return not isinstance(other, MaxKey) def __repr__(self): return "MaxKey()" mockupdb-1.3.0/mockupdb/_bson/min_key.py0000644000076700000240000000245412573362134021747 0ustar emptysquarestaff00000000000000# Copyright 2010-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Representation for the MongoDB internal MinKey type. """ class MinKey(object): """MongoDB internal MinKey type. .. versionchanged:: 2.7 ``MinKey`` now implements comparison operators. """ _type_marker = 255 def __eq__(self, other): return isinstance(other, MinKey) def __hash__(self): return hash(self._type_marker) def __ne__(self, other): return not self == other def __le__(self, dummy): return True def __lt__(self, other): return not isinstance(other, MinKey) def __ge__(self, other): return isinstance(other, MinKey) def __gt__(self, dummy): return False def __repr__(self): return "MinKey()" mockupdb-1.3.0/mockupdb/_bson/objectid.py0000644000076700000240000002213413241373712022071 0ustar emptysquarestaff00000000000000# Copyright 2009-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Tools for working with MongoDB `ObjectIds `_. """ import binascii import calendar import datetime import hashlib import os import random import socket import struct import threading import time from mockupdb._bson.errors import InvalidId from mockupdb._bson.py3compat import PY3, bytes_from_hex, string_type, text_type from mockupdb._bson.tz_util import utc def _machine_bytes(): """Get the machine portion of an ObjectId. """ machine_hash = hashlib.md5() if PY3: # gethostname() returns a unicode string in python 3.x # while update() requires a byte string. machine_hash.update(socket.gethostname().encode()) else: # Calling encode() here will fail with non-ascii hostnames machine_hash.update(socket.gethostname()) return machine_hash.digest()[0:3] def _raise_invalid_id(oid): raise InvalidId( "%r is not a valid ObjectId, it must be a 12-byte input" " or a 24-character hex string" % oid) class ObjectId(object): """A MongoDB ObjectId. """ _inc = random.randint(0, 0xFFFFFF) _inc_lock = threading.Lock() _machine_bytes = _machine_bytes() __slots__ = ('__id') _type_marker = 7 def __init__(self, oid=None): """Initialize a new ObjectId. An ObjectId is a 12-byte unique identifier consisting of: - a 4-byte value representing the seconds since the Unix epoch, - a 3-byte machine identifier, - a 2-byte process id, and - a 3-byte counter, starting with a random value. By default, ``ObjectId()`` creates a new unique identifier. The optional parameter `oid` can be an :class:`ObjectId`, or any 12 :class:`bytes` or, in Python 2, any 12-character :class:`str`. For example, the 12 bytes b'foo-bar-quux' do not follow the ObjectId specification but they are acceptable input:: >>> ObjectId(b'foo-bar-quux') ObjectId('666f6f2d6261722d71757578') `oid` can also be a :class:`unicode` or :class:`str` of 24 hex digits:: >>> ObjectId('0123456789ab0123456789ab') ObjectId('0123456789ab0123456789ab') >>> >>> # A u-prefixed unicode literal: >>> ObjectId(u'0123456789ab0123456789ab') ObjectId('0123456789ab0123456789ab') Raises :class:`~bson.errors.InvalidId` if `oid` is not 12 bytes nor 24 hex digits, or :class:`TypeError` if `oid` is not an accepted type. :Parameters: - `oid` (optional): a valid ObjectId. .. mongodoc:: objectids """ if oid is None: self.__generate() elif isinstance(oid, bytes) and len(oid) == 12: self.__id = oid else: self.__validate(oid) @classmethod def from_datetime(cls, generation_time): """Create a dummy ObjectId instance with a specific generation time. This method is useful for doing range queries on a field containing :class:`ObjectId` instances. .. warning:: It is not safe to insert a document containing an ObjectId generated using this method. This method deliberately eliminates the uniqueness guarantee that ObjectIds generally provide. ObjectIds generated with this method should be used exclusively in queries. `generation_time` will be converted to UTC. Naive datetime instances will be treated as though they already contain UTC. An example using this helper to get documents where ``"_id"`` was generated before January 1, 2010 would be: >>> gen_time = datetime.datetime(2010, 1, 1) >>> dummy_id = ObjectId.from_datetime(gen_time) >>> result = collection.find({"_id": {"$lt": dummy_id}}) :Parameters: - `generation_time`: :class:`~datetime.datetime` to be used as the generation time for the resulting ObjectId. """ if generation_time.utcoffset() is not None: generation_time = generation_time - generation_time.utcoffset() timestamp = calendar.timegm(generation_time.timetuple()) oid = struct.pack( ">i", int(timestamp)) + b"\x00\x00\x00\x00\x00\x00\x00\x00" return cls(oid) @classmethod def is_valid(cls, oid): """Checks if a `oid` string is valid or not. :Parameters: - `oid`: the object id to validate .. versionadded:: 2.3 """ if not oid: return False try: ObjectId(oid) return True except (InvalidId, TypeError): return False def __generate(self): """Generate a new value for this ObjectId. """ # 4 bytes current time oid = struct.pack(">i", int(time.time())) # 3 bytes machine oid += ObjectId._machine_bytes # 2 bytes pid oid += struct.pack(">H", os.getpid() % 0xFFFF) # 3 bytes inc with ObjectId._inc_lock: oid += struct.pack(">i", ObjectId._inc)[1:4] ObjectId._inc = (ObjectId._inc + 1) % 0xFFFFFF self.__id = oid def __validate(self, oid): """Validate and use the given id for this ObjectId. Raises TypeError if id is not an instance of (:class:`basestring` (:class:`str` or :class:`bytes` in python 3), ObjectId) and InvalidId if it is not a valid ObjectId. :Parameters: - `oid`: a valid ObjectId """ if isinstance(oid, ObjectId): self.__id = oid.binary # bytes or unicode in python 2, str in python 3 elif isinstance(oid, string_type): if len(oid) == 24: try: self.__id = bytes_from_hex(oid) except (TypeError, ValueError): _raise_invalid_id(oid) else: _raise_invalid_id(oid) else: raise TypeError("id must be an instance of (bytes, %s, ObjectId), " "not %s" % (text_type.__name__, type(oid))) @property def binary(self): """12-byte binary representation of this ObjectId. """ return self.__id @property def generation_time(self): """A :class:`datetime.datetime` instance representing the time of generation for this :class:`ObjectId`. The :class:`datetime.datetime` is timezone aware, and represents the generation time in UTC. It is precise to the second. """ timestamp = struct.unpack(">i", self.__id[0:4])[0] return datetime.datetime.fromtimestamp(timestamp, utc) def __getstate__(self): """return value of object for pickling. needed explicitly because __slots__() defined. """ return self.__id def __setstate__(self, value): """explicit state set from pickling """ # Provide backwards compatibility with OIDs # pickled with pymongo-1.9 or older. if isinstance(value, dict): oid = value["_ObjectId__id"] else: oid = value # ObjectIds pickled in python 2.x used `str` for __id. # In python 3.x this has to be converted to `bytes` # by encoding latin-1. if PY3 and isinstance(oid, text_type): self.__id = oid.encode('latin-1') else: self.__id = oid def __str__(self): if PY3: return binascii.hexlify(self.__id).decode() return binascii.hexlify(self.__id) def __repr__(self): return "ObjectId('%s')" % (str(self),) def __eq__(self, other): if isinstance(other, ObjectId): return self.__id == other.binary return NotImplemented def __ne__(self, other): if isinstance(other, ObjectId): return self.__id != other.binary return NotImplemented def __lt__(self, other): if isinstance(other, ObjectId): return self.__id < other.binary return NotImplemented def __le__(self, other): if isinstance(other, ObjectId): return self.__id <= other.binary return NotImplemented def __gt__(self, other): if isinstance(other, ObjectId): return self.__id > other.binary return NotImplemented def __ge__(self, other): if isinstance(other, ObjectId): return self.__id >= other.binary return NotImplemented def __hash__(self): """Get a hash value for this :class:`ObjectId`.""" return hash(self.__id) mockupdb-1.3.0/mockupdb/_bson/py3compat.py0000644000076700000240000000471412547632703022237 0ustar emptysquarestaff00000000000000# Copyright 2009-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); you # may not use this file except in compliance with the License. You # may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. See the License for the specific language governing # permissions and limitations under the License. """Utility functions and definitions for python3 compatibility.""" import sys PY3 = sys.version_info[0] == 3 if PY3: import codecs import _thread as thread from io import BytesIO as StringIO MAXSIZE = sys.maxsize imap = map def b(s): # BSON and socket operations deal in binary data. In # python 3 that means instances of `bytes`. In python # 2.6 and 2.7 you can create an alias for `bytes` using # the b prefix (e.g. b'foo'). # See http://python3porting.com/problems.html#nicer-solutions return codecs.latin_1_encode(s)[0] def u(s): # PY3 strings may already be treated as unicode literals return s def bytes_from_hex(h): return bytes.fromhex(h) def iteritems(d): return iter(d.items()) def itervalues(d): return iter(d.values()) def reraise(exctype, value, trace=None): raise exctype(str(value)).with_traceback(trace) def _unicode(s): return s text_type = str string_type = str integer_types = int else: import thread from itertools import imap try: from cStringIO import StringIO except ImportError: from StringIO import StringIO MAXSIZE = sys.maxint def b(s): # See comments above. In python 2.x b('foo') is just 'foo'. return s def u(s): """Replacement for unicode literal prefix.""" return unicode(s.replace('\\', '\\\\'), 'unicode_escape') def bytes_from_hex(h): return h.decode('hex') def iteritems(d): return d.iteritems() def itervalues(d): return d.itervalues() # "raise x, y, z" raises SyntaxError in Python 3 exec("""def reraise(exctype, value, trace=None): raise exctype, str(value), trace """) _unicode = unicode string_type = basestring text_type = unicode integer_types = (int, long) mockupdb-1.3.0/mockupdb/_bson/regex.py0000644000076700000240000001032312574371305021421 0ustar emptysquarestaff00000000000000# Copyright 2013-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Tools for representing MongoDB regular expressions. """ import re from mockupdb._bson.son import RE_TYPE from mockupdb._bson.py3compat import string_type, text_type def str_flags_to_int(str_flags): flags = 0 if "i" in str_flags: flags |= re.IGNORECASE if "l" in str_flags: flags |= re.LOCALE if "m" in str_flags: flags |= re.MULTILINE if "s" in str_flags: flags |= re.DOTALL if "u" in str_flags: flags |= re.UNICODE if "x" in str_flags: flags |= re.VERBOSE return flags class Regex(object): """BSON regular expression data.""" _type_marker = 11 @classmethod def from_native(cls, regex): """Convert a Python regular expression into a ``Regex`` instance. Note that in Python 3, a regular expression compiled from a :class:`str` has the ``re.UNICODE`` flag set. If it is undesirable to store this flag in a BSON regular expression, unset it first:: >>> pattern = re.compile('.*') >>> regex = Regex.from_native(pattern) >>> regex.flags ^= re.UNICODE >>> db.collection.insert({'pattern': regex}) :Parameters: - `regex`: A regular expression object from ``re.compile()``. .. warning:: Python regular expressions use a different syntax and different set of flags than MongoDB, which uses `PCRE`_. A regular expression retrieved from the server may not compile in Python, or may match a different set of strings in Python than when used in a MongoDB query. .. _PCRE: http://www.pcre.org/ """ if not isinstance(regex, RE_TYPE): raise TypeError( "regex must be a compiled regular expression, not %s" % type(regex)) return Regex(regex.pattern, regex.flags) def __init__(self, pattern, flags=0): """BSON regular expression data. This class is useful to store and retrieve regular expressions that are incompatible with Python's regular expression dialect. :Parameters: - `pattern`: string - `flags`: (optional) an integer bitmask, or a string of flag characters like "im" for IGNORECASE and MULTILINE """ if not isinstance(pattern, (text_type, bytes)): raise TypeError("pattern must be a string, not %s" % type(pattern)) self.pattern = pattern if isinstance(flags, string_type): self.flags = str_flags_to_int(flags) elif isinstance(flags, int): self.flags = flags else: raise TypeError( "flags must be a string or int, not %s" % type(flags)) def __eq__(self, other): if isinstance(other, Regex): return self.pattern == self.pattern and self.flags == other.flags else: return NotImplemented __hash__ = None def __ne__(self, other): return not self == other def __repr__(self): return "Regex(%r, %r)" % (self.pattern, self.flags) def try_compile(self): """Compile this :class:`Regex` as a Python regular expression. .. warning:: Python regular expressions use a different syntax and different set of flags than MongoDB, which uses `PCRE`_. A regular expression retrieved from the server may not compile in Python, or may match a different set of strings in Python than when used in a MongoDB query. :meth:`try_compile()` may raise :exc:`re.error`. .. _PCRE: http://www.pcre.org/ """ return re.compile(self.pattern, self.flags) mockupdb-1.3.0/mockupdb/_bson/son.py0000644000076700000240000002100012574371305021100 0ustar emptysquarestaff00000000000000# Copyright 2009-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Tools for creating and manipulating SON, the Serialized Ocument Notation. Regular dictionaries can be used instead of SON objects, but not when the order of keys is important. A SON object can be used just like a normal Python dictionary.""" import collections import copy import re from mockupdb._bson.py3compat import iteritems # This sort of sucks, but seems to be as good as it gets... # This is essentially the same as re._pattern_type RE_TYPE = type(re.compile("")) class SON(dict): """SON data. A subclass of dict that maintains ordering of keys and provides a few extra niceties for dealing with SON. SON objects can be converted to and from mockupdb._bson. The mapping from Python types to BSON types is as follows: ======================================= ============= =================== Python Type BSON Type Supported Direction ======================================= ============= =================== None null both bool boolean both int [#int]_ int32 / int64 py -> bson long int64 py -> bson `bson.int64.Int64` int64 both float number (real) both string string py -> bson unicode string both list array both dict / `SON` object both datetime.datetime [#dt]_ [#dt2]_ date both `bson.regex.Regex` regex both compiled re [#re]_ regex py -> bson `bson.binary.Binary` binary both `bson.objectid.ObjectId` oid both `bson.dbref.DBRef` dbref both None undefined bson -> py unicode code bson -> py `bson.code.Code` code py -> bson unicode symbol bson -> py bytes (Python 3) [#bytes]_ binary both ======================================= ============= =================== Note that to save binary data it must be wrapped as an instance of `bson.binary.Binary`. Otherwise it will be saved as a BSON string and retrieved as unicode. .. [#int] A Python int will be saved as a BSON int32 or BSON int64 depending on its size. A BSON int32 will always decode to a Python int. A BSON int64 will always decode to a :class:`~bson.int64.Int64`. .. [#dt] datetime.datetime instances will be rounded to the nearest millisecond when saved .. [#dt2] all datetime.datetime instances are treated as *naive*. clients should always use UTC. .. [#re] :class:`~bson.regex.Regex` instances and regular expression objects from ``re.compile()`` are both saved as BSON regular expressions. BSON regular expressions are decoded as :class:`~bson.regex.Regex` instances. .. [#bytes] The bytes type from Python 3.x is encoded as BSON binary with subtype 0. In Python 3.x it will be decoded back to bytes. In Python 2.x it will be decoded to an instance of :class:`~bson.binary.Binary` with subtype 0. """ def __init__(self, data=None, **kwargs): self.__keys = [] dict.__init__(self) self.update(data) self.update(kwargs) def __new__(cls, *args, **kwargs): instance = super(SON, cls).__new__(cls, *args, **kwargs) instance.__keys = [] return instance def __repr__(self): result = [] for key in self.__keys: result.append("(%r, %r)" % (key, self[key])) return "SON([%s])" % ", ".join(result) def __setitem__(self, key, value): if key not in self.__keys: self.__keys.append(key) dict.__setitem__(self, key, value) def __delitem__(self, key): self.__keys.remove(key) dict.__delitem__(self, key) def keys(self): return list(self.__keys) def copy(self): other = SON() other.update(self) return other # TODO this is all from UserDict.DictMixin. it could probably be made more # efficient. # second level definitions support higher levels def __iter__(self): for k in self.__keys: yield k def has_key(self, key): return key in self.__keys # third level takes advantage of second level definitions def iteritems(self): for k in self: yield (k, self[k]) def iterkeys(self): return self.__iter__() # fourth level uses definitions from lower levels def itervalues(self): for _, v in self.iteritems(): yield v def values(self): return [v for _, v in self.iteritems()] def items(self): return [(key, self[key]) for key in self] def clear(self): self.__keys = [] super(SON, self).clear() def setdefault(self, key, default=None): try: return self[key] except KeyError: self[key] = default return default def pop(self, key, *args): if len(args) > 1: raise TypeError("pop expected at most 2 arguments, got "\ + repr(1 + len(args))) try: value = self[key] except KeyError: if args: return args[0] raise del self[key] return value def popitem(self): try: k, v = next(self.iteritems()) except StopIteration: raise KeyError('container is empty') del self[k] return (k, v) def update(self, other=None, **kwargs): # Make progressively weaker assumptions about "other" if other is None: pass elif hasattr(other, 'iteritems'): # iteritems saves memory and lookups for k, v in other.iteritems(): self[k] = v elif hasattr(other, 'keys'): for k in other.keys(): self[k] = other[k] else: for k, v in other: self[k] = v if kwargs: self.update(kwargs) def get(self, key, default=None): try: return self[key] except KeyError: return default def __eq__(self, other): """Comparison to another SON is order-sensitive while comparison to a regular dictionary is order-insensitive. """ if isinstance(other, SON): return len(self) == len(other) and self.items() == other.items() return self.to_dict() == other def __ne__(self, other): return not self == other def __len__(self): return len(self.__keys) def to_dict(self): """Convert a SON document to a normal Python dictionary instance. This is trickier than just *dict(...)* because it needs to be recursive. """ def transform_value(value): if isinstance(value, list): return [transform_value(v) for v in value] elif isinstance(value, collections.Mapping): return dict([ (k, transform_value(v)) for k, v in iteritems(value)]) else: return value return transform_value(dict(self)) def __deepcopy__(self, memo): out = SON() val_id = id(self) if val_id in memo: return memo.get(val_id) memo[val_id] = out for k, v in self.iteritems(): if not isinstance(v, RE_TYPE): v = copy.deepcopy(v, memo) out[k] = v return out mockupdb-1.3.0/mockupdb/_bson/timestamp.py0000644000076700000240000000756012574371305022323 0ustar emptysquarestaff00000000000000# Copyright 2010-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Tools for representing MongoDB internal Timestamps. """ import calendar import datetime from mockupdb._bson.py3compat import integer_types from mockupdb._bson.tz_util import utc UPPERBOUND = 4294967296 class Timestamp(object): """MongoDB internal timestamps used in the opLog. """ _type_marker = 17 def __init__(self, time, inc): """Create a new :class:`Timestamp`. This class is only for use with the MongoDB opLog. If you need to store a regular timestamp, please use a :class:`~datetime.datetime`. Raises :class:`TypeError` if `time` is not an instance of :class: `int` or :class:`~datetime.datetime`, or `inc` is not an instance of :class:`int`. Raises :class:`ValueError` if `time` or `inc` is not in [0, 2**32). :Parameters: - `time`: time in seconds since epoch UTC, or a naive UTC :class:`~datetime.datetime`, or an aware :class:`~datetime.datetime` - `inc`: the incrementing counter """ if isinstance(time, datetime.datetime): if time.utcoffset() is not None: time = time - time.utcoffset() time = int(calendar.timegm(time.timetuple())) if not isinstance(time, integer_types): raise TypeError("time must be an instance of int") if not isinstance(inc, integer_types): raise TypeError("inc must be an instance of int") if not 0 <= time < UPPERBOUND: raise ValueError("time must be contained in [0, 2**32)") if not 0 <= inc < UPPERBOUND: raise ValueError("inc must be contained in [0, 2**32)") self.__time = time self.__inc = inc @property def time(self): """Get the time portion of this :class:`Timestamp`. """ return self.__time @property def inc(self): """Get the inc portion of this :class:`Timestamp`. """ return self.__inc def __eq__(self, other): if isinstance(other, Timestamp): return (self.__time == other.time and self.__inc == other.inc) else: return NotImplemented def __hash__(self): return hash(self.time) ^ hash(self.inc) def __ne__(self, other): return not self == other def __lt__(self, other): if isinstance(other, Timestamp): return (self.time, self.inc) < (other.time, other.inc) return NotImplemented def __le__(self, other): if isinstance(other, Timestamp): return (self.time, self.inc) <= (other.time, other.inc) return NotImplemented def __gt__(self, other): if isinstance(other, Timestamp): return (self.time, self.inc) > (other.time, other.inc) return NotImplemented def __ge__(self, other): if isinstance(other, Timestamp): return (self.time, self.inc) >= (other.time, other.inc) return NotImplemented def __repr__(self): return "Timestamp(%s, %s)" % (self.__time, self.__inc) def as_datetime(self): """Return a :class:`~datetime.datetime` instance corresponding to the time portion of this :class:`Timestamp`. The returned datetime's timezone is UTC. """ return datetime.datetime.fromtimestamp(self.__time, utc) mockupdb-1.3.0/mockupdb/_bson/tz_util.py0000644000076700000240000000275612547632703022016 0ustar emptysquarestaff00000000000000# Copyright 2010-2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Timezone related utilities for BSON.""" from datetime import (timedelta, tzinfo) ZERO = timedelta(0) class FixedOffset(tzinfo): """Fixed offset timezone, in minutes east from UTC. Implementation based from the Python `standard library documentation `_. Defining __getinitargs__ enables pickling / copying. """ def __init__(self, offset, name): if isinstance(offset, timedelta): self.__offset = offset else: self.__offset = timedelta(minutes=offset) self.__name = name def __getinitargs__(self): return self.__offset, self.__name def utcoffset(self, dt): return self.__offset def tzname(self, dt): return self.__name def dst(self, dt): return ZERO utc = FixedOffset(0, "UTC") """Fixed offset timezone representing UTC.""" mockupdb-1.3.0/mockupdb/server.pem0000644000076700000240000000371213241707757020660 0ustar emptysquarestaff00000000000000-----BEGIN PRIVATE KEY----- MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAK53miP9GczBWXnq NxHwQkgVqsDuesjwJbWilMK4gf3fjnf2PN3qDpnGbZbPD0ij8975pIKtSPoDycFm A8Mogip0yU2Lv2lL56CWthSBftOFDL2CWIsmuuURFXZPiVLtLytfI9oLASZFlywW Cs83qEDTvdW8VoVhVsxV1JFDnpXLAgMBAAECgYBoGBgxrMt97UazhNkCrPT/CV5t 6lv8E7yMGMrlOyzkCkR4ssQyK3o2qbutJTGbR6czvIM5LKbD9Qqlh3ZrNHokWmTR VQQpJxt8HwP5boQvwRHg9+KSGr4JvRko1qxFs9C7Bzjt4r9VxdjhwZPdy0McGI/z yPXyQHjqBayrHV1EwQJBANorfCKeIxLhH3LAeUZuRS8ACldJ2N1kL6Ov43/v+0S/ OprQeBTODuTds3sv7FCT1aYDTOe6JLNOwN2i4YVOMBsCQQDMuCozrwqftD17D06P 9+lRXUekY5kFBs5j28Xnl8t8jnuxsXtQUTru660LD0QrmDNSauhpEmlpJknicnGt hmwRAkEA12MI6bBPlir0/jgxQqxI1w7mJqj8Vg27zpEuO7dzzLoyJHddpcSNBbwu npaAakiZK42klj26T9+XHvjYRuAbMwJBAJ5WnwWEkGH/pUHGEAyYQdSVojDKe/MA Vae0tzguFswK5C8GyArSGRPsItYYA7D4MlG/sGx8Oh2C6MiFndkJzBECQDcP1y4r Qsek151t1zArLKH4gG5dQAeZ0Lc2VeC4nLMUqVwrHcZDdd1RzLlSaH3j1MekFVfT 6v6rrcNLEVbeuk4= -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- MIIC7jCCAlegAwIBAgIBCjANBgkqhkiG9w0BAQUFADCBkjELMAkGA1UEBhMCVVMx ETAPBgNVBAgMCE5ldyBZb3JrMRYwFAYDVQQHDA1OZXcgWW9yayBDaXR5MQ4wDAYD VQQKDAUxMEdlbjEPMA0GA1UECwwGS2VybmVsMRowGAYDVQQDDBFNeSBDZXJ0IEF1 dGhvcml0eTEbMBkGCSqGSIb3DQEJARYMcm9vdEBsYXphcnVzMB4XDTEzMTIwNTEz MjU0MFoXDTQxMDQyMTEzMjU0MFowajELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE5l dyBZb3JrMRYwFAYDVQQHDA1OZXcgWW9yayBDaXR5MQ4wDAYDVQQKDAUxMEdlbjEP MA0GA1UECwwGS2VybmVsMQ8wDQYDVQQDDAZzZXJ2ZXIwgZ8wDQYJKoZIhvcNAQEB BQADgY0AMIGJAoGBAK53miP9GczBWXnqNxHwQkgVqsDuesjwJbWilMK4gf3fjnf2 PN3qDpnGbZbPD0ij8975pIKtSPoDycFmA8Mogip0yU2Lv2lL56CWthSBftOFDL2C WIsmuuURFXZPiVLtLytfI9oLASZFlywWCs83qEDTvdW8VoVhVsxV1JFDnpXLAgMB AAGjezB5MAkGA1UdEwQCMAAwLAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJh dGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBQgCkKiZhUV9/Zo7RwYYwm2cNK6tzAf BgNVHSMEGDAWgBQHQRk6n37FtyJOt7zV3+T8CbhkFjANBgkqhkiG9w0BAQUFAAOB gQCbsfr+Q4pty4Fy38lSxoCgnbB4pX6+Ex3xyw5zxDYR3xUlb/uHBiNZ1dBrXBxU ekU8dEvf+hx4iRDSW/C5N6BGnBBhCHcrPabo2bEEWKVsbUC3xchTB5rNGkvnMt9t G9ol7vanuzjL3S8/2PB33OshkBH570CxqqPflQbdjwt9dg== -----END CERTIFICATE----- mockupdb-1.3.0/mockupdb.egg-info/0000755000076700000240000000000013242634066020327 5ustar emptysquarestaff00000000000000mockupdb-1.3.0/mockupdb.egg-info/dependency_links.txt0000664000076700000240000000000113242634066024377 0ustar emptysquarestaff00000000000000 mockupdb-1.3.0/mockupdb.egg-info/not-zip-safe0000664000076700000240000000000112757075745022574 0ustar emptysquarestaff00000000000000 mockupdb-1.3.0/mockupdb.egg-info/PKG-INFO0000664000076700000240000000661313242634066021434 0ustar emptysquarestaff00000000000000Metadata-Version: 1.1 Name: mockupdb Version: 1.3.0 Summary: MongoDB Wire Protocol server library Home-page: https://github.com/ajdavis/mongo-mockup-db Author: A. Jesse Jiryu Davis Author-email: jesse@mongodb.com License: Apache License, Version 2.0 Description-Content-Type: UNKNOWN Description: ======== MockupDB ======== Mock server for testing MongoDB clients and creating MongoDB Wire Protocol servers. * Documentation: http://mockupdb.readthedocs.org/ Changelog ========= 1.3.0 (2018-02-19) ------------------ Support Windows. Log a traceback if a bad client request causes an assert. Fix SSL. Make errors less likely on shutdown. Enable testing on Travis and Appveyor. Fix doctests and interactive server for modern MongoDB protocol. 1.2.1 (2017-12-06) ------------------ Set minWireVersion to 0, not to 2. I had been wrong about MongoDB 3.6's wire version range: it's actually 0 to 6. MockupDB now reports the same wire version range as MongoDB 3.6 by default. 1.2.0 (2017-09-22) ------------------ Update for MongoDB 3.6: report minWireVersion 2 and maxWireVersion 6 by default. 1.1.3 (2017-04-23) ------------------ Avoid rare RuntimeError in close(), if a client thread shuts down a socket as MockupDB iterates its list of sockets. 1.1.2 (2016-08-23) ------------------ Properly detect closed sockets so ``MockupDB.stop()`` doesn't take 10 seconds per connection. Thanks to Sean Purcell. 1.1.1 (2016-08-01) ------------------ Don't use "client" as a keyword arg for ``Request``, it conflicts with the actual "client" field in drivers' new handshake protocol. 1.1.0 (2016-02-11) ------------------ Add cursor_id property to OpGetMore, and ssl parameter to interactive_server. 1.0.3 (2015-09-12) ------------------ ``MockupDB(auto_ismaster=True)`` had just responded ``{"ok": 1}``, but this isn't enough to convince PyMongo 3 it's talking to a valid standalone, so auto-respond ``{"ok": 1, "ismaster": True}``. 1.0.2 (2015-09-11) ------------------ Restore Request.assert_matches method, used in pymongo-mockup-tests. 1.0.1 (2015-09-11) ------------------ Allow co-installation with PyMongo. 1.0.0 (2015-09-10) ------------------ First release. 0.1.0 (2015-02-25) ------------------ Development begun. Keywords: mongo,mongodb,wire protocol,mockupdb,mock Platform: UNKNOWN Classifier: Development Status :: 2 - Pre-Alpha Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: Apache Software License Classifier: Natural Language :: English Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 mockupdb-1.3.0/mockupdb.egg-info/SOURCES.txt0000664000076700000240000000165613242634066022225 0ustar emptysquarestaff00000000000000AUTHORS.rst CHANGELOG.rst CONTRIBUTING.rst LICENSE MANIFEST.in README.rst setup.cfg setup.py docs/Makefile docs/authors.rst docs/changelog.rst docs/conf.py docs/contributing.rst docs/index.rst docs/installation.rst docs/make.bat docs/reference.rst docs/tutorial.rst mockupdb/__init__.py mockupdb/__main__.py mockupdb/server.pem mockupdb.egg-info/PKG-INFO mockupdb.egg-info/SOURCES.txt mockupdb.egg-info/dependency_links.txt mockupdb.egg-info/not-zip-safe mockupdb.egg-info/top_level.txt mockupdb/_bson/__init__.py mockupdb/_bson/binary.py mockupdb/_bson/code.py mockupdb/_bson/codec_options.py mockupdb/_bson/dbref.py mockupdb/_bson/errors.py mockupdb/_bson/int64.py mockupdb/_bson/json_util.py mockupdb/_bson/max_key.py mockupdb/_bson/min_key.py mockupdb/_bson/objectid.py mockupdb/_bson/py3compat.py mockupdb/_bson/regex.py mockupdb/_bson/son.py mockupdb/_bson/timestamp.py mockupdb/_bson/tz_util.py tests/__init__.py tests/test_mockupdb.pymockupdb-1.3.0/mockupdb.egg-info/top_level.txt0000664000076700000240000000001113242634066023053 0ustar emptysquarestaff00000000000000mockupdb mockupdb-1.3.0/PKG-INFO0000644000076700000240000000661313242634066016134 0ustar emptysquarestaff00000000000000Metadata-Version: 1.1 Name: mockupdb Version: 1.3.0 Summary: MongoDB Wire Protocol server library Home-page: https://github.com/ajdavis/mongo-mockup-db Author: A. Jesse Jiryu Davis Author-email: jesse@mongodb.com License: Apache License, Version 2.0 Description-Content-Type: UNKNOWN Description: ======== MockupDB ======== Mock server for testing MongoDB clients and creating MongoDB Wire Protocol servers. * Documentation: http://mockupdb.readthedocs.org/ Changelog ========= 1.3.0 (2018-02-19) ------------------ Support Windows. Log a traceback if a bad client request causes an assert. Fix SSL. Make errors less likely on shutdown. Enable testing on Travis and Appveyor. Fix doctests and interactive server for modern MongoDB protocol. 1.2.1 (2017-12-06) ------------------ Set minWireVersion to 0, not to 2. I had been wrong about MongoDB 3.6's wire version range: it's actually 0 to 6. MockupDB now reports the same wire version range as MongoDB 3.6 by default. 1.2.0 (2017-09-22) ------------------ Update for MongoDB 3.6: report minWireVersion 2 and maxWireVersion 6 by default. 1.1.3 (2017-04-23) ------------------ Avoid rare RuntimeError in close(), if a client thread shuts down a socket as MockupDB iterates its list of sockets. 1.1.2 (2016-08-23) ------------------ Properly detect closed sockets so ``MockupDB.stop()`` doesn't take 10 seconds per connection. Thanks to Sean Purcell. 1.1.1 (2016-08-01) ------------------ Don't use "client" as a keyword arg for ``Request``, it conflicts with the actual "client" field in drivers' new handshake protocol. 1.1.0 (2016-02-11) ------------------ Add cursor_id property to OpGetMore, and ssl parameter to interactive_server. 1.0.3 (2015-09-12) ------------------ ``MockupDB(auto_ismaster=True)`` had just responded ``{"ok": 1}``, but this isn't enough to convince PyMongo 3 it's talking to a valid standalone, so auto-respond ``{"ok": 1, "ismaster": True}``. 1.0.2 (2015-09-11) ------------------ Restore Request.assert_matches method, used in pymongo-mockup-tests. 1.0.1 (2015-09-11) ------------------ Allow co-installation with PyMongo. 1.0.0 (2015-09-10) ------------------ First release. 0.1.0 (2015-02-25) ------------------ Development begun. Keywords: mongo,mongodb,wire protocol,mockupdb,mock Platform: UNKNOWN Classifier: Development Status :: 2 - Pre-Alpha Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: Apache Software License Classifier: Natural Language :: English Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 mockupdb-1.3.0/README.rst0000664000076700000240000000024312574441203016515 0ustar emptysquarestaff00000000000000======== MockupDB ======== Mock server for testing MongoDB clients and creating MongoDB Wire Protocol servers. * Documentation: http://mockupdb.readthedocs.org/ mockupdb-1.3.0/setup.cfg0000644000076700000240000000007513242634066016654 0ustar emptysquarestaff00000000000000[wheel] universal = 1 [egg_info] tag_build = tag_date = 0 mockupdb-1.3.0/setup.py0000755000076700000240000000314313242634017016543 0ustar emptysquarestaff00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- import sys try: from setuptools import setup except ImportError: from distutils.core import setup with open('README.rst') as readme_file: readme = readme_file.read() with open('CHANGELOG.rst') as changelog_file: changelog = changelog_file.read().replace('.. :changelog:', '') requirements = [] test_requirements = ['pymongo>=3'] if sys.version_info[:2] == (2, 6): requirements.append('ordereddict') test_requirements.append('unittest2') setup( name='mockupdb', version='1.3.0', description="MongoDB Wire Protocol server library", long_description=readme + '\n\n' + changelog, author="A. Jesse Jiryu Davis", author_email='jesse@mongodb.com', url='https://github.com/ajdavis/mongo-mockup-db', packages=['mockupdb'], package_dir={'mockupdb': 'mockupdb'}, include_package_data=True, install_requires=requirements, license="Apache License, Version 2.0", zip_safe=False, keywords=["mongo", "mongodb", "wire protocol", "mockupdb", "mock"], classifiers=[ 'Development Status :: 2 - Pre-Alpha', 'Intended Audience :: Developers', "License :: OSI Approved :: Apache Software License", 'Natural Language :: English', "Programming Language :: Python :: 2", 'Programming Language :: Python :: 2.6', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.3', 'Programming Language :: Python :: 3.4', ], test_suite='tests', tests_require=test_requirements ) mockupdb-1.3.0/tests/0000755000076700000240000000000013242634066016173 5ustar emptysquarestaff00000000000000mockupdb-1.3.0/tests/__init__.py0000755000076700000240000000015512475522243020310 0ustar emptysquarestaff00000000000000# -*- coding: utf-8 -*- import unittest try: import unittest2 as unittest except ImportError: pass mockupdb-1.3.0/tests/test_mockupdb.py0000755000076700000240000002377213242633741021425 0ustar emptysquarestaff00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- """Test MockupDB.""" import contextlib import os import ssl import sys if sys.version_info[0] < 3: from io import BytesIO as StringIO else: from io import StringIO try: from queue import Queue except ImportError: from Queue import Queue # Tests depend on PyMongo's BSON implementation, but MockupDB itself does not. from bson import SON from bson.codec_options import CodecOptions from pymongo import MongoClient, message, WriteConcern from mockupdb import (go, going, Command, Matcher, MockupDB, Request, OpDelete, OpInsert, OpQuery, OpUpdate, DELETE_FLAGS, INSERT_FLAGS, UPDATE_FLAGS, QUERY_FLAGS) from tests import unittest # unittest2 on Python 2.6. @contextlib.contextmanager def capture_stderr(): sio = StringIO() stderr, sys.stderr = sys.stderr, sio try: yield sio finally: sys.stderr = stderr sio.seek(0) class TestGoing(unittest.TestCase): def test_nested_errors(self): def thrower(): raise AssertionError("thrown") with capture_stderr() as stderr: with self.assertRaises(ZeroDivisionError): with going(thrower) as future: 1 / 0 self.assertIn('error in going(', stderr.getvalue()) self.assertIn('AssertionError: thrown', stderr.getvalue()) # Future keeps raising. self.assertRaises(AssertionError, future) self.assertRaises(AssertionError, future) class TestRequest(unittest.TestCase): def _pack_request(self, ns, slave_ok): flags = 4 if slave_ok else 0 request_id, msg_bytes, max_doc_size = message.query( flags, ns, 0, 0, {}, None, CodecOptions()) # Skip 16-byte standard header. return msg_bytes[16:], request_id def test_flags(self): request = Request() self.assertIsNone(request.flags) self.assertFalse(request.slave_ok) msg_bytes, request_id = self._pack_request('db.collection', False) request = OpQuery.unpack(msg_bytes, None, None, request_id) self.assertIsInstance(request, OpQuery) self.assertNotIsInstance(request, Command) self.assertEqual(0, request.flags) self.assertFalse(request.slave_ok) self.assertFalse(request.slave_okay) # Synonymous. msg_bytes, request_id = self._pack_request('db.$cmd', False) request = OpQuery.unpack(msg_bytes, None, None, request_id) self.assertIsInstance(request, Command) self.assertEqual(0, request.flags) msg_bytes, request_id = self._pack_request('db.collection', True) request = OpQuery.unpack(msg_bytes, None, None, request_id) self.assertEqual(4, request.flags) self.assertTrue(request.slave_ok) msg_bytes, request_id = self._pack_request('db.$cmd', True) request = OpQuery.unpack(msg_bytes, None, None, request_id) self.assertEqual(4, request.flags) def test_fields(self): self.assertIsNone(OpQuery({}).fields) self.assertEqual({'_id': False, 'a': 1}, OpQuery({}, fields={'_id': False, 'a': 1}).fields) def test_repr(self): self.assertEqual('Request()', repr(Request())) self.assertEqual('Request({})', repr(Request({}))) self.assertEqual('Request({})', repr(Request([{}]))) self.assertEqual('Request(flags=4)', repr(Request(flags=4))) self.assertEqual('OpQuery({})', repr(OpQuery())) self.assertEqual('OpQuery({})', repr(OpQuery({}))) self.assertEqual('OpQuery({})', repr(OpQuery([{}]))) self.assertEqual('OpQuery({}, flags=SlaveOkay)', repr(OpQuery(flags=4))) self.assertEqual('OpQuery({}, flags=SlaveOkay)', repr(OpQuery({}, flags=4))) self.assertEqual('OpQuery({}, flags=TailableCursor|AwaitData)', repr(OpQuery({}, flags=34))) self.assertEqual('Command({})', repr(Command())) self.assertEqual('Command({"foo": 1})', repr(Command('foo'))) son = SON([('b', 1), ('a', 1), ('c', 1)]) self.assertEqual('Command({"b": 1, "a": 1, "c": 1})', repr(Command(son))) self.assertEqual('Command({}, flags=SlaveOkay)', repr(Command(flags=4))) self.assertEqual('OpInsert({}, {})', repr(OpInsert([{}, {}]))) self.assertEqual('OpInsert({}, {})', repr(OpInsert({}, {}))) def test_assert_matches(self): request = OpQuery({'x': 17}, flags=QUERY_FLAGS['SlaveOkay']) request.assert_matches(request) with self.assertRaises(AssertionError): request.assert_matches(Command('foo')) class TestLegacyWrites(unittest.TestCase): def setUp(self): self.server = MockupDB(auto_ismaster=True) self.server.run() self.addCleanup(self.server.stop) self.client = MongoClient(self.server.uri) self.collection = self.client.db.get_collection( 'collection', write_concern=WriteConcern(w=0)) def test_insert_one(self): with going(self.collection.insert_one, {'_id': 1}): self.server.receives(OpInsert({'_id': 1}, flags=0)) def test_insert_many(self): collection = self.collection.with_options( write_concern=WriteConcern(0)) flags = INSERT_FLAGS['ContinueOnError'] docs = [{'_id': 1}, {'_id': 2}] with going(collection.insert_many, docs, ordered=False): self.server.receives(OpInsert(docs, flags=flags)) def test_replace_one(self): with going(self.collection.replace_one, {}, {}): self.server.receives(OpUpdate({}, {}, flags=0)) def test_update_many(self): flags = UPDATE_FLAGS['MultiUpdate'] with going(self.collection.update_many, {}, {'$unset': 'a'}): update = self.server.receives(OpUpdate({}, {}, flags=flags)) self.assertEqual(2, update.flags) def test_delete_one(self): flags = DELETE_FLAGS['SingleRemove'] with going(self.collection.delete_one, {}): delete = self.server.receives(OpDelete({}, flags=flags)) self.assertEqual(1, delete.flags) def test_delete_many(self): with going(self.collection.delete_many, {}): delete = self.server.receives(OpDelete({}, flags=0)) self.assertEqual(0, delete.flags) class TestMatcher(unittest.TestCase): def test_command_name_case_insensitive(self): self.assertTrue( Matcher(Command('ismaster')).matches(Command('IsMaster'))) def test_command_first_arg(self): self.assertFalse( Matcher(Command(ismaster=1)).matches(Command(ismaster=2))) def test_command_fields(self): self.assertTrue( Matcher(Command('a', b=1)).matches(Command('a', b=1))) self.assertFalse( Matcher(Command('a', b=1)).matches(Command('a', b=2))) class TestAutoresponds(unittest.TestCase): def test_auto_dequeue(self): server = MockupDB(auto_ismaster=True) server.run() client = MongoClient(server.uri) future = go(client.admin.command, 'ping') server.autoresponds('ping') # Should dequeue the request. future() def test_autoresponds_case_insensitive(self): server = MockupDB(auto_ismaster=True) # Little M. Note this is only case-insensitive because it's a Command. server.autoresponds(Command('fooBar'), foo='bar') server.run() response = MongoClient(server.uri).admin.command('Foobar') self.assertEqual('bar', response['foo']) class TestSSL(unittest.TestCase): def test_ssl_uri(self): server = MockupDB(ssl=True) server.run() self.addCleanup(server.stop) self.assertEqual( 'mongodb://localhost:%d/?ssl=true' % server.port, server.uri) def test_ssl_basic(self): server = MockupDB(ssl=True, auto_ismaster=True) server.run() self.addCleanup(server.stop) client = MongoClient(server.uri, ssl_cert_reqs=ssl.CERT_NONE) client.db.command('ismaster') class TestMockupDB(unittest.TestCase): def test_iteration(self): server = MockupDB(auto_ismaster={'maxWireVersion': 3}) server.run() self.addCleanup(server.stop) client = MongoClient(server.uri) def send_three_docs(): for i in range(3): client.test.test.insert({'_id': i}) with going(send_three_docs): j = 0 # The "for request in server" statement is the point of this test. for request in server: self.assertTrue(request.matches({'insert': 'test', 'documents': [{'_id': j}]})) request.ok() j += 1 if j == 3: break def test_default_wire_version(self): server = MockupDB(auto_ismaster=True) server.run() self.addCleanup(server.stop) ismaster = MongoClient(server.uri).admin.command('isMaster') self.assertEqual(ismaster['minWireVersion'], 0) self.assertEqual(ismaster['maxWireVersion'], 6) def test_wire_version(self): server = MockupDB(auto_ismaster=True, min_wire_version=1, max_wire_version=42) server.run() self.addCleanup(server.stop) ismaster = MongoClient(server.uri).admin.command('isMaster') self.assertEqual(ismaster['minWireVersion'], 1) self.assertEqual(ismaster['maxWireVersion'], 42) class TestResponse(unittest.TestCase): def test_ok(self): server = MockupDB(auto_ismaster={'maxWireVersion': 3}) server.run() self.addCleanup(server.stop) client = MongoClient(server.uri) with going(client.test.command, {'foo': 1}) as future: server.receives().ok(3) response = future() self.assertEqual(3, response['ok']) if __name__ == '__main__': unittest.main()