mockupdb-1.7.0/0000755000076700000240000000000013400765404015032 5ustar emptysquarestaff00000000000000mockupdb-1.7.0/PKG-INFO0000644000076700000240000001233213400765404016130 0ustar emptysquarestaff00000000000000Metadata-Version: 1.1 Name: mockupdb Version: 1.7.0 Summary: MongoDB Wire Protocol server library Home-page: https://github.com/ajdavis/mongo-mockup-db Author: A. Jesse Jiryu Davis Author-email: jesse@mongodb.com License: Apache License, Version 2.0 Description-Content-Type: UNKNOWN Description: ======== MockupDB ======== Mock server for testing MongoDB clients and creating MongoDB Wire Protocol servers. * Documentation: http://mockupdb.readthedocs.org/ Changelog ========= 1.7.0 (2018-12-02) ------------------ Improve datetime support in match expressions. Python datetimes have microsecond precision but BSON only has milliseconds, so expressions like this always failed:: server.receives(Command('foo', when=datetime(2018, 12, 1, 6, 6, 6, 12345))) Now, the matching logic has been rewritten to recurse through arrays and subdocuments, comparing them value by value. It compares datetime values with only millisecond precision. 1.6.0 (2018-11-16) ------------------ Remove vendored BSON library. Instead, require PyMongo and use its BSON library. This avoids surprising problems where a BSON type created with PyMongo does not appear equal to one created with MockupDB, and it avoids the occasional need to update the vendored code to support new BSON features. 1.5.0 (2018-11-02) ------------------ Support for Unix domain paths with ``uds_path`` parameter. The ``interactive_server()`` function now prepares the server to autorespond to the ``getFreeMonitoringStatus`` command from the mongo shell. 1.4.1 (2018-06-30) ------------------ Fix an inadvertent dependency on PyMongo, which broke the docs build. 1.4.0 (2018-06-29) ------------------ Support, and expect, OP_MSG requests from clients. Thanks to Shane Harvey for the contribution. Update vendored bson library from PyMongo. Support the Decimal128 BSON type. Fix Matcher so it equates BSON objects from PyMongo like ``ObjectId(...)`` with equivalent objects created from MockupDB's vendored bson library. 1.3.0 (2018-02-19) ------------------ Support Windows. Log a traceback if a bad client request causes an assert. Fix SSL. Make errors less likely on shutdown. Enable testing on Travis and Appveyor. Fix doctests and interactive server for modern MongoDB protocol. 1.2.1 (2017-12-06) ------------------ Set minWireVersion to 0, not to 2. I had been wrong about MongoDB 3.6's wire version range: it's actually 0 to 6. MockupDB now reports the same wire version range as MongoDB 3.6 by default. 1.2.0 (2017-09-22) ------------------ Update for MongoDB 3.6: report minWireVersion 2 and maxWireVersion 6 by default. 1.1.3 (2017-04-23) ------------------ Avoid rare RuntimeError in close(), if a client thread shuts down a socket as MockupDB iterates its list of sockets. 1.1.2 (2016-08-23) ------------------ Properly detect closed sockets so ``MockupDB.stop()`` doesn't take 10 seconds per connection. Thanks to Sean Purcell. 1.1.1 (2016-08-01) ------------------ Don't use "client" as a keyword arg for ``Request``, it conflicts with the actual "client" field in drivers' new handshake protocol. 1.1.0 (2016-02-11) ------------------ Add cursor_id property to OpGetMore, and ssl parameter to interactive_server. 1.0.3 (2015-09-12) ------------------ ``MockupDB(auto_ismaster=True)`` had just responded ``{"ok": 1}``, but this isn't enough to convince PyMongo 3 it's talking to a valid standalone, so auto-respond ``{"ok": 1, "ismaster": True}``. 1.0.2 (2015-09-11) ------------------ Restore Request.assert_matches method, used in pymongo-mockup-tests. 1.0.1 (2015-09-11) ------------------ Allow co-installation with PyMongo. 1.0.0 (2015-09-10) ------------------ First release. 0.1.0 (2015-02-25) ------------------ Development begun. Keywords: mongo,mongodb,wire protocol,mockupdb,mock Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: Apache Software License Classifier: Natural Language :: English Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 mockupdb-1.7.0/LICENSE0000644000076700000240000002613512474510641016047 0ustar emptysquarestaff00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. mockupdb-1.7.0/CONTRIBUTING.rst0000644000076700000240000000605312574434213017501 0ustar emptysquarestaff00000000000000============ Contributing ============ Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given. You can contribute in many ways: Types of Contributions ---------------------- Report Bugs ~~~~~~~~~~~ Report bugs at https://github.com/ajdavis/mongo-mockup-db/issues. If you are reporting a bug, please include: * Your operating system name and version. * Any details about your local setup that might be helpful in troubleshooting. * Detailed steps to reproduce the bug. Fix Bugs ~~~~~~~~ Look through the GitHub issues for bugs. Anything tagged with "bug" is open to whoever wants to implement it. Implement Features ~~~~~~~~~~~~~~~~~~ Look through the GitHub issues for features. Anything tagged with "feature" is open to whoever wants to implement it. Write Documentation ~~~~~~~~~~~~~~~~~~~ MockupDB could always use more documentation, whether as part of the official MockupDB docs, in docstrings, or even on the web in blog posts, articles, and such. Submit Feedback ~~~~~~~~~~~~~~~ The best way to send feedback is to file an issue at https://github.com/ajdavis/mongo-mockup-db/issues. If you are proposing a feature: * Explain in detail how it would work. * Keep the scope as narrow as possible, to make it easier to implement. * Remember that this is a volunteer-driven project, and that contributions are welcome :) Get Started! ------------ Ready to contribute? Here's how to set up MockupDB for local development. 1. Fork the `mongo-mockup-db` repo on GitHub. 2. Clone your fork locally:: $ git clone git@github.com:your_name_here/mongo-mockup-db.git 3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:: $ mkvirtualenv mongo-mockup-db $ cd mongo-mockup-db/ $ python setup.py develop 4. Create a branch for local development:: $ git checkout -b name-of-your-bugfix-or-feature Now you can make your changes locally. 5. When you're done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox:: $ flake8 mockupdb tests $ python setup.py test $ tox To get flake8 and tox, just pip install them into your virtualenv. 6. Commit your changes and push your branch to GitHub:: $ git add . $ git commit -m "Your detailed description of your changes." $ git push origin name-of-your-bugfix-or-feature 7. Submit a pull request through the GitHub website. Pull Request Guidelines ----------------------- Before you submit a pull request, check that it meets these guidelines: 1. The pull request should include tests. 2. If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring, and add the feature to the list in README.rst. 3. The pull request should work for Python 2.6, 2.7, 3.3, and 3.4. Check that tests pass in all versions with `tox`. Tips ---- To run a subset of tests:: $ python setup.py test -s tests.test_mockupdb mockupdb-1.7.0/tests/0000755000076700000240000000000013400765404016174 5ustar emptysquarestaff00000000000000mockupdb-1.7.0/tests/__init__.py0000755000076700000240000000015512475522243020314 0ustar emptysquarestaff00000000000000# -*- coding: utf-8 -*- import unittest try: import unittest2 as unittest except ImportError: pass mockupdb-1.7.0/tests/test_mockupdb.py0000755000076700000240000003215313400765373021425 0ustar emptysquarestaff00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- """Test MockupDB.""" import contextlib import datetime import os import ssl import sys import tempfile if sys.version_info[0] < 3: from io import BytesIO as StringIO else: from io import StringIO try: from queue import Queue except ImportError: from Queue import Queue from bson import (Binary, Code, DBRef, Decimal128, MaxKey, MinKey, ObjectId, Regex, SON, Timestamp) from bson.codec_options import CodecOptions from pymongo import MongoClient, message, WriteConcern from mockupdb import (go, going, Command, CommandBase, Matcher, MockupDB, Request, OpInsert, OpMsg, OpQuery, QUERY_FLAGS) from tests import unittest # unittest2 on Python 2.6. @contextlib.contextmanager def capture_stderr(): sio = StringIO() stderr, sys.stderr = sys.stderr, sio try: yield sio finally: sys.stderr = stderr sio.seek(0) class TestGoing(unittest.TestCase): def test_nested_errors(self): def thrower(): raise AssertionError("thrown") with capture_stderr() as stderr: with self.assertRaises(ZeroDivisionError): with going(thrower) as future: 1 / 0 self.assertIn('error in going(', stderr.getvalue()) self.assertIn('AssertionError: thrown', stderr.getvalue()) # Future keeps raising. self.assertRaises(AssertionError, future) self.assertRaises(AssertionError, future) class TestRequest(unittest.TestCase): def _pack_request(self, ns, slave_ok): flags = 4 if slave_ok else 0 request_id, msg_bytes, max_doc_size = message.query( flags, ns, 0, 0, {}, None, CodecOptions()) # Skip 16-byte standard header. return msg_bytes[16:], request_id def test_flags(self): request = Request() self.assertIsNone(request.flags) self.assertFalse(request.slave_ok) msg_bytes, request_id = self._pack_request('db.collection', False) request = OpQuery.unpack(msg_bytes, None, None, request_id) self.assertIsInstance(request, OpQuery) self.assertNotIsInstance(request, Command) self.assertEqual(0, request.flags) self.assertFalse(request.slave_ok) self.assertFalse(request.slave_okay) # Synonymous. msg_bytes, request_id = self._pack_request('db.$cmd', False) request = OpQuery.unpack(msg_bytes, None, None, request_id) self.assertIsInstance(request, Command) self.assertEqual(0, request.flags) msg_bytes, request_id = self._pack_request('db.collection', True) request = OpQuery.unpack(msg_bytes, None, None, request_id) self.assertEqual(4, request.flags) self.assertTrue(request.slave_ok) msg_bytes, request_id = self._pack_request('db.$cmd', True) request = OpQuery.unpack(msg_bytes, None, None, request_id) self.assertEqual(4, request.flags) def test_fields(self): self.assertIsNone(OpQuery({}).fields) self.assertEqual({'_id': False, 'a': 1}, OpQuery({}, fields={'_id': False, 'a': 1}).fields) def test_repr(self): self.assertEqual('Request()', repr(Request())) self.assertEqual('Request({})', repr(Request({}))) self.assertEqual('Request({})', repr(Request([{}]))) self.assertEqual('Request(flags=4)', repr(Request(flags=4))) self.assertEqual('OpQuery({})', repr(OpQuery())) self.assertEqual('OpQuery({})', repr(OpQuery({}))) self.assertEqual('OpQuery({})', repr(OpQuery([{}]))) self.assertEqual('OpQuery({}, flags=SlaveOkay)', repr(OpQuery(flags=4))) self.assertEqual('OpQuery({}, flags=SlaveOkay)', repr(OpQuery({}, flags=4))) self.assertEqual('OpQuery({}, flags=TailableCursor|AwaitData)', repr(OpQuery({}, flags=34))) self.assertEqual('Command({})', repr(Command())) self.assertEqual('Command({"foo": 1})', repr(Command('foo'))) son = SON([('b', 1), ('a', 1), ('c', 1)]) self.assertEqual('Command({"b": 1, "a": 1, "c": 1})', repr(Command(son))) self.assertEqual('Command({}, flags=SlaveOkay)', repr(Command(flags=4))) self.assertEqual('OpInsert({}, {})', repr(OpInsert([{}, {}]))) self.assertEqual('OpInsert({}, {})', repr(OpInsert({}, {}))) def test_assert_matches(self): request = OpQuery({'x': 17}, flags=QUERY_FLAGS['SlaveOkay']) request.assert_matches(request) with self.assertRaises(AssertionError): request.assert_matches(Command('foo')) class TestUnacknowledgedWrites(unittest.TestCase): def setUp(self): self.server = MockupDB(auto_ismaster=True) self.server.run() self.addCleanup(self.server.stop) self.client = MongoClient(self.server.uri) self.collection = self.client.db.get_collection( 'collection', write_concern=WriteConcern(w=0)) def test_insert_one(self): with going(self.collection.insert_one, {'_id': 1}): # The moreToCome flag = 2. self.server.receives( OpMsg('insert', 'collection', writeConcern={'w': 0}, flags=2)) def test_insert_many(self): collection = self.collection.with_options( write_concern=WriteConcern(0)) docs = [{'_id': 1}, {'_id': 2}] with going(collection.insert_many, docs, ordered=False): self.server.receives(OpMsg(SON([ ('insert', 'collection'), ('ordered', False), ('writeConcern', {'w': 0})]), flags=2)) def test_replace_one(self): with going(self.collection.replace_one, {}, {}): self.server.receives(OpMsg(SON([ ('update', 'collection'), ('writeConcern', {'w': 0}) ]), flags=2)) def test_update_many(self): with going(self.collection.update_many, {}, {'$unset': 'a'}): self.server.receives(OpMsg(SON([ ('update', 'collection'), ('ordered', True), ('writeConcern', {'w': 0}) ]), flags=2)) def test_delete_one(self): with going(self.collection.delete_one, {}): self.server.receives(OpMsg(SON([ ('delete', 'collection'), ('writeConcern', {'w': 0}) ]), flags=2)) def test_delete_many(self): with going(self.collection.delete_many, {}): self.server.receives(OpMsg(SON([ ('delete', 'collection'), ('writeConcern', {'w': 0})]), flags=2)) class TestMatcher(unittest.TestCase): def test_command_name_case_insensitive(self): self.assertTrue( Matcher(Command('ismaster')).matches(Command('IsMaster'))) def test_command_first_arg(self): self.assertFalse( Matcher(Command(ismaster=1)).matches(Command(ismaster=2))) def test_command_fields(self): self.assertTrue( Matcher(Command('a', b=1)).matches(Command('a', b=1))) self.assertFalse( Matcher(Command('a', b=1)).matches(Command('a', b=2))) def test_bson_classes(self): _id = '5a918f9fa08bff9c7688d3e1' for a, b in [ (Binary(b'foo'), Binary(b'foo')), (Code('foo'), Code('foo')), (Code('foo', {'x': 1}), Code('foo', {'x': 1})), (DBRef('coll', 1), DBRef('coll', 1)), (DBRef('coll', 1, 'db'), DBRef('coll', 1, 'db')), (Decimal128('1'), Decimal128('1')), (MaxKey(), MaxKey()), (MinKey(), MinKey()), (ObjectId(_id), ObjectId(_id)), (Regex('foo', 'i'), Regex('foo', 'i')), (Timestamp(1, 2), Timestamp(1, 2)), ]: # Basic case. self.assertTrue( Matcher(Command(y=b)).matches(Command(y=b)), "MockupDB %r doesn't equal itself" % (b,)) # First Command argument is special, try comparing the second also. self.assertTrue( Matcher(Command('x', y=b)).matches(Command('x', y=b)), "MockupDB %r doesn't equal itself" % (b,)) # In practice, users pass PyMongo classes in message specs. self.assertTrue( Matcher(Command(y=b)).matches(Command(y=a)), "PyMongo %r != MockupDB %r" % (a, b)) self.assertTrue( Matcher(Command('x', y=b)).matches(Command('x', y=a)), "PyMongo %r != MockupDB %r" % (a, b)) def test_datetime(self): server = MockupDB(auto_ismaster=True) server.run() client = MongoClient(server.uri) # Python datetimes have microsecond precision, BSON only millisecond. # Ensure this datetime matches itself despite the truncation. dt = datetime.datetime(2018, 12, 1, 6, 6, 6, 12345) doc = SON([('_id', 1), ('dt', dt)]) with going(client.db.collection.insert_one, doc): server.receives( OpMsg('insert', 'collection', documents=[doc])).ok() class TestAutoresponds(unittest.TestCase): def test_auto_dequeue(self): server = MockupDB(auto_ismaster=True) server.run() client = MongoClient(server.uri) future = go(client.admin.command, 'ping') server.autoresponds('ping') # Should dequeue the request. future() def test_autoresponds_case_insensitive(self): server = MockupDB(auto_ismaster=True) # Little M. Note this is only case-insensitive because it's a Command. server.autoresponds(CommandBase('fooBar'), foo='bar') server.run() response = MongoClient(server.uri).admin.command('Foobar') self.assertEqual('bar', response['foo']) class TestSSL(unittest.TestCase): def test_ssl_uri(self): server = MockupDB(ssl=True) server.run() self.addCleanup(server.stop) self.assertEqual( 'mongodb://localhost:%d/?ssl=true' % server.port, server.uri) def test_ssl_basic(self): server = MockupDB(ssl=True, auto_ismaster=True) server.run() self.addCleanup(server.stop) client = MongoClient(server.uri, ssl_cert_reqs=ssl.CERT_NONE) client.db.command('ismaster') class TestMockupDB(unittest.TestCase): def test_iteration(self): server = MockupDB(auto_ismaster={'maxWireVersion': 3}) server.run() self.addCleanup(server.stop) client = MongoClient(server.uri) def send_three_docs(): for i in range(3): client.test.test.insert({'_id': i}) with going(send_three_docs): j = 0 # The "for request in server" statement is the point of this test. for request in server: self.assertTrue(request.matches({'insert': 'test', 'documents': [{'_id': j}]})) request.ok() j += 1 if j == 3: break def test_default_wire_version(self): server = MockupDB(auto_ismaster=True) server.run() self.addCleanup(server.stop) ismaster = MongoClient(server.uri).admin.command('isMaster') self.assertEqual(ismaster['minWireVersion'], 0) self.assertEqual(ismaster['maxWireVersion'], 6) def test_wire_version(self): server = MockupDB(auto_ismaster=True, min_wire_version=1, max_wire_version=42) server.run() self.addCleanup(server.stop) ismaster = MongoClient(server.uri).admin.command('isMaster') self.assertEqual(ismaster['minWireVersion'], 1) self.assertEqual(ismaster['maxWireVersion'], 42) @unittest.skipIf(sys.platform == 'win32', 'Windows') def test_unix_domain_socket(self): tmp = tempfile.NamedTemporaryFile(delete=False, suffix='.sock') tmp.close() server = MockupDB(auto_ismaster={'maxWireVersion': 3}, uds_path=tmp.name) server.run() self.assertTrue(server.uri.endswith('.sock'), 'Expected URI "%s" to end with ".sock"' % (server.uri,)) self.assertEqual(server.host, tmp.name) self.assertEqual(server.port, 0) self.assertEqual(server.address, (tmp.name, 0)) self.assertEqual(server.address_string, tmp.name) client = MongoClient(server.uri) with going(client.test.command, {'foo': 1}) as future: server.receives().ok() response = future() self.assertEqual(1, response['ok']) server.stop() self.assertFalse(os.path.exists(tmp.name)) class TestResponse(unittest.TestCase): def test_ok(self): server = MockupDB(auto_ismaster={'maxWireVersion': 3}) server.run() self.addCleanup(server.stop) client = MongoClient(server.uri) with going(client.test.command, {'foo': 1}) as future: server.receives().ok(3) response = future() self.assertEqual(3, response['ok']) if __name__ == '__main__': unittest.main() mockupdb-1.7.0/MANIFEST.in0000644000076700000240000000042312574437536016604 0ustar emptysquarestaff00000000000000include AUTHORS.rst include CONTRIBUTING.rst include CHANGELOG.rst include LICENSE include README.rst recursive-include mockupdb * recursive-include tests * recursive-exclude * __pycache__ recursive-exclude * *.py[co] recursive-include docs *.rst conf.py Makefile make.bat mockupdb-1.7.0/docs/0000755000076700000240000000000013400765404015762 5ustar emptysquarestaff00000000000000mockupdb-1.7.0/docs/index.rst0000644000076700000240000000144213400765373017631 0ustar emptysquarestaff00000000000000.. mongo-mockup-db documentation master file, created by sphinx-quickstart on Tue Jul 9 22:26:36 2013. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. MockupDB ======== .. image:: _static/mask.jpg Mock server for testing MongoDB clients and creating `MongoDB Wire Protocol`_ servers. The :doc:`tutorial` is the primary documentation. Contents: .. toctree:: :maxdepth: 1 installation tutorial reference contributing authors changelog Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` Image Credit: `gnuckx `_ .. _MongoDB Wire Protocol: https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/ mockupdb-1.7.0/docs/contributing.rst0000644000076700000240000000004112474506007021220 0ustar emptysquarestaff00000000000000.. include:: ../CONTRIBUTING.rst mockupdb-1.7.0/docs/reference.rst0000644000076700000240000000010312475516614020454 0ustar emptysquarestaff00000000000000API Reference ============= .. automodule:: mockupdb :members: mockupdb-1.7.0/docs/Makefile0000644000076700000240000001521612474506007017431 0ustar emptysquarestaff00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # User-friendly check for sphinx-build ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) endif # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " xml to make Docutils-native XML files" @echo " pseudoxml to make pseudoxml-XML files for display purposes" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/mongo-mockup-db.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/mongo-mockup-db.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/mongo-mockup-db" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/mongo-mockup-db" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." latexpdfja: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through platex and dvipdfmx..." $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." xml: $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml @echo @echo "Build finished. The XML files are in $(BUILDDIR)/xml." pseudoxml: $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml @echo @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." mockupdb-1.7.0/docs/conf.py0000755000076700000240000002142213242633741017267 0ustar emptysquarestaff00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # # mongo-mockup-db documentation build configuration file, created by # sphinx-quickstart on Tue Jul 9 22:26:36 2013. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys import os # If extensions (or modules to document with autodoc) are in another # directory, add these directories to sys.path here. If the directory is # relative to the documentation root, use os.path.abspath to make it # absolute, like shown here. #sys.path.insert(0, os.path.abspath('.')) # Get the project root dir, which is the parent dir of this cwd = os.getcwd() project_root = os.path.dirname(cwd) # Insert the project root dir as the first element in the PYTHONPATH. # This lets us ensure that the source package is imported, and that its # version is used. sys.path.insert(0, project_root) import mockupdb # -- General configuration --------------------------------------------- # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.coverage', 'sphinx.ext.todo', 'sphinx.ext.intersphinx', ] intersphinx_mapping = { 'python': ('https://docs.python.org/3/', None), 'pymongo': ('http://api.mongodb.com/python/current/', None), } primary_domain = 'py' default_role = 'py:obj' doctest_global_setup = """ try: from collections import OrderedDict except: from ordereddict import OrderedDict # Python 2.6, "pip install ordereddict" from mockupdb import * """ # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = 'MockupDB' copyright = '2015, MongoDB, Inc.' # The version info for the project you're documenting, acts as replacement # for |version| and |release|, also used in various other places throughout # the built documents. # # The short X.Y version. version = mockupdb.__version__ # The full version, including alpha/beta/rc tags. release = mockupdb.__version__ # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to # some non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build'] # The reST default role (used for this markup: `text`) to use for all # documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built # documents. #keep_warnings = False # -- Options for HTML output ------------------------------------------- # Theme gratefully vendored from CPython source. html_theme = "pydoctheme" html_theme_path = ["."] html_theme_options = {'collapsiblesidebar': True} # Theme options are theme-specific and customize the look and feel of a # theme further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as # html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the # top of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon # of the docs. This file should be a Windows icon file (.ico) being # 16x16 or 32x32 pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) # here, relative to this directory. They are copied after the builtin # static files, so a file named "default.css" will overwrite the builtin # "default.css". html_static_path = ['_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page # bottom, using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names # to template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. # Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. # Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages # will contain a tag referring to it. The value of this option # must be the base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'mockupdbdoc' # -- Options for LaTeX output ------------------------------------------ latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass # [howto/manual]). latex_documents = [ ('index', 'mockupdb.tex', 'MockupDB Documentation', 'A. Jesse Jiryu Davis', 'manual'), ] # The name of an image file (relative to this directory) to place at # the top of the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings # are parts, not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output ------------------------------------ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'mockupdb', 'MockupDB Documentation', ['A. Jesse Jiryu Davis'], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ---------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ('index', 'mockupdb', 'MockupDB Documentation', 'A. Jesse Jiryu Davis', 'mockupdb', ('Mock server for testing MongoDB clients and creating MongoDB Wire Protocol' ' servers.'), 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' # If true, do not generate a @detailmenu in the "Top" node's menu. #texinfo_no_detailmenu = False mockupdb-1.7.0/docs/tutorial.rst0000644000076700000240000003712713400765373020376 0ustar emptysquarestaff00000000000000======== Tutorial ======== .. currentmodule:: mockupdb This tutorial is the primary documentation for the MockupDB project. I assume some familiarity with PyMongo_ and the `MongoDB Wire Protocol`_. .. contents:: Introduction ------------ Begin by running a :class:`.MockupDB` and connecting to it with PyMongo's `~pymongo.mongo_client.MongoClient`: >>> from mockupdb import * >>> server = MockupDB() >>> port = server.run() # Returns the TCP port number it listens on. >>> from pymongo import MongoClient >>> client = MongoClient(server.uri) When the client connects it calls the "ismaster" command, then blocks until the server responds. MockupDB receives the "ismaster" command but does not respond until you tell it to: >>> request = server.receives() >>> request.command_name 'ismaster' We respond: >>> request.replies({'ok': 1, 'maxWireVersion': 6}) True The `~MockupDB.receives` call blocks until it receives a request from the client. Responding to each "ismaster" call is tiresome, so tell the client to send the default response to all ismaster calls: >>> responder = server.autoresponds('ismaster', maxWireVersion=6) >>> client.admin.command('ismaster') == {'ok': 1, 'maxWireVersion': 6} True A call to `~MockupDB.receives` now blocks waiting for some request that does *not* match "ismaster". (Notice that `~Request.replies` returns True. This makes more advanced uses of `~MockupDB.autoresponds` easier, see the reference document.) Reply To Write Commands ----------------------- If PyMongo sends an unacknowledged OP_INSERT it does not block waiting for you to call `~Request.replies`. However, for acknowledged operations it does block. Use `~test.utils.go` to defer PyMongo to a background thread so you can respond from the main thread: >>> collection = client.db.coll >>> from mockupdb import go >>> # Default write concern is acknowledged. >>> future = go(collection.insert_one, {'_id': 1}) Pass a method and its arguments to the `go` function, the same as to `functools.partial`. It launches `~pymongo.collection.Collection.insert_one` on a thread and returns a handle to its future outcome. Meanwhile, wait for the client's request to arrive on the main thread: >>> cmd = server.receives() >>> cmd OpMsg({"insert": "coll", "ordered": true, "$db": "db", "$readPreference": {"mode": "primary"}, "documents": [{"_id": 1}]}, namespace="db") (Note how MockupDB renders requests and replies as JSON, not Python. The chief differences are that "true" and "false" are lower-case, and the order of keys and values is faithfully shown, even in Python versions with unordered dicts.) Respond thus: >>> cmd.ok() True The server's response unblocks the client, so its future contains the return value of `~pymongo.collection.Collection.insert_one`, which is an `~pymongo.results.InsertOneResult`: >>> write_result = future() >>> write_result # doctest: +ELLIPSIS >>> write_result.inserted_id 1 If you don't need the future's return value, you can express this more tersely with `going`: >>> with going(collection.insert_one, {'_id': 1}): ... server.receives().ok() True Simulate a command error: >>> future = go(collection.insert_one, {'_id': 1}) >>> server.receives(insert='coll').command_err(11000, 'eek!') True >>> future() Traceback (most recent call last): ... DuplicateKeyError: eek! Or a network error: >>> future = go(collection.insert_one, {'_id': 1}) >>> server.receives(insert='coll').hangup() True >>> future() Traceback (most recent call last): ... AutoReconnect: connection closed Pattern-Match Requests ---------------------- MockupDB's pattern-matching is useful for testing: you can tell the server to verify any aspect of the expected client request. Pass a pattern to `~.MockupDB.receives` to test that the next request matches the pattern: >>> future = go(client.db.command, 'commandFoo') >>> request = server.receives('commandBar') # doctest: +NORMALIZE_WHITESPACE Traceback (most recent call last): ... AssertionError: expected to receive Command({"commandBar": 1}), got Command({"commandFoo": 1}) Even if the pattern does not match, the request is still popped from the queue. If you do not know what order you need to accept requests, you can make a little loop: >>> import traceback >>> def loop(): ... try: ... while server.running: ... # Match queries most restrictive first. ... if server.got(OpMsg('find', 'coll', filter={'a': {'$gt': 1}})): ... server.reply(cursor={'id': 0, 'firstBatch':[{'a': 2}]}) ... elif server.got('break'): ... server.ok() ... break ... elif server.got(OpMsg('find', 'coll')): ... server.reply( ... cursor={'id': 0, 'firstBatch':[{'a': 1}, {'a': 2}]}) ... else: ... server.command_err(errmsg='unrecognized request') ... except: ... traceback.print_exc() ... raise ... >>> future = go(loop) >>> >>> list(client.db.coll.find()) [{'a': 1}, {'a': 2}] >>> list(client.db.coll.find({'a': {'$gt': 1}})) [{'a': 2}] >>> client.db.command('break') {'ok': 1} >>> future() You can even implement the "shutdown" command: >>> def loop(): ... try: ... while server.running: ... if server.got('shutdown'): ... server.stop() # Hangs up. ... else: ... server.command_err('unrecognized request') ... except: ... traceback.print_exc() ... raise ... >>> future = go(loop) >>> client.db.command('shutdown') Traceback (most recent call last): ... AutoReconnect: connection closed >>> future() >>> server.running False >>> client.close() To show off a difficult test that MockupDB makes easy, assert that PyMongo sends a ``writeConcern`` argument if you specify ``w=1``: >>> server = MockupDB() >>> responder = server.autoresponds('ismaster', maxWireVersion=6) >>> port = server.run() >>> >>> # Specify w=1. This is distinct from the default write concern. >>> client = MongoClient(server.uri, w=1) >>> collection = client.db.coll >>> future = go(collection.insert_one, {'_id': 4}) >>> server.receives({'writeConcern': {'w': 1}}).sends() True >>> client.close() ... but not by default: >>> # Accept the default write concern. >>> client = MongoClient(server.uri) >>> collection = client.db.coll >>> future = go(collection.insert_one, {'_id': 5}) >>> assert 'writeConcern' not in server.receives() >>> client.close() .. _message spec: Message Specs ------------- We've seen some examples of ways to specify messages to send, and examples of ways to assert that a reply matches an expected pattern. Both are "message specs", a flexible syntax for describing wire protocol messages. Matching a request '''''''''''''''''' One of MockupDB's most useful features for testing your application is that it can assert that your application's requests match a particular pattern: >>> client = MongoClient(server.uri) >>> future = go(client.db.collection.insert, {'_id': 1}) >>> # Assert the command name is "insert" and its parameter is "collection". >>> request = server.receives(OpMsg('insert', 'collection')) >>> request.ok() True >>> assert future() If the request did not match, MockupDB would raise an `AssertionError`. The arguments to `OpMsg` above are an example of a message spec. The pattern-matching rules are implemented in `Matcher`. Here are some more examples. The empty matcher matches anything: >>> Matcher().matches({'a': 1}) True >>> Matcher().matches({'a': 1}, {'a': 1}) True >>> Matcher().matches('ismaster') True A matcher's document matches if its key-value pairs are a subset of the request's: >>> Matcher({'a': 1}).matches({'a': 1}) True >>> Matcher({'a': 2}).matches({'a': 1}) False >>> Matcher({'a': 1}).matches({'a': 1, 'b': 1}) True Prohibit a field: >>> Matcher({'field': absent}) Matcher(Request({"field": {"absent": 1}})) >>> Matcher({'field': absent}).matches({'field': 1}) False >>> Matcher({'field': absent}).matches({'otherField': 1}) True Order matters if you use an OrderedDict: >>> doc0 = OrderedDict([('a', 1), ('b', 1)]) >>> doc1 = OrderedDict([('b', 1), ('a', 1)]) >>> Matcher(doc0).matches(doc0) True >>> Matcher(doc0).matches(doc1) False The matcher must have the same number of documents as the request: >>> Matcher().matches() True >>> Matcher([]).matches([]) True >>> Matcher({'a': 2}).matches({'a': 1}, {'a': 1}) False By default, it matches any opcode: >>> m = Matcher() >>> m.matches(OpQuery) True >>> m.matches(OpInsert) True You can specify what request opcode to match: >>> m = Matcher(OpQuery) >>> m.matches(OpInsert, {'_id': 1}) False >>> m.matches(OpQuery, {'_id': 1}) True Commands in MongoDB 3.6 and later use the OP_MSG wire protocol message. The command name is matched case-insensitively: >>> Matcher(OpMsg('ismaster')).matches(OpMsg('IsMaster')) True You can match properties specific to certain opcodes: >>> m = Matcher(OpGetMore, num_to_return=3) >>> m.matches(OpGetMore()) False >>> m.matches(OpGetMore(num_to_return=2)) False >>> m.matches(OpGetMore(num_to_return=3)) True >>> m = Matcher(OpQuery(namespace='db.collection')) >>> m.matches(OpQuery) False >>> m.matches(OpQuery(namespace='db.collection')) True It matches any wire protocol header bits you specify: >>> m = Matcher(flags=QUERY_FLAGS['SlaveOkay']) >>> m.matches(OpQuery({'_id': 1})) False >>> m.matches(OpQuery({'_id': 1}, flags=QUERY_FLAGS['SlaveOkay'])) True If you match on flags, be careful to also match on opcode. For example, if you simply check that the flag in bit position 0 is set: >>> m = Matcher(flags=INSERT_FLAGS['ContinueOnError']) ... you will match any request with that flag: >>> m.matches(OpDelete, flags=DELETE_FLAGS['SingleRemove']) True So specify the opcode, too: >>> m = Matcher(OpInsert, flags=INSERT_FLAGS['ContinueOnError']) >>> m.matches(OpDelete, flags=DELETE_FLAGS['SingleRemove']) False Sending a reply ''''''''''''''' The default reply is ``{'ok': 1}``: .. code-block:: pycon3 >>> request = server.receives() >>> request.ok() # Send {'ok': 1}. You can send additional information with the `~Request.ok` method: .. code-block:: pycon3 >>> request.ok(field='value') # Send {'ok': 1, 'field': 'value'}. Simulate a server error with `~Request.command_err`: .. code-block:: pycon3 >>> request.command_err(code=11000, errmsg='Duplicate key', field='value') All methods for sending replies parse their arguments with the `make_reply` internal function. The function interprets its first argument as the "ok" field value if it is a number, otherwise interprets it as the first field of the reply document and assumes the value is 1: >>> import mockupdb >>> mockupdb.make_op_msg_reply() OpMsgReply() >>> mockupdb.make_op_msg_reply(0) OpMsgReply({"ok": 0}) >>> mockupdb.make_op_msg_reply("foo") OpMsgReply({"foo": 1}) You can pass a dict or OrderedDict of fields instead of using keyword arguments. This is best for fieldnames that are not valid Python identifiers: >>> mockupdb.make_op_msg_reply(OrderedDict([('ok', 0), ('$err', 'bad')])) OpMsgReply({"ok": 0, "$err": "bad"}) You can customize the OP_REPLY header flags with the "flags" keyword argument: >>> r = mockupdb.make_op_msg_reply(OrderedDict([('ok', 0), ('$err', 'bad')]), ... flags=OP_MSG_FLAGS['checksumPresent']) >>> repr(r) 'OpMsgReply({"ok": 0, "$err": "bad"}, flags=checksumPresent)' Although these examples call `make_op_msg_reply` explicitly, this is only to illustrate how replies are specified. Your code will pass these arguments to a `Request` method like `~Request.replies`. Wait For A Request Impatiently ------------------------------ If your test waits for PyMongo to send a request but receives none, it times out after 10 seconds by default. This way MockupDB ensures that even failing tests all take finite time. To abbreviate the wait, pass a timeout in seconds to `~MockupDB.receives`: >>> try: ... server.receives(timeout=0.1) ... except AssertionError as err: ... print("Error: %s" % err) Error: expected to receive Request(), got nothing Test Cursor Behavior -------------------- Test what happens when a query fails: >>> cursor = collection.find().batch_size(1) >>> future = go(next, cursor) >>> server.receives(OpMsg('find', 'coll')).command_err() True >>> future() Traceback (most recent call last): ... OperationFailure: database error: MockupDB command failure You can simulate normal querying, too: >>> cursor = collection.find().batch_size(2) >>> future = go(list, cursor) >>> documents = [{'_id': 1}, {'x': 2}, {'foo': 'bar'}, {'beauty': True}] >>> request = server.receives(OpMsg('find', 'coll')) >>> n = request['batchSize'] >>> request.replies(cursor={'id': 123, 'firstBatch': documents[:n]}) True >>> while True: ... getmore = server.receives(OpMsg('getMore', 123)) ... n = getmore['batchSize'] ... if documents: ... cursor_id = 123 ... else: ... cursor_id = 0 ... getmore.ok(cursor={'id': cursor_id, 'nextBatch': documents[:n]}) ... print('returned %d' % len(documents[:n])) ... del documents[:n] ... if cursor_id == 0: ... break True returned 2 True returned 2 True returned 0 The loop receives three getMore commands and replies three times (``True`` is printed each time we call ``getmore.ok``), sending a cursor id of 0 on the last iteration to tell PyMongo that the cursor is finished. The cursor receives all documents: >>> future() [{'_id': 1}, {'x': 2}, {'_id': 1}, {'x': 2}, {'foo': 'bar'}, {'beauty': True}] But this is just a parlor trick. Let us test something serious. Test Server Discovery And Monitoring ------------------------------------ To test PyMongo's server monitor, make the server a secondary: >>> hosts = [server.address_string] >>> secondary_reply = OpReply({ ... 'ismaster': False, ... 'secondary': True, ... 'setName': 'rs', ... 'hosts': hosts, ... 'maxWireVersion': 6}) >>> responder = server.autoresponds('ismaster', secondary_reply) Connect to the replica set: >>> client = MongoClient(server.uri, replicaSet='rs') >>> from mockupdb import wait_until >>> wait_until(lambda: server.address in client.secondaries, ... 'discover secondary') True Add a primary to the host list: >>> primary = MockupDB() >>> port = primary.run() >>> hosts.append(primary.address_string) >>> primary_reply = OpReply({ ... 'ismaster': True, ... 'secondary': False, ... 'setName': 'rs', ... 'hosts': hosts, ... 'maxWireVersion': 6}) >>> responder = primary.autoresponds('ismaster', primary_reply) Client discovers it quickly if there's a pending operation: >>> with going(client.db.command, 'buildinfo'): ... wait_until(lambda: primary.address == client.primary, ... 'discovery primary') ... primary.pop('buildinfo').ok() True True .. _PyMongo: https://pypi.python.org/pypi/pymongo/ .. _MongoDB Wire Protocol: https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/ .. _serverStatus: http://docs.mongodb.org/manual/reference/server-status/ .. _collect: https://docs.python.org/2/library/gc.html#gc.collect mockupdb-1.7.0/docs/make.bat0000644000076700000240000001451512474506007017377 0ustar emptysquarestaff00000000000000@ECHO OFF REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set BUILDDIR=_build set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . set I18NSPHINXOPTS=%SPHINXOPTS% . if NOT "%PAPER%" == "" ( set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% ) if "%1" == "" goto help if "%1" == "help" ( :help echo.Please use `make ^` where ^ is one of echo. html to make standalone HTML files echo. dirhtml to make HTML files named index.html in directories echo. singlehtml to make a single large HTML file echo. pickle to make pickle files echo. json to make JSON files echo. htmlhelp to make HTML files and a HTML help project echo. qthelp to make HTML files and a qthelp project echo. devhelp to make HTML files and a Devhelp project echo. epub to make an epub echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter echo. text to make text files echo. man to make manual pages echo. texinfo to make Texinfo files echo. gettext to make PO message catalogs echo. changes to make an overview over all changed/added/deprecated items echo. xml to make Docutils-native XML files echo. pseudoxml to make pseudoxml-XML files for display purposes echo. linkcheck to check all external links for integrity echo. doctest to run all doctests embedded in the documentation if enabled goto end ) if "%1" == "clean" ( for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i del /q /s %BUILDDIR%\* goto end ) %SPHINXBUILD% 2> nul if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx echo.installed, then set the SPHINXBUILD environment variable to point echo.to the full path of the 'sphinx-build' executable. Alternatively you echo.may add the Sphinx directory to PATH. echo. echo.If you don't have Sphinx installed, grab it from echo.http://sphinx-doc.org/ exit /b 1 ) if "%1" == "html" ( %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/html. goto end ) if "%1" == "dirhtml" ( %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. goto end ) if "%1" == "singlehtml" ( %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. goto end ) if "%1" == "pickle" ( %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the pickle files. goto end ) if "%1" == "json" ( %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the JSON files. goto end ) if "%1" == "htmlhelp" ( %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run HTML Help Workshop with the ^ .hhp project file in %BUILDDIR%/htmlhelp. goto end ) if "%1" == "qthelp" ( %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run "qcollectiongenerator" with the ^ .qhcp project file in %BUILDDIR%/qthelp, like this: echo.^> qcollectiongenerator %BUILDDIR%\qthelp\mongo-mockup-db.qhcp echo.To view the help file: echo.^> assistant -collectionFile %BUILDDIR%\qthelp\mongo-mockup-db.ghc goto end ) if "%1" == "devhelp" ( %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp if errorlevel 1 exit /b 1 echo. echo.Build finished. goto end ) if "%1" == "epub" ( %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub if errorlevel 1 exit /b 1 echo. echo.Build finished. The epub file is in %BUILDDIR%/epub. goto end ) if "%1" == "latex" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex if errorlevel 1 exit /b 1 echo. echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdf" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf cd %BUILDDIR%/.. echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdfja" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf-ja cd %BUILDDIR%/.. echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "text" ( %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text if errorlevel 1 exit /b 1 echo. echo.Build finished. The text files are in %BUILDDIR%/text. goto end ) if "%1" == "man" ( %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man if errorlevel 1 exit /b 1 echo. echo.Build finished. The manual pages are in %BUILDDIR%/man. goto end ) if "%1" == "texinfo" ( %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo if errorlevel 1 exit /b 1 echo. echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. goto end ) if "%1" == "gettext" ( %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale if errorlevel 1 exit /b 1 echo. echo.Build finished. The message catalogs are in %BUILDDIR%/locale. goto end ) if "%1" == "changes" ( %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes if errorlevel 1 exit /b 1 echo. echo.The overview file is in %BUILDDIR%/changes. goto end ) if "%1" == "linkcheck" ( %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck if errorlevel 1 exit /b 1 echo. echo.Link check complete; look for any errors in the above output ^ or in %BUILDDIR%/linkcheck/output.txt. goto end ) if "%1" == "doctest" ( %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest if errorlevel 1 exit /b 1 echo. echo.Testing of doctests in the sources finished, look at the ^ results in %BUILDDIR%/doctest/output.txt. goto end ) if "%1" == "xml" ( %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml if errorlevel 1 exit /b 1 echo. echo.Build finished. The XML files are in %BUILDDIR%/xml. goto end ) if "%1" == "pseudoxml" ( %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml if errorlevel 1 exit /b 1 echo. echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. goto end ) :end mockupdb-1.7.0/docs/installation.rst0000644000076700000240000000016013003675262021213 0ustar emptysquarestaff00000000000000============ Installation ============ Install MockupDB with pip: $ python -m pip install mongo-mockup-db mockupdb-1.7.0/docs/authors.rst0000644000076700000240000000003412474506007020200 0ustar emptysquarestaff00000000000000.. include:: ../AUTHORS.rst mockupdb-1.7.0/docs/changelog.rst0000644000076700000240000000003612474512511020441 0ustar emptysquarestaff00000000000000.. include:: ../CHANGELOG.rst mockupdb-1.7.0/mockupdb.egg-info/0000755000076700000240000000000013400765404020330 5ustar emptysquarestaff00000000000000mockupdb-1.7.0/mockupdb.egg-info/PKG-INFO0000664000076700000240000001233213400765404021430 0ustar emptysquarestaff00000000000000Metadata-Version: 1.1 Name: mockupdb Version: 1.7.0 Summary: MongoDB Wire Protocol server library Home-page: https://github.com/ajdavis/mongo-mockup-db Author: A. Jesse Jiryu Davis Author-email: jesse@mongodb.com License: Apache License, Version 2.0 Description-Content-Type: UNKNOWN Description: ======== MockupDB ======== Mock server for testing MongoDB clients and creating MongoDB Wire Protocol servers. * Documentation: http://mockupdb.readthedocs.org/ Changelog ========= 1.7.0 (2018-12-02) ------------------ Improve datetime support in match expressions. Python datetimes have microsecond precision but BSON only has milliseconds, so expressions like this always failed:: server.receives(Command('foo', when=datetime(2018, 12, 1, 6, 6, 6, 12345))) Now, the matching logic has been rewritten to recurse through arrays and subdocuments, comparing them value by value. It compares datetime values with only millisecond precision. 1.6.0 (2018-11-16) ------------------ Remove vendored BSON library. Instead, require PyMongo and use its BSON library. This avoids surprising problems where a BSON type created with PyMongo does not appear equal to one created with MockupDB, and it avoids the occasional need to update the vendored code to support new BSON features. 1.5.0 (2018-11-02) ------------------ Support for Unix domain paths with ``uds_path`` parameter. The ``interactive_server()`` function now prepares the server to autorespond to the ``getFreeMonitoringStatus`` command from the mongo shell. 1.4.1 (2018-06-30) ------------------ Fix an inadvertent dependency on PyMongo, which broke the docs build. 1.4.0 (2018-06-29) ------------------ Support, and expect, OP_MSG requests from clients. Thanks to Shane Harvey for the contribution. Update vendored bson library from PyMongo. Support the Decimal128 BSON type. Fix Matcher so it equates BSON objects from PyMongo like ``ObjectId(...)`` with equivalent objects created from MockupDB's vendored bson library. 1.3.0 (2018-02-19) ------------------ Support Windows. Log a traceback if a bad client request causes an assert. Fix SSL. Make errors less likely on shutdown. Enable testing on Travis and Appveyor. Fix doctests and interactive server for modern MongoDB protocol. 1.2.1 (2017-12-06) ------------------ Set minWireVersion to 0, not to 2. I had been wrong about MongoDB 3.6's wire version range: it's actually 0 to 6. MockupDB now reports the same wire version range as MongoDB 3.6 by default. 1.2.0 (2017-09-22) ------------------ Update for MongoDB 3.6: report minWireVersion 2 and maxWireVersion 6 by default. 1.1.3 (2017-04-23) ------------------ Avoid rare RuntimeError in close(), if a client thread shuts down a socket as MockupDB iterates its list of sockets. 1.1.2 (2016-08-23) ------------------ Properly detect closed sockets so ``MockupDB.stop()`` doesn't take 10 seconds per connection. Thanks to Sean Purcell. 1.1.1 (2016-08-01) ------------------ Don't use "client" as a keyword arg for ``Request``, it conflicts with the actual "client" field in drivers' new handshake protocol. 1.1.0 (2016-02-11) ------------------ Add cursor_id property to OpGetMore, and ssl parameter to interactive_server. 1.0.3 (2015-09-12) ------------------ ``MockupDB(auto_ismaster=True)`` had just responded ``{"ok": 1}``, but this isn't enough to convince PyMongo 3 it's talking to a valid standalone, so auto-respond ``{"ok": 1, "ismaster": True}``. 1.0.2 (2015-09-11) ------------------ Restore Request.assert_matches method, used in pymongo-mockup-tests. 1.0.1 (2015-09-11) ------------------ Allow co-installation with PyMongo. 1.0.0 (2015-09-10) ------------------ First release. 0.1.0 (2015-02-25) ------------------ Development begun. Keywords: mongo,mongodb,wire protocol,mockupdb,mock Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: Apache Software License Classifier: Natural Language :: English Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 mockupdb-1.7.0/mockupdb.egg-info/not-zip-safe0000664000076700000240000000000112757075745022600 0ustar emptysquarestaff00000000000000 mockupdb-1.7.0/mockupdb.egg-info/SOURCES.txt0000664000076700000240000000105613400765404022220 0ustar emptysquarestaff00000000000000AUTHORS.rst CHANGELOG.rst CONTRIBUTING.rst LICENSE MANIFEST.in README.rst setup.cfg setup.py docs/Makefile docs/authors.rst docs/changelog.rst docs/conf.py docs/contributing.rst docs/index.rst docs/installation.rst docs/make.bat docs/reference.rst docs/tutorial.rst mockupdb/__init__.py mockupdb/__main__.py mockupdb/server.pem mockupdb.egg-info/PKG-INFO mockupdb.egg-info/SOURCES.txt mockupdb.egg-info/dependency_links.txt mockupdb.egg-info/not-zip-safe mockupdb.egg-info/requires.txt mockupdb.egg-info/top_level.txt tests/__init__.py tests/test_mockupdb.pymockupdb-1.7.0/mockupdb.egg-info/requires.txt0000644000076700000240000000001313400765404022722 0ustar emptysquarestaff00000000000000pymongo>=3 mockupdb-1.7.0/mockupdb.egg-info/top_level.txt0000664000076700000240000000001113400765404023054 0ustar emptysquarestaff00000000000000mockupdb mockupdb-1.7.0/mockupdb.egg-info/dependency_links.txt0000664000076700000240000000000113400765404024400 0ustar emptysquarestaff00000000000000 mockupdb-1.7.0/setup.py0000755000076700000240000000315313400765373016556 0ustar emptysquarestaff00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- import sys try: from setuptools import setup except ImportError: from distutils.core import setup with open('README.rst') as readme_file: readme = readme_file.read() with open('CHANGELOG.rst') as changelog_file: changelog = changelog_file.read().replace('.. :changelog:', '') requirements = ['pymongo>=3'] test_requirements = [] if sys.version_info[:2] == (2, 6): requirements.append('ordereddict') test_requirements.append('unittest2') setup( name='mockupdb', version='1.7.0', description="MongoDB Wire Protocol server library", long_description=readme + '\n\n' + changelog, author="A. Jesse Jiryu Davis", author_email='jesse@mongodb.com', url='https://github.com/ajdavis/mongo-mockup-db', packages=['mockupdb'], package_dir={'mockupdb': 'mockupdb'}, include_package_data=True, install_requires=requirements, license="Apache License, Version 2.0", zip_safe=False, keywords=["mongo", "mongodb", "wire protocol", "mockupdb", "mock"], classifiers=[ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', "License :: OSI Approved :: Apache Software License", 'Natural Language :: English', "Programming Language :: Python :: 2", 'Programming Language :: Python :: 2.6', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.3', 'Programming Language :: Python :: 3.4', ], test_suite='tests', tests_require=test_requirements ) mockupdb-1.7.0/AUTHORS.rst0000644000076700000240000000024213373651532016714 0ustar emptysquarestaff00000000000000======= Credits ======= Development Lead ---------------- * A. Jesse Jiryu Davis Contributors ------------ * George Wilson * Shane Harvey mockupdb-1.7.0/mockupdb/0000755000076700000240000000000013400765404016636 5ustar emptysquarestaff00000000000000mockupdb-1.7.0/mockupdb/server.pem0000644000076700000240000000371213241707757020664 0ustar emptysquarestaff00000000000000-----BEGIN PRIVATE KEY----- MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAK53miP9GczBWXnq NxHwQkgVqsDuesjwJbWilMK4gf3fjnf2PN3qDpnGbZbPD0ij8975pIKtSPoDycFm A8Mogip0yU2Lv2lL56CWthSBftOFDL2CWIsmuuURFXZPiVLtLytfI9oLASZFlywW Cs83qEDTvdW8VoVhVsxV1JFDnpXLAgMBAAECgYBoGBgxrMt97UazhNkCrPT/CV5t 6lv8E7yMGMrlOyzkCkR4ssQyK3o2qbutJTGbR6czvIM5LKbD9Qqlh3ZrNHokWmTR VQQpJxt8HwP5boQvwRHg9+KSGr4JvRko1qxFs9C7Bzjt4r9VxdjhwZPdy0McGI/z yPXyQHjqBayrHV1EwQJBANorfCKeIxLhH3LAeUZuRS8ACldJ2N1kL6Ov43/v+0S/ OprQeBTODuTds3sv7FCT1aYDTOe6JLNOwN2i4YVOMBsCQQDMuCozrwqftD17D06P 9+lRXUekY5kFBs5j28Xnl8t8jnuxsXtQUTru660LD0QrmDNSauhpEmlpJknicnGt hmwRAkEA12MI6bBPlir0/jgxQqxI1w7mJqj8Vg27zpEuO7dzzLoyJHddpcSNBbwu npaAakiZK42klj26T9+XHvjYRuAbMwJBAJ5WnwWEkGH/pUHGEAyYQdSVojDKe/MA Vae0tzguFswK5C8GyArSGRPsItYYA7D4MlG/sGx8Oh2C6MiFndkJzBECQDcP1y4r Qsek151t1zArLKH4gG5dQAeZ0Lc2VeC4nLMUqVwrHcZDdd1RzLlSaH3j1MekFVfT 6v6rrcNLEVbeuk4= -----END PRIVATE KEY----- -----BEGIN CERTIFICATE----- MIIC7jCCAlegAwIBAgIBCjANBgkqhkiG9w0BAQUFADCBkjELMAkGA1UEBhMCVVMx ETAPBgNVBAgMCE5ldyBZb3JrMRYwFAYDVQQHDA1OZXcgWW9yayBDaXR5MQ4wDAYD VQQKDAUxMEdlbjEPMA0GA1UECwwGS2VybmVsMRowGAYDVQQDDBFNeSBDZXJ0IEF1 dGhvcml0eTEbMBkGCSqGSIb3DQEJARYMcm9vdEBsYXphcnVzMB4XDTEzMTIwNTEz MjU0MFoXDTQxMDQyMTEzMjU0MFowajELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE5l dyBZb3JrMRYwFAYDVQQHDA1OZXcgWW9yayBDaXR5MQ4wDAYDVQQKDAUxMEdlbjEP MA0GA1UECwwGS2VybmVsMQ8wDQYDVQQDDAZzZXJ2ZXIwgZ8wDQYJKoZIhvcNAQEB BQADgY0AMIGJAoGBAK53miP9GczBWXnqNxHwQkgVqsDuesjwJbWilMK4gf3fjnf2 PN3qDpnGbZbPD0ij8975pIKtSPoDycFmA8Mogip0yU2Lv2lL56CWthSBftOFDL2C WIsmuuURFXZPiVLtLytfI9oLASZFlywWCs83qEDTvdW8VoVhVsxV1JFDnpXLAgMB AAGjezB5MAkGA1UdEwQCMAAwLAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJh dGVkIENlcnRpZmljYXRlMB0GA1UdDgQWBBQgCkKiZhUV9/Zo7RwYYwm2cNK6tzAf BgNVHSMEGDAWgBQHQRk6n37FtyJOt7zV3+T8CbhkFjANBgkqhkiG9w0BAQUFAAOB gQCbsfr+Q4pty4Fy38lSxoCgnbB4pX6+Ex3xyw5zxDYR3xUlb/uHBiNZ1dBrXBxU ekU8dEvf+hx4iRDSW/C5N6BGnBBhCHcrPabo2bEEWKVsbUC3xchTB5rNGkvnMt9t G9ol7vanuzjL3S8/2PB33OshkBH570CxqqPflQbdjwt9dg== -----END CERTIFICATE----- mockupdb-1.7.0/mockupdb/__init__.py0000755000076700000240000017646613400765373021003 0ustar emptysquarestaff00000000000000# -*- coding: utf-8 -*- # Copyright 2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Simulate a MongoDB server, for use in unittests.""" from __future__ import print_function __author__ = 'A. Jesse Jiryu Davis' __email__ = 'jesse@mongodb.com' __version__ = '1.7.0' import atexit import contextlib import datetime import errno import functools import inspect import os import random import select import ssl as _ssl import socket import struct import traceback import threading import time import weakref import sys from codecs import utf_8_decode as _utf_8_decode try: from queue import Queue, Empty except ImportError: from Queue import Queue, Empty try: from collections.abc import Mapping except: from collections import Mapping try: from collections import OrderedDict except: from ordereddict import OrderedDict # Python 2.6, "pip install ordereddict" try: from io import StringIO except ImportError: from cStringIO import StringIO try: from urllib.parse import quote_plus except ImportError: # Python 2 from urllib import quote_plus import bson from bson import codec_options, json_util CODEC_OPTIONS = codec_options.CodecOptions(document_class=OrderedDict) PY3 = sys.version_info[0] == 3 if PY3: string_type = str text_type = str def reraise(exctype, value, trace=None): raise exctype(str(value)).with_traceback(trace) else: string_type = basestring text_type = unicode # "raise x, y, z" raises SyntaxError in Python 3. exec ("""def reraise(exctype, value, trace=None): raise exctype, str(value), trace """) __all__ = [ 'MockupDB', 'go', 'going', 'Future', 'wait_until', 'interactive_server', 'OP_REPLY', 'OP_UPDATE', 'OP_INSERT', 'OP_QUERY', 'OP_GET_MORE', 'OP_DELETE', 'OP_KILL_CURSORS', 'OP_MSG', 'QUERY_FLAGS', 'UPDATE_FLAGS', 'INSERT_FLAGS', 'DELETE_FLAGS', 'REPLY_FLAGS', 'OP_MSG_FLAGS', 'Request', 'Command', 'OpQuery', 'OpGetMore', 'OpKillCursors', 'OpInsert', 'OpUpdate', 'OpDelete', 'OpReply', 'OpMsg', 'Matcher', 'absent', ] def go(fn, *args, **kwargs): """Launch an operation on a thread and get a handle to its future result. >>> from time import sleep >>> def print_sleep_print(duration): ... sleep(duration) ... print('hello from background thread') ... sleep(duration) ... print('goodbye from background thread') ... return 'return value' ... >>> future = go(print_sleep_print, 0.1) >>> sleep(0.15) hello from background thread >>> print('main thread') main thread >>> result = future() goodbye from background thread >>> result 'return value' """ if not callable(fn): raise TypeError('go() requires a function, not %r' % (fn,)) result = [None] error = [] def target(): try: result[0] = fn(*args, **kwargs) except Exception: # Are we in interpreter shutdown? if sys: error.extend(sys.exc_info()) t = threading.Thread(target=target) t.daemon = True t.start() def get_result(timeout=10): t.join(timeout) if t.is_alive(): raise AssertionError('timed out waiting for %r' % fn) if error: reraise(*error) return result[0] return get_result @contextlib.contextmanager def going(fn, *args, **kwargs): """Launch a thread and wait for its result before exiting the code block. >>> with going(lambda: 'return value') as future: ... pass >>> future() # Won't block, the future is ready by now. 'return value' Or discard the result: >>> with going(lambda: "don't care"): ... pass If an exception is raised within the context, the result is lost: >>> with going(lambda: 'return value') as future: ... assert 1 == 0 Traceback (most recent call last): ... AssertionError """ future = go(fn, *args, **kwargs) try: yield future except: # We are raising an exception, just try to clean up the future. exc_info = sys.exc_info() try: # Shorter than normal timeout. future(timeout=1) except: log_message = ('\nerror in %s:\n' % format_call(inspect.currentframe())) sys.stderr.write(log_message) traceback.print_exc() # sys.stderr.write('exc in %s' % format_call(inspect.currentframe())) reraise(*exc_info) else: # Raise exception or discard result. future(timeout=10) class Future(object): def __init__(self): self._result = None self._event = threading.Event() def result(self, timeout=None): self._event.wait(timeout) # wait() always returns None in Python 2.6. if not self._event.is_set(): raise AssertionError('timed out waiting for Future') return self._result def set_result(self, result): if self._event.is_set(): raise RuntimeError("Future is already resolved") self._result = result self._event.set() def wait_until(predicate, success_description, timeout=10): """Wait up to 10 seconds (by default) for predicate to be true. E.g.: wait_until(lambda: client.primary == ('a', 1), 'connect to the primary') If the lambda-expression isn't true after 10 seconds, we raise AssertionError("Didn't ever connect to the primary"). Returns the predicate's first true value. """ start = time.time() while True: retval = predicate() if retval: return retval if time.time() - start > timeout: raise AssertionError("Didn't ever %s" % success_description) time.sleep(0.1) OP_REPLY = 1 OP_UPDATE = 2001 OP_INSERT = 2002 OP_QUERY = 2004 OP_GET_MORE = 2005 OP_DELETE = 2006 OP_KILL_CURSORS = 2007 OP_MSG = 2013 QUERY_FLAGS = OrderedDict([ ('TailableCursor', 2), ('SlaveOkay', 4), ('OplogReplay', 8), ('NoTimeout', 16), ('AwaitData', 32), ('Exhaust', 64), ('Partial', 128)]) UPDATE_FLAGS = OrderedDict([ ('Upsert', 1), ('MultiUpdate', 2)]) INSERT_FLAGS = OrderedDict([ ('ContinueOnError', 1)]) DELETE_FLAGS = OrderedDict([ ('SingleRemove', 1)]) REPLY_FLAGS = OrderedDict([ ('CursorNotFound', 1), ('QueryFailure', 2)]) OP_MSG_FLAGS = OrderedDict([ ('checksumPresent', 1), ('moreToCome', 2)]) _UNPACK_BYTE = struct.Struct(">> {'_id': 0} in OpInsert({'_id': 0}) True >>> {'_id': 1} in OpInsert({'_id': 0}) False >>> {'_id': 1} in OpInsert([{'_id': 0}, {'_id': 1}]) True >>> {'_id': 1} == OpInsert([{'_id': 0}, {'_id': 1}])[1] True >>> 'field' in OpMsg(field=1) True >>> 'field' in OpMsg() False >>> 'field' in OpMsg('ismaster') False >>> OpMsg(ismaster=False)['ismaster'] is False True """ opcode = None is_command = None _non_matched_attrs = 'doc', 'docs' _flags_map = None def __init__(self, *args, **kwargs): self._flags = kwargs.pop('flags', None) self._namespace = kwargs.pop('namespace', None) self._client = kwargs.pop('_client', None) self._request_id = kwargs.pop('request_id', None) self._server = kwargs.pop('_server', None) self._verbose = self._server and self._server.verbose self._server_port = kwargs.pop('server_port', None) self._docs = make_docs(*args, **kwargs) if not all(_ismap(doc) for doc in self._docs): raise_args_err() @property def doc(self): """The request document, if there is exactly one. Use this for queries, commands, and legacy deletes. Legacy writes may have many documents, OP_GET_MORE and OP_KILL_CURSORS have none. """ assert len(self.docs) == 1, '%r has more than one document' % self return self.docs[0] @property def docs(self): """The request documents, if any.""" return self._docs @property def namespace(self): """The operation namespace or None.""" return self._namespace @property def flags(self): """The request flags or None.""" return self._flags @property def slave_ok(self): """True if the SlaveOkay wire protocol flag is set.""" return self._flags and bool( self._flags & QUERY_FLAGS['SlaveOkay']) slave_okay = slave_ok """Synonym for `.slave_ok`.""" @property def request_id(self): """The request id or None.""" return self._request_id @property def client_port(self): """Client connection's TCP port.""" address = self._client.getpeername() if isinstance(address, tuple): return address[1] # Maybe a Unix domain socket connection. return 0 @property def server(self): """The `.MockupDB` server.""" return self._server def assert_matches(self, *args, **kwargs): """Assert this matches a :ref:`message spec `. Returns self. """ matcher = make_matcher(*args, **kwargs) if not matcher.matches(self): raise AssertionError('%r does not match %r' % (self, matcher)) return self def matches(self, *args, **kwargs): """True if this matches a :ref:`message spec `.""" return make_matcher(*args, **kwargs).matches(self) def replies(self, *args, **kwargs): """Send an `OpReply` to the client. The default reply to a command is ``{'ok': 1}``, otherwise the default is empty (no documents). Returns True so it is suitable as an `~MockupDB.autoresponds` handler. """ self._replies(*args, **kwargs) return True ok = send = sends = reply = replies """Synonym for `.replies`.""" def fail(self, err='MockupDB query failure', *args, **kwargs): """Reply to a query with the QueryFailure flag and an '$err' key. Returns True so it is suitable as an `~MockupDB.autoresponds` handler. """ kwargs.setdefault('flags', 0) kwargs['flags'] |= REPLY_FLAGS['QueryFailure'] kwargs['$err'] = err self.replies(*args, **kwargs) return True def command_err(self, code=1, errmsg='MockupDB command failure', *args, **kwargs): """Error reply to a command. Returns True so it is suitable as an `~MockupDB.autoresponds` handler. """ kwargs.setdefault('ok', 0) kwargs['code'] = code kwargs['errmsg'] = errmsg self.replies(*args, **kwargs) return True def hangup(self): """Close the connection. Returns True so it is suitable as an `~MockupDB.autoresponds` handler. """ if self._server: self._server._log('\t%d\thangup' % self.client_port) self._client.shutdown(socket.SHUT_RDWR) return True hangs_up = hangup """Synonym for `.hangup`.""" def _matches_docs(self, docs, other_docs): """Overridable method.""" for doc, other_doc in zip(docs, other_docs): if not self._match_map(doc, other_doc): return False return True def _match_map(self, doc, other_doc): for key, val in doc.items(): if val is absent: if key in other_doc: return False elif not self._match_val(val, other_doc.get(key, None)): return False if isinstance(doc, (OrderedDict, bson.SON)): if not isinstance(other_doc, (OrderedDict, bson.SON)): raise TypeError( "Can't compare ordered and unordered document types:" " %r, %r" % (doc, other_doc)) keys = [key for key, val in doc.items() if val is not absent] if not seq_match(keys, list(other_doc.keys())): return False return True def _match_list(self, lst, other_lst): if len(lst) != len(other_lst): return False for val, other_val in zip(lst, other_lst): if not self._match_val(val, other_val): return False return True def _match_val(self, val, other_val): if _ismap(val) and _ismap(other_val): if not self._match_map(val, other_val): return False elif _islist(val) and _islist(other_val): if not self._match_list(val, other_val): return False elif (isinstance(val, datetime.datetime) and isinstance(other_val, datetime.datetime)): if _dt_rounded(val) != _dt_rounded(other_val): return False elif val != other_val: return False return True def _replies(self, *args, **kwargs): """Overridable method.""" reply_msg = make_reply(*args, **kwargs) if self._server: self._server._log('\t%d\t<-- %r' % (self.client_port, reply_msg)) reply_bytes = reply_msg.reply_bytes(self) self._client.sendall(reply_bytes) def __contains__(self, item): if item in self.docs: return True if len(self.docs) == 1 and isinstance(item, (string_type, text_type)): return item in self.doc return False def __getitem__(self, item): return self.doc[item] if len(self.docs) == 1 else self.docs[item] def __str__(self): return docs_repr(*self.docs) def __repr__(self): name = self.__class__.__name__ parts = [] if self.docs: parts.append(docs_repr(*self.docs)) if self._flags: if self._flags_map: parts.append('flags=%s' % ( '|'.join(name for name, value in self._flags_map.items() if self._flags & value))) else: parts.append('flags=%d' % self._flags) if self._namespace: parts.append('namespace="%s"' % self._namespace) return '%s(%s)' % (name, ', '.join(str(part) for part in parts)) class CommandBase(Request): """A command the client executes on the server.""" is_command = True # Check command name case-insensitively. _non_matched_attrs = Request._non_matched_attrs + ('command_name',) @property def command_name(self): """The command name or None. >>> OpMsg({'count': 'collection'}).command_name 'count' >>> OpMsg('aggregate', 'collection', cursor=absent).command_name 'aggregate' """ if self.docs and self.docs[0]: return list(self.docs[0])[0] def _matches_docs(self, docs, other_docs): assert len(docs) == len(other_docs) == 1 doc, = docs other_doc, = other_docs items = list(doc.items()) other_items = list(other_doc.items()) # Compare command name case-insensitively. if items and other_items: if items[0][0].lower() != other_items[0][0].lower(): return False if items[0][1] != other_items[0][1]: return False return super(CommandBase, self)._matches_docs( [OrderedDict(items[1:])], [OrderedDict(other_items[1:])]) class OpMsg(CommandBase): """An OP_MSG request the client executes on the server.""" opcode = OP_MSG is_command = True _flags_map = OP_MSG_FLAGS @classmethod def unpack(cls, msg, client, server, request_id): """Parse message and return an `OpMsg`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ flags, = _UNPACK_UINT(msg[:4]) pos = 4 first_payload_type, = _UNPACK_BYTE(msg[pos:pos + 1]) pos += 1 first_payload_size, = _UNPACK_INT(msg[pos:pos + 4]) if flags != 0 and flags != 2: raise ValueError('OP_MSG flag must be 0 or 2 not %r' % (flags,)) if first_payload_type != 0: raise ValueError('First OP_MSG payload type must be 0 not %r' % ( first_payload_type,)) # Parse the initial document and add the optional payload type 1. payload_document = bson.decode_all(msg[pos:pos + first_payload_size], CODEC_OPTIONS)[0] pos += first_payload_size if len(msg) != pos: payload_type, = _UNPACK_BYTE(msg[pos:pos + 1]) pos += 1 if payload_type != 1: raise ValueError('Second OP_MSG payload type must be 1 not %r' % (payload_type,)) section_size, = _UNPACK_INT(msg[pos:pos + 4]) if len(msg) != pos + section_size: raise ValueError('More than two OP_MSG sections unsupported') pos += 4 identifier, pos = _get_c_string(msg, pos) documents = bson.decode_all(msg[pos:], CODEC_OPTIONS) payload_document[identifier] = documents database = payload_document['$db'] return OpMsg(payload_document, namespace=database, flags=flags, _client=client, request_id=request_id, _server=server) def __init__(self, *args, **kwargs): super(OpMsg, self).__init__(*args, **kwargs) if len(self._docs) > 1: raise_args_err('OpMsg too many documents', ValueError) @property def slave_ok(self): """True if this OpMsg can read from a secondary.""" read_preference = self.doc.get('$readPreference') return read_preference and read_preference.get('mode') != 'primary' slave_okay = slave_ok """Synonym for `.slave_ok`.""" @property def command_name(self): """The command name or None. >>> OpMsg({'count': 'collection'}).command_name 'count' >>> OpMsg('aggregate', 'collection', cursor=absent).command_name 'aggregate' """ if self.docs and self.docs[0]: return list(self.docs[0])[0] def _replies(self, *args, **kwargs): if self.flags & OP_MSG_FLAGS['moreToCome']: assert False, "Cannot reply to OpMsg with moreToCome: %r" % (self,) reply = make_op_msg_reply(*args, **kwargs) if not reply.docs: reply.docs = [{'ok': 1}] else: if len(reply.docs) > 1: raise ValueError('OP_MSG reply with multiple documents: %s' % (reply.docs,)) reply.doc.setdefault('ok', 1) super(OpMsg, self)._replies(reply) class OpQuery(Request): """A query (besides a command) the client executes on the server. >>> OpQuery({'i': {'$gt': 2}}, fields={'j': False}) OpQuery({"i": {"$gt": 2}}, fields={"j": false}) """ opcode = OP_QUERY is_command = False _flags_map = QUERY_FLAGS @classmethod def unpack(cls, msg, client, server, request_id): """Parse message and return an `OpQuery` or `Command`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ flags, = _UNPACK_INT(msg[:4]) namespace, pos = _get_c_string(msg, 4) is_command = namespace.endswith('.$cmd') num_to_skip, = _UNPACK_INT(msg[pos:pos + 4]) pos += 4 num_to_return, = _UNPACK_INT(msg[pos:pos + 4]) pos += 4 docs = bson.decode_all(msg[pos:], CODEC_OPTIONS) if is_command: assert len(docs) == 1 command_ns = namespace[:-len('.$cmd')] return Command(docs, namespace=command_ns, flags=flags, _client=client, request_id=request_id, _server=server) else: if len(docs) == 1: fields = None else: assert len(docs) == 2 fields = docs[1] return OpQuery(docs[0], fields=fields, namespace=namespace, flags=flags, num_to_skip=num_to_skip, num_to_return=num_to_return, _client=client, request_id=request_id, _server=server) def __init__(self, *args, **kwargs): fields = kwargs.pop('fields', None) if fields is not None and not _ismap(fields): raise_args_err() self._fields = fields self._num_to_skip = kwargs.pop('num_to_skip', None) self._num_to_return = kwargs.pop('num_to_return', None) super(OpQuery, self).__init__(*args, **kwargs) if not self._docs: self._docs = [{}] # Default query filter. elif len(self._docs) > 1: raise_args_err('OpQuery too many documents', ValueError) @property def num_to_skip(self): """Client query's numToSkip or None.""" return self._num_to_skip @property def num_to_return(self): """Client query's numToReturn or None.""" return self._num_to_return @property def fields(self): """Client query's fields selector or None.""" return self._fields def __repr__(self): rep = super(OpQuery, self).__repr__().rstrip(')') if self._fields: rep += ', fields=%s' % docs_repr(self._fields) if self._num_to_skip is not None: rep += ', numToSkip=%d' % self._num_to_skip if self._num_to_return is not None: rep += ', numToReturn=%d' % self._num_to_return return rep + ')' class Command(CommandBase, OpQuery): """A command the client executes on the server.""" def _replies(self, *args, **kwargs): reply = make_reply(*args, **kwargs) if not reply.docs: reply.docs = [{'ok': 1}] else: if len(reply.docs) > 1: raise ValueError('Command reply with multiple documents: %s' % (reply.docs,)) reply.doc.setdefault('ok', 1) super(Command, self)._replies(reply) def replies_to_gle(self, **kwargs): """Send a getlasterror response. Defaults to ``{ok: 1, err: null}``. Add or override values by passing keyword arguments. Returns True so it is suitable as an `~MockupDB.autoresponds` handler. """ kwargs.setdefault('err', None) return self.replies(**kwargs) class OpGetMore(Request): """An OP_GET_MORE the client executes on the server.""" @classmethod def unpack(cls, msg, client, server, request_id): """Parse message and return an `OpGetMore`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ flags, = _UNPACK_INT(msg[:4]) namespace, pos = _get_c_string(msg, 4) num_to_return, = _UNPACK_INT(msg[pos:pos + 4]) pos += 4 cursor_id, = _UNPACK_LONG(msg[pos:pos + 8]) return OpGetMore(namespace=namespace, flags=flags, _client=client, num_to_return=num_to_return, cursor_id=cursor_id, request_id=request_id, _server=server) def __init__(self, **kwargs): self._num_to_return = kwargs.pop('num_to_return', None) self._cursor_id = kwargs.pop('cursor_id', None) super(OpGetMore, self).__init__(**kwargs) @property def num_to_return(self): """The client message's numToReturn field.""" return self._num_to_return @property def cursor_id(self): """The client message's cursorId field.""" return self._cursor_id class OpKillCursors(Request): """An OP_KILL_CURSORS the client executes on the server.""" @classmethod def unpack(cls, msg, client, server, _): """Parse message and return an `OpKillCursors`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ # Leading 4 bytes are reserved. num_of_cursor_ids, = _UNPACK_INT(msg[4:8]) cursor_ids = [] pos = 8 for _ in range(num_of_cursor_ids): cursor_ids.append(_UNPACK_INT(msg[pos:pos + 4])[0]) pos += 4 return OpKillCursors(_client=client, cursor_ids=cursor_ids, _server=server) def __init__(self, **kwargs): self._cursor_ids = kwargs.pop('cursor_ids', None) super(OpKillCursors, self).__init__(**kwargs) @property def cursor_ids(self): """List of cursor ids the client wants to kill.""" return self._cursor_ids def __repr__(self): return '%s(%s)' % (self.__class__.__name__, self._cursor_ids) class _LegacyWrite(Request): is_command = False class OpInsert(_LegacyWrite): """A legacy OP_INSERT the client executes on the server.""" opcode = OP_INSERT _flags_map = INSERT_FLAGS @classmethod def unpack(cls, msg, client, server, request_id): """Parse message and return an `OpInsert`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ flags, = _UNPACK_INT(msg[:4]) namespace, pos = _get_c_string(msg, 4) docs = bson.decode_all(msg[pos:], CODEC_OPTIONS) return cls(*docs, namespace=namespace, flags=flags, _client=client, request_id=request_id, _server=server) class OpUpdate(_LegacyWrite): """A legacy OP_UPDATE the client executes on the server.""" opcode = OP_UPDATE _flags_map = UPDATE_FLAGS @classmethod def unpack(cls, msg, client, server, request_id): """Parse message and return an `OpUpdate`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ # First 4 bytes of OP_UPDATE are "reserved". namespace, pos = _get_c_string(msg, 4) flags, = _UNPACK_INT(msg[pos:pos + 4]) docs = bson.decode_all(msg[pos + 4:], CODEC_OPTIONS) return cls(*docs, namespace=namespace, flags=flags, _client=client, request_id=request_id, _server=server) class OpDelete(_LegacyWrite): """A legacy OP_DELETE the client executes on the server.""" opcode = OP_DELETE _flags_map = DELETE_FLAGS @classmethod def unpack(cls, msg, client, server, request_id): """Parse message and return an `OpDelete`. Takes the client message as bytes, the client and server socket objects, and the client request id. """ # First 4 bytes of OP_DELETE are "reserved". namespace, pos = _get_c_string(msg, 4) flags, = _UNPACK_INT(msg[pos:pos + 4]) docs = bson.decode_all(msg[pos + 4:], CODEC_OPTIONS) return cls(*docs, namespace=namespace, flags=flags, _client=client, request_id=request_id, _server=server) class Reply(object): """A reply from `MockupDB` to the client.""" def __init__(self, *args, **kwargs): self._flags = kwargs.pop('flags', 0) self._docs = make_docs(*args, **kwargs) @property def doc(self): """Contents of reply. Useful for replies to commands; replies to other messages may have no documents or multiple documents. """ assert len(self._docs) == 1, '%s has more than one document' % self return self._docs[0] def __str__(self): return docs_repr(*self._docs) def __repr__(self): rep = '%s(%s' % (self.__class__.__name__, self) if self._flags: rep += ', flags=' + '|'.join( name for name, value in REPLY_FLAGS.items() if self._flags & value) return rep + ')' class OpReply(Reply): """An OP_REPLY reply from `MockupDB` to the client.""" def __init__(self, *args, **kwargs): self._cursor_id = kwargs.pop('cursor_id', 0) self._starting_from = kwargs.pop('starting_from', 0) super(OpReply, self).__init__(*args, **kwargs) @property def docs(self): """The reply documents, if any.""" return self._docs @docs.setter def docs(self, docs): self._docs = make_docs(docs) def update(self, *args, **kwargs): """Update the document. Same as ``dict().update()``. >>> reply = OpReply({'ismaster': True}) >>> reply.update(maxWireVersion=3) >>> reply.doc['maxWireVersion'] 3 >>> reply.update({'maxWriteBatchSize': 10, 'msg': 'isdbgrid'}) """ self.doc.update(*args, **kwargs) def reply_bytes(self, request): """Take a `Request` and return an OP_REPLY message as bytes.""" flags = struct.pack(">> reply = OpMsgReply({'ismaster': True}) >>> reply.update(maxWireVersion=3) >>> reply.doc['maxWireVersion'] 3 >>> reply.update({'maxWriteBatchSize': 10, 'msg': 'isdbgrid'}) """ self.doc.update(*args, **kwargs) def reply_bytes(self, request): """Take a `Request` and return an OP_MSG message as bytes.""" flags = struct.pack("`. Used by `~MockupDB.receives` to assert the client sent the expected request, and by `~MockupDB.got` to test if it did and return ``True`` or ``False``. Used by `.autoresponds` to match requests with autoresponses. """ def __init__(self, *args, **kwargs): self._kwargs = kwargs self._prototype = make_prototype_request(*args, **kwargs) def matches(self, *args, **kwargs): """Test if a request matches a :ref:`message spec `. Returns ``True`` or ``False``. """ request = make_prototype_request(*args, **kwargs) if self._prototype.opcode not in (None, request.opcode): return False if self._prototype.is_command not in (None, request.is_command): return False for name in dir(self._prototype): if name.startswith('_') or name in request._non_matched_attrs: # Ignore privates, and handle documents specially. continue prototype_value = getattr(self._prototype, name, None) if inspect.ismethod(prototype_value): continue actual_value = getattr(request, name, None) if prototype_value not in (None, actual_value): return False if len(self._prototype.docs) not in (0, len(request.docs)): return False return self._prototype._matches_docs(self._prototype.docs, request.docs) @property def prototype(self): """The prototype `.Request` used to match actual requests with.""" return self._prototype def __repr__(self): return '%s(%r)' % (self.__class__.__name__, self._prototype) def _synchronized(meth): """Call method while holding a lock.""" @functools.wraps(meth) def wrapper(self, *args, **kwargs): with self._lock: return meth(self, *args, **kwargs) return wrapper class _AutoResponder(object): def __init__(self, server, matcher, *args, **kwargs): self._server = server if inspect.isfunction(matcher) or inspect.ismethod(matcher): if args or kwargs: raise_args_err() self._matcher = Matcher() # Match anything. self._handler = matcher self._args = () self._kwargs = {} else: self._matcher = make_matcher(matcher) if args and callable(args[0]): self._handler = args[0] if args[1:] or kwargs: raise_args_err() self._args = () self._kwargs = {} else: self._handler = None self._args = args self._kwargs = kwargs def handle(self, request): if self._matcher.matches(request): if self._handler: return self._handler(request) else: # Command.replies() overrides Request.replies() with special # logic, which is why we saved args and kwargs until now to # pass it into request.replies, instead of making an OpReply # ourselves in __init__. request.replies(*self._args, **self._kwargs) return True def cancel(self): """Stop autoresponding.""" self._server.cancel_responder(self) def __repr__(self): return '_AutoResponder(%r, %r, %r)' % ( self._matcher, self._args, self._kwargs) _shutting_down = False _global_threads = weakref.WeakKeyDictionary() def _shut_down(threads): global _shutting_down _shutting_down = True for t in threads: try: t.join(10) except: pass atexit.register(_shut_down, _global_threads) class MockupDB(object): """A simulated mongod or mongos. Call `run` to start the server, and always `close` it to avoid exceptions during interpreter shutdown. See the tutorial for comprehensive examples. :Optional parameters: - `port`: listening port number. If not specified, choose some unused port and return the port number from `run`. - `verbose`: if ``True``, print requests and replies to stdout. - `request_timeout`: seconds to wait for the next client request, or else assert. Default 10 seconds. Pass int(1e6) to disable. - `auto_ismaster`: pass ``True`` to autorespond ``{'ok': 1}`` to ismaster requests, or pass a dict or `OpReply`. - `ssl`: pass ``True`` to require SSL. - `min_wire_version`: the minWireVersion to include in ismaster responses if `auto_ismaster` is True, default 0. - `max_wire_version`: the maxWireVersion to include in ismaster responses if `auto_ismaster` is True, default 6. - `uds_path`: a Unix domain socket path. MockupDB will attempt to delete the path if it already exists. """ def __init__(self, port=None, verbose=False, request_timeout=10, auto_ismaster=None, ssl=False, min_wire_version=0, max_wire_version=6, uds_path=None): if port is not None and uds_path is not None: raise TypeError( ("You can't pass port=%s and uds_path=%s," " pass only one or neither") % (port, uds_path)) self._uds_path = uds_path if uds_path: self._address = (uds_path, 0) else: self._address = ('localhost', port) self._verbose = verbose self._label = None self._ssl = ssl self._request_timeout = request_timeout self._listening_sock = None self._accept_thread = None # Track sockets that we want to close in stop(). Keys are sockets, # values are None (this could be a WeakSet but it's new in Python 2.7). self._server_threads = weakref.WeakKeyDictionary() self._server_socks = weakref.WeakKeyDictionary() self._stopped = False self._request_q = _PeekableQueue() self._requests_count = 0 self._lock = threading.Lock() # List of (request_matcher, args, kwargs), where args and kwargs are # like those sent to request.reply(). self._autoresponders = [] if auto_ismaster is True: self.autoresponds(CommandBase('ismaster'), {'ismaster': True, 'minWireVersion': min_wire_version, 'maxWireVersion': max_wire_version}) elif auto_ismaster: self.autoresponds(CommandBase('ismaster'), auto_ismaster) @_synchronized def run(self): """Begin serving. Returns the bound port, or 0 for domain socket.""" self._listening_sock, self._address = ( bind_domain_socket(self._address) if self._uds_path else bind_tcp_socket(self._address)) if self._ssl: certfile = os.path.join(os.path.dirname(__file__), 'server.pem') self._listening_sock = _ssl.wrap_socket( self._listening_sock, certfile=certfile, server_side=True) self._accept_thread = threading.Thread(target=self._accept_loop) self._accept_thread.daemon = True self._accept_thread.start() return self.port @_synchronized def stop(self): """Stop serving. Always call this to clean up after yourself.""" self._stopped = True threads = [self._accept_thread] threads.extend(self._server_threads) self._listening_sock.close() for sock in list(self._server_socks): try: sock.shutdown(socket.SHUT_RDWR) except socket.error: pass try: sock.close() except socket.error: pass with self._unlock(): for thread in threads: thread.join(10) if self._uds_path: try: os.unlink(self._uds_path) except OSError: pass def receives(self, *args, **kwargs): """Pop the next `Request` and assert it matches. Returns None if the server is stopped. Pass a `Request` or request pattern to specify what client request to expect. See the tutorial for examples. Pass ``timeout`` as a keyword argument to override this server's ``request_timeout``. """ timeout = kwargs.pop('timeout', self._request_timeout) end = time.time() + timeout matcher = Matcher(*args, **kwargs) while not self._stopped: try: # Short timeout so we notice if the server is stopped. request = self._request_q.get(timeout=0.05) except Empty: if time.time() > end: raise AssertionError('expected to receive %r, got nothing' % matcher.prototype) else: if matcher.matches(request): return request else: raise AssertionError('expected to receive %r, got %r' % (matcher.prototype, request)) gets = pop = receive = receives """Synonym for `receives`.""" def got(self, *args, **kwargs): """Does `.request` match the given :ref:`message spec `? >>> s = MockupDB(auto_ismaster=True) >>> port = s.run() >>> s.got(timeout=0) # No request enqueued. False >>> from pymongo import MongoClient >>> client = MongoClient(s.uri) >>> future = go(client.db.command, 'foo') >>> s.got('foo') True >>> s.got(OpMsg('foo', namespace='db')) True >>> s.got(OpMsg('foo', key='value')) False >>> s.ok() >>> future() == {'ok': 1} True >>> s.stop() """ timeout = kwargs.pop('timeout', self._request_timeout) end = time.time() + timeout matcher = make_matcher(*args, **kwargs) while not self._stopped: try: # Short timeout so we notice if the server is stopped. request = self._request_q.peek(timeout=timeout) except Empty: if time.time() > end: return False else: return matcher.matches(request) wait = got """Synonym for `got`.""" def replies(self, *args, **kwargs): """Call `~Request.reply` on the currently enqueued request.""" self.pop().replies(*args, **kwargs) ok = send = sends = reply = replies """Synonym for `.replies`.""" def fail(self, *args, **kwargs): """Call `~Request.fail` on the currently enqueued request.""" self.pop().fail(*args, **kwargs) def command_err(self, *args, **kwargs): """Call `~Request.command_err` on the currently enqueued request.""" self.pop().command_err(*args, **kwargs) def hangup(self): """Call `~Request.hangup` on the currently enqueued request.""" self.pop().hangup() hangs_up = hangup """Synonym for `.hangup`.""" @_synchronized def autoresponds(self, matcher, *args, **kwargs): """Send a canned reply to all matching client requests. ``matcher`` is a `Matcher` or a command name, or an instance of `OpInsert`, `OpQuery`, etc. >>> s = MockupDB() >>> port = s.run() >>> >>> from pymongo import MongoClient >>> client = MongoClient(s.uri) >>> responder = s.autoresponds('ismaster', maxWireVersion=6) >>> client.admin.command('ismaster') == {'ok': 1, 'maxWireVersion': 6} True The remaining arguments are a :ref:`message spec `: >>> # ok >>> responder = s.autoresponds('bar', ok=0, errmsg='err') >>> client.db.command('bar') Traceback (most recent call last): ... OperationFailure: command SON([('bar', 1)]) on namespace db.$cmd failed: err >>> responder = s.autoresponds(OpMsg('find', 'collection'), ... {'cursor': {'id': 0, 'firstBatch': [{'_id': 1}, {'_id': 2}]}}) >>> # ok >>> list(client.db.collection.find()) == [{'_id': 1}, {'_id': 2}] True >>> responder = s.autoresponds(OpMsg('find', 'collection'), ... {'cursor': {'id': 0, 'firstBatch': [{'a': 1}, {'a': 2}]}}) >>> # bad >>> list(client.db.collection.find()) == [{'a': 1}, {'a': 2}] True Remove an autoresponder like: >>> responder.cancel() If the request currently at the head of the queue matches, it is popped and replied to. Future matching requests skip the queue. >>> future = go(client.db.command, 'baz') >>> # bad >>> responder = s.autoresponds('baz', {'key': 'value'}) >>> future() == {'ok': 1, 'key': 'value'} True Responders are applied in order, most recently added first, until one matches: >>> responder = s.autoresponds('baz') >>> client.db.command('baz') == {'ok': 1} True >>> responder.cancel() >>> # The previous responder takes over again. >>> client.db.command('baz') == {'ok': 1, 'key': 'value'} True You can pass a request handler in place of the message spec. Return True if you handled the request: >>> responder = s.autoresponds('baz', lambda r: r.ok(a=2)) The standard `Request.ok`, `~Request.replies`, `~Request.fail`, `~Request.hangup` and so on all return True to make them suitable as handler functions. >>> client.db.command('baz') == {'ok': 1, 'a': 2} True If the request is not handled, it is checked against the remaining responders, or enqueued if none match. You can pass the handler as the only argument so it receives *all* requests. For example you could log them, then return None to allow other handlers to run: >>> def logger(request): ... if not request.matches('ismaster'): ... print('logging: %r' % request) >>> responder = s.autoresponds(logger) >>> client.db.command('baz') == {'ok': 1, 'a': 2} logging: OpMsg({"baz": 1, "$db": "db", "$readPreference": {"mode": "primaryPreferred"}}, namespace="db") True The synonym `subscribe` better expresses your intent if your handler never returns True: >>> subscriber = s.subscribe(logger) .. doctest: :hide: >>> client.close() >>> s.stop() """ responder = _AutoResponder(self, matcher, *args, **kwargs) self._autoresponders.append(responder) try: request = self._request_q.peek(block=False) except Empty: pass else: if responder.handle(request): self._request_q.get_nowait() # Pop it. return responder subscribe = autoresponds """Synonym for `.autoresponds`.""" @_synchronized def cancel_responder(self, responder): """Cancel a responder that was registered with `autoresponds`.""" self._autoresponders.remove(responder) @property def address(self): """The listening (host, port).""" return self._address @property def address_string(self): """The listening "host:port".""" return format_addr(self._address) @property def host(self): """The listening hostname.""" return self._address[0] @property def port(self): """The listening port.""" return self._address[1] @property def uri(self): """Connection string to pass to `~pymongo.mongo_client.MongoClient`.""" if self._uds_path: uri = 'mongodb://%s' % (quote_plus(self._uds_path),) else: uri = 'mongodb://%s' % (format_addr(self._address),) return uri + '/?ssl=true' if self._ssl else uri @property def verbose(self): """If verbose logging is turned on.""" return self._verbose @verbose.setter def verbose(self, value): if not isinstance(value, bool): raise TypeError('value must be True or False, not %r' % value) self._verbose = value @property def label(self): """Label for logging, or None.""" return self._label @label.setter def label(self, value): self._label = value @property def requests_count(self): """Number of requests this server has received. Includes autoresponded requests. """ return self._requests_count @property def request(self): """The currently enqueued `Request`, or None. .. warning:: This property is useful to check what the current request is, but the pattern ``server.request.replies()`` is dangerous: you must follow it with ``server.pop()`` or the current request remains enqueued. Better to reply with ``server.pop().replies()`` than ``server.request.replies()`` or any variation on it. """ return self.got() or None @property @_synchronized def running(self): """If this server is started and not stopped.""" return self._accept_thread and not self._stopped def _accept_loop(self): """Accept client connections and spawn a thread for each.""" self._listening_sock.setblocking(0) while not self._stopped and not _shutting_down: try: # Wait a short time to accept. if select.select([self._listening_sock.fileno()], [], [], 1): client, client_addr = self._listening_sock.accept() client.setblocking(True) self._log('connection from %s' % format_addr(client_addr)) server_thread = threading.Thread( target=functools.partial( self._server_loop, client, client_addr)) # Store weakrefs to the thread and socket, so we can # dispose them in stop(). self._server_threads[server_thread] = None self._server_socks[client] = None server_thread.daemon = True server_thread.start() except socket.error as error: if error.errno not in ( errno.EAGAIN, errno.EBADF, errno.EWOULDBLOCK): raise except select.error as error: if error.args[0] == errno.EBADF: # Closed. break else: raise @_synchronized def _server_loop(self, client, client_addr): """Read requests from one client socket, 'client'.""" while not self._stopped and not _shutting_down: try: with self._unlock(): request = mock_server_receive_request(client, self) self._requests_count += 1 self._log('%d\t%r' % (request.client_port, request)) # Give most recently added responders precedence. for responder in reversed(self._autoresponders): if responder.handle(request): self._log('\t(autoresponse)') break else: self._request_q.put(request) except socket.error as error: if error.errno in (errno.ECONNRESET, errno.EBADF): # We hung up, or the client did. break raise except select.error as error: if error.args[0] == errno.EBADF: # Closed. break else: raise except AssertionError: traceback.print_exc() break self._log('disconnected: %s' % format_addr(client_addr)) client.close() def _log(self, msg): if self._verbose: if self._label: msg = '%s:\t%s' % (self._label, msg) print(msg) @contextlib.contextmanager def _unlock(self): """Temporarily release the lock.""" self._lock.release() try: yield finally: self._lock.acquire() def __iter__(self): return self def next(self): request = self.receives() if request is None: # Server stopped. raise StopIteration() return request __next__ = next def __repr__(self): if self._uds_path: return 'MockupDB(uds_path=%s)' % (self._uds_path,) return 'MockupDB(%s, %s)' % self._address def format_addr(address): """Turn a TCP or Unix domain socket address into a string.""" if isinstance(address, tuple): if address[1]: return '%s:%d' % address else: return address[0] return address def bind_tcp_socket(address): """Takes (host, port) and returns (socket_object, (host, port)). If the passed-in port is None, bind an unused port and return it. """ host, port = address for res in set(socket.getaddrinfo(host, port, socket.AF_INET, socket.SOCK_STREAM, 0, socket.AI_PASSIVE)): family, socktype, proto, _, sock_addr = res sock = socket.socket(family, socktype, proto) if os.name != 'nt': sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) # Automatic port allocation with port=None. sock.bind(sock_addr) sock.listen(128) bound_port = sock.getsockname()[1] return sock, (host, bound_port) raise socket.error('could not bind socket') def bind_domain_socket(address): """Takes (socket path, 0) and returns (socket_object, (path, 0)).""" path, _ = address try: os.unlink(path) except OSError: pass sock = socket.socket(socket.AF_UNIX) sock.bind(path) sock.listen(128) return sock, (path, 0) OPCODES = {OP_MSG: OpMsg, OP_QUERY: OpQuery, OP_INSERT: OpInsert, OP_UPDATE: OpUpdate, OP_DELETE: OpDelete, OP_GET_MORE: OpGetMore, OP_KILL_CURSORS: OpKillCursors} def mock_server_receive_request(client, server): """Take a client socket and return a Request.""" header = mock_server_receive(client, 16) length = _UNPACK_INT(header[:4])[0] request_id = _UNPACK_INT(header[4:8])[0] opcode = _UNPACK_INT(header[12:])[0] msg_bytes = mock_server_receive(client, length - 16) if opcode not in OPCODES: raise NotImplementedError("Don't know how to unpack opcode %d yet" % opcode) return OPCODES[opcode].unpack(msg_bytes, client, server, request_id) def _errno_from_exception(exc): if hasattr(exc, 'errno'): return exc.errno elif exc.args: return exc.args[0] else: return None def mock_server_receive(sock, length): """Receive `length` bytes from a socket object.""" msg = b'' while length: chunk = sock.recv(length) if chunk == b'': raise socket.error(errno.ECONNRESET, 'closed') length -= len(chunk) msg += chunk return msg def make_docs(*args, **kwargs): """Make the documents for a `Request` or `Reply`. Takes a variety of argument styles, returns a list of dicts. Used by `make_prototype_request` and `make_reply`, which are in turn used by `MockupDB.receives`, `Request.replies`, and so on. See examples in tutorial. """ err_msg = "Can't interpret args: " if not args and not kwargs: return [] if not args: # OpReply(ok=1, ismaster=True). return [kwargs] if isinstance(args[0], (int, float, bool)): # server.receives().ok(0, err='uh oh'). if args[1:]: raise_args_err(err_msg, ValueError) doc = OrderedDict({'ok': args[0]}) doc.update(kwargs) return [doc] if isinstance(args[0], (list, tuple)): # Send a batch: OpReply([{'a': 1}, {'a': 2}]). if not all(isinstance(doc, (OpReply, Mapping)) for doc in args[0]): raise_args_err('each doc must be a dict:') if kwargs: raise_args_err(err_msg, ValueError) return list(args[0]) if isinstance(args[0], (string_type, text_type)): if args[2:]: raise_args_err(err_msg, ValueError) if len(args) == 2: # Command('aggregate', 'collection', {'cursor': {'batchSize': 1}}). doc = OrderedDict({args[0]: args[1]}) else: # OpReply('ismaster', me='a.com'). doc = OrderedDict({args[0]: 1}) doc.update(kwargs) return [doc] if kwargs: raise_args_err(err_msg, ValueError) # Send a batch as varargs: OpReply({'a': 1}, {'a': 2}). if not all(isinstance(doc, (OpReply, Mapping)) for doc in args): raise_args_err('each doc must be a dict') return args def make_matcher(*args, **kwargs): """Make a Matcher from a :ref:`message spec `: >>> make_matcher() Matcher(Request()) >>> make_matcher({'ismaster': 1}, namespace='admin') Matcher(Request({"ismaster": 1}, namespace="admin")) >>> make_matcher({}, {'_id': 1}) Matcher(Request({}, {"_id": 1})) See more examples in the tutorial section for :ref:`Message Specs`. """ if args and isinstance(args[0], Matcher): if args[1:] or kwargs: raise_args_err("can't interpret args") return args[0] return Matcher(*args, **kwargs) def make_prototype_request(*args, **kwargs): """Make a prototype Request for a Matcher.""" if args and inspect.isclass(args[0]) and issubclass(args[0], Request): request_cls, arg_list = args[0], args[1:] return request_cls(*arg_list, **kwargs) if args and isinstance(args[0], Request): if args[1:] or kwargs: raise_args_err("can't interpret args") return args[0] # Match any opcode. return Request(*args, **kwargs) def make_reply(*args, **kwargs): # Error we might raise. if args and isinstance(args[0], (OpReply, OpMsgReply)): if args[1:] or kwargs: raise_args_err("can't interpret args") return args[0] return OpReply(*args, **kwargs) def make_op_msg_reply(*args, **kwargs): # Error we might raise. if args and isinstance(args[0], (OpReply, OpMsgReply)): if args[1:] or kwargs: raise_args_err("can't interpret args") return args[0] return OpMsgReply(*args, **kwargs) def unprefixed(bson_str): rep = unicode(repr(bson_str)) if rep.startswith(u'u"') or rep.startswith(u"u'"): return rep[1:] else: return rep def docs_repr(*args): """Stringify ordered dicts like a regular ones. Preserve order, remove 'u'-prefix on unicodes in Python 2: >>> print(docs_repr(OrderedDict([(u'_id', 2)]))) {"_id": 2} >>> print(docs_repr(OrderedDict([(u'_id', 2), (u'a', u'b')]), ... OrderedDict([(u'a', 1)]))) {"_id": 2, "a": "b"}, {"a": 1} >>> >>> import datetime >>> now = datetime.datetime.utcfromtimestamp(123456) >>> print(docs_repr(OrderedDict([(u'ts', now)]))) {"ts": {"$date": 123456000}} >>> >>> oid = bson.ObjectId(b'123456781234567812345678') >>> print(docs_repr(OrderedDict([(u'oid', oid)]))) {"oid": {"$oid": "123456781234567812345678"}} """ sio = StringIO() for doc_idx, doc in enumerate(args): if doc_idx > 0: sio.write(u', ') sio.write(text_type(json_util.dumps(doc))) return sio.getvalue() def seq_match(seq0, seq1): """True if seq0 is a subset of seq1 and their elements are in same order. >>> seq_match([], []) True >>> seq_match([1], [1]) True >>> seq_match([1, 1], [1]) False >>> seq_match([1], [1, 2]) True >>> seq_match([1, 1], [1, 1]) True >>> seq_match([3], [1, 2, 3]) True >>> seq_match([1, 3], [1, 2, 3]) True >>> seq_match([2, 1], [1, 2, 3]) False """ len_seq1 = len(seq1) if len_seq1 < len(seq0): return False seq1_idx = 0 for i, elem in enumerate(seq0): while seq1_idx < len_seq1: if seq1[seq1_idx] == elem: break seq1_idx += 1 if seq1_idx >= len_seq1 or seq1[seq1_idx] != elem: return False seq1_idx += 1 return True def format_call(frame): fn_name = inspect.getframeinfo(frame)[2] arg_info = inspect.getargvalues(frame) args = [repr(arg_info.locals[arg]) for arg in arg_info.args] varargs = [repr(x) for x in arg_info.locals[arg_info.varargs]] kwargs = [', '.join("%s=%r" % (key, value) for key, value in arg_info.locals[arg_info.keywords].items())] return '%s(%s)' % (fn_name, ', '.join(args + varargs + kwargs)) def raise_args_err(message='bad arguments', error_class=TypeError): """Throw an error with standard message, displaying function call. >>> def f(a, *args, **kwargs): ... raise_args_err() ... >>> f(1, 2, x='y') Traceback (most recent call last): ... TypeError: bad arguments: f(1, 2, x='y') """ frame = inspect.currentframe().f_back raise error_class(message + ': ' + format_call(frame)) def interactive_server(port=27017, verbose=True, all_ok=False, name='MockupDB', ssl=False, uds_path=None): """A `MockupDB` that the mongo shell can connect to. Call `~.MockupDB.run` on the returned server, and clean it up with `~.MockupDB.stop`. If ``all_ok`` is True, replies {ok: 1} to anything unmatched by a specific responder. """ if uds_path is not None: port = None server = MockupDB(port=port, verbose=verbose, request_timeout=int(1e6), ssl=ssl, auto_ismaster=True, uds_path=uds_path) if all_ok: server.autoresponds({}) server.autoresponds('whatsmyuri', you='localhost:12345') server.autoresponds({'getLog': 'startupWarnings'}, log=['hello from %s!' % name]) server.autoresponds(OpMsg('buildInfo'), version='MockupDB ' + __version__) server.autoresponds(OpMsg('listCollections')) server.autoresponds('replSetGetStatus', ok=0) server.autoresponds('getFreeMonitoringStatus', ok=0) return server mockupdb-1.7.0/mockupdb/__main__.py0000644000076700000240000000312013242633741020726 0ustar emptysquarestaff00000000000000# -*- coding: utf-8 -*- # Copyright 2015 MongoDB, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Demonstrate a mocked MongoDB server.""" import time from mockupdb import interactive_server def main(): """Start an interactive `MockupDB`. Use like ``python -m mockupdb``. """ from optparse import OptionParser parser = OptionParser('Start mock MongoDB server') parser.add_option('-p', '--port', dest='port', default=27017, help='port on which mock mongod listens') parser.add_option('-q', '--quiet', action='store_false', dest='verbose', default=True, help="don't print messages to stdout") options, cmdline_args = parser.parse_args() if cmdline_args: parser.error('Unrecognized argument(s): %s' % ' '.join(cmdline_args)) server = interactive_server(port=options.port, verbose=options.verbose) try: server.run() print('Listening on port %d' % server.port) time.sleep(1e6) except KeyboardInterrupt: server.stop() if __name__ == '__main__': main() mockupdb-1.7.0/setup.cfg0000644000076700000240000000007513400765404016655 0ustar emptysquarestaff00000000000000[wheel] universal = 1 [egg_info] tag_build = tag_date = 0 mockupdb-1.7.0/README.rst0000664000076700000240000000024312574441203016521 0ustar emptysquarestaff00000000000000======== MockupDB ======== Mock server for testing MongoDB clients and creating MongoDB Wire Protocol servers. * Documentation: http://mockupdb.readthedocs.org/ mockupdb-1.7.0/CHANGELOG.rst0000644000076700000240000000635013400765373017064 0ustar emptysquarestaff00000000000000.. :changelog: Changelog ========= 1.7.0 (2018-12-02) ------------------ Improve datetime support in match expressions. Python datetimes have microsecond precision but BSON only has milliseconds, so expressions like this always failed:: server.receives(Command('foo', when=datetime(2018, 12, 1, 6, 6, 6, 12345))) Now, the matching logic has been rewritten to recurse through arrays and subdocuments, comparing them value by value. It compares datetime values with only millisecond precision. 1.6.0 (2018-11-16) ------------------ Remove vendored BSON library. Instead, require PyMongo and use its BSON library. This avoids surprising problems where a BSON type created with PyMongo does not appear equal to one created with MockupDB, and it avoids the occasional need to update the vendored code to support new BSON features. 1.5.0 (2018-11-02) ------------------ Support for Unix domain paths with ``uds_path`` parameter. The ``interactive_server()`` function now prepares the server to autorespond to the ``getFreeMonitoringStatus`` command from the mongo shell. 1.4.1 (2018-06-30) ------------------ Fix an inadvertent dependency on PyMongo, which broke the docs build. 1.4.0 (2018-06-29) ------------------ Support, and expect, OP_MSG requests from clients. Thanks to Shane Harvey for the contribution. Update vendored bson library from PyMongo. Support the Decimal128 BSON type. Fix Matcher so it equates BSON objects from PyMongo like ``ObjectId(...)`` with equivalent objects created from MockupDB's vendored bson library. 1.3.0 (2018-02-19) ------------------ Support Windows. Log a traceback if a bad client request causes an assert. Fix SSL. Make errors less likely on shutdown. Enable testing on Travis and Appveyor. Fix doctests and interactive server for modern MongoDB protocol. 1.2.1 (2017-12-06) ------------------ Set minWireVersion to 0, not to 2. I had been wrong about MongoDB 3.6's wire version range: it's actually 0 to 6. MockupDB now reports the same wire version range as MongoDB 3.6 by default. 1.2.0 (2017-09-22) ------------------ Update for MongoDB 3.6: report minWireVersion 2 and maxWireVersion 6 by default. 1.1.3 (2017-04-23) ------------------ Avoid rare RuntimeError in close(), if a client thread shuts down a socket as MockupDB iterates its list of sockets. 1.1.2 (2016-08-23) ------------------ Properly detect closed sockets so ``MockupDB.stop()`` doesn't take 10 seconds per connection. Thanks to Sean Purcell. 1.1.1 (2016-08-01) ------------------ Don't use "client" as a keyword arg for ``Request``, it conflicts with the actual "client" field in drivers' new handshake protocol. 1.1.0 (2016-02-11) ------------------ Add cursor_id property to OpGetMore, and ssl parameter to interactive_server. 1.0.3 (2015-09-12) ------------------ ``MockupDB(auto_ismaster=True)`` had just responded ``{"ok": 1}``, but this isn't enough to convince PyMongo 3 it's talking to a valid standalone, so auto-respond ``{"ok": 1, "ismaster": True}``. 1.0.2 (2015-09-11) ------------------ Restore Request.assert_matches method, used in pymongo-mockup-tests. 1.0.1 (2015-09-11) ------------------ Allow co-installation with PyMongo. 1.0.0 (2015-09-10) ------------------ First release. 0.1.0 (2015-02-25) ------------------ Development begun.