jsonpickle-1.2/0000775000372000037200000000000013471677155014377 5ustar travistravis00000000000000jsonpickle-1.2/COPYING0000664000372000037200000000272513471677136015437 0ustar travistravis00000000000000Copyright (C) 2008 John Paulett (john -at- paulett.org) Copyright (C) 2009-2018 David Aguilar (davvid -at- gmail.com) All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. jsonpickle-1.2/MANIFEST.in0000664000372000037200000000024113471677136016131 0ustar travistravis00000000000000include COPYING README.rst include Rakefile include *.py include jsonpickle/*.py include tests/*.py include docs/source/** include jsonpickleJS/** include *.txt jsonpickle-1.2/PKG-INFO0000664000372000037200000000215613471677155015500 0ustar travistravis00000000000000Metadata-Version: 1.1 Name: jsonpickle Version: 1.2 Summary: Python library for serializing any arbitrary object graph into JSON Home-page: https://jsonpickle.github.io/ Author: David Aguilar Author-email: davvid@gmail.com License: BSD Description: jsonpickle converts complex Python objects to and from JSON. Keywords: json pickle,json,pickle,marshal,serialization,JavaScript Object Notation Platform: POSIX Platform: Windows Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: Programming Language :: Python Classifier: Programming Language :: JavaScript jsonpickle-1.2/README.rst0000664000372000037200000000564313471677136016075 0ustar travistravis00000000000000.. image:: https://badge.fury.io/py/jsonpickle.svg :target: https://badge.fury.io/py/jsonpickle :alt: PyPI .. image:: https://img.shields.io/badge/docs-passing-green.svg :target: http://jsonpickle.github.io/ :alt: docs .. image:: https://travis-ci.org/jsonpickle/jsonpickle.svg?branch=master :target: https://travis-ci.org/jsonpickle/jsonpickle :alt: travis .. image:: https://img.shields.io/badge/License-BSD%203--Clause-blue.svg :target: https://github.com/jsonpickle/jsonpickle/blob/master/COPYING :alt: BSD jsonpickle ========== jsonpickle is a library for the two-way conversion of complex Python objects and `JSON `_. jsonpickle builds upon the existing JSON encoders, such as simplejson, json, and demjson. For complete documentation, please visit the `jsonpickle homepage `_. Bug reports and merge requests are encouraged at the `jsonpickle repository on github `_. jsonpickle supports Python 2.7 and Python 3.4 or greater. Why jsonpickle? =============== Data serialized with python's pickle (or cPickle or dill) is not easily readable outside of python. Using the json format, jsonpickle allows simple data types to be stored in a human-readable format, and more complex data types such as numpy arrays and pandas dataframes, to be machine-readable on any platform that supports json. E.g., unlike pickled data, jsonpickled data stored in an Amazon S3 bucket is indexible by Amazon's Athena. Install ======= Install from pip for the latest stable release: :: pip install jsonpickle Install from github for the latest changes: :: pip install git+https://github.com/jsonpickle/jsonpickle.git If you have the files checked out for development: :: git clone https://github.com/jsonpickle/jsonpickle.git cd jsonpickle python setup.py develop Numpy Support ============= jsonpickle includes a built-in numpy extension. If would like to encode sklearn models, numpy arrays, and other numpy-based data then you must enable the numpy extension by registering its handlers:: >>> import jsonpickle.ext.numpy as jsonpickle_numpy >>> jsonpickle_numpy.register_handlers() Pandas Support ============= jsonpickle includes a built-in pandas extension. If would like to encode pandas DataFrame or Series objects then you must enable the pandas extension by registering its handlers:: >>> import jsonpickle.ext.pandas as jsonpickle_pandas >>> jsonpickle_pandas.register_handlers() jsonpickleJS ============ `jsonpickleJS `_ is a javascript implementation of jsonpickle by Michael Scott Cuthbert. jsonpickleJS can be extremely useful for projects that have parallel data structures between Python and Javascript. License ======= Licensed under the BSD License. See COPYING for details. See jsonpickleJS/LICENSE for details about the jsonpickleJS license. jsonpickle-1.2/docs/0000775000372000037200000000000013471677155015327 5ustar travistravis00000000000000jsonpickle-1.2/docs/source/0000775000372000037200000000000013471677155016627 5ustar travistravis00000000000000jsonpickle-1.2/docs/source/api.rst0000664000372000037200000000763113471677136020140 0ustar travistravis00000000000000.. _jsonpickle-api: ============== jsonpickle API ============== .. testsetup:: * import jsonpickle import jsonpickle.pickler import jsonpickle.unpickler import jsonpickle.handlers import jsonpickle.util .. contents:: :mod:`jsonpickle` -- High Level API =================================== .. autofunction:: jsonpickle.encode .. autofunction:: jsonpickle.decode Choosing and Loading Backends ----------------------------- jsonpickle allows the user to specify what JSON backend to use when encoding and decoding. By default, jsonpickle will try to use, in the following order: :mod:`simplejson`, :mod:`json`, and :mod:`demjson`. The prefered backend can be set via :func:`jsonpickle.set_preferred_backend`. Additional JSON backends can be used via :func:`jsonpickle.load_backend`. For example, users of `Django `_ can use the version of simplejson that is bundled in Django:: jsonpickle.load_backend('django.util.simplejson', 'dumps', 'loads', ValueError)) jsonpickle.set_preferred_backend('django.util.simplejson') Supported backends: * :mod:`json` * `simplejson `_ * `demjson `_ Experimental backends: * `jsonlib `_ * yajl via `py-yajl `_ * `ujson `_ .. autofunction:: jsonpickle.set_preferred_backend .. autofunction:: jsonpickle.load_backend .. autofunction:: jsonpickle.remove_backend .. autofunction:: jsonpickle.set_encoder_options .. autofunction:: jsonpickle.set_decoder_options Customizing JSON output ----------------------- jsonpickle supports the standard :mod:`pickle` `__getstate__` and `__setstate__` protocol for representating object instances. .. method:: object.__getstate__() Classes can further influence how their instances are pickled; if the class defines the method :meth:`__getstate__`, it is called and the return state is pickled as the contents for the instance, instead of the contents of the instance's dictionary. If there is no :meth:`__getstate__` method, the instance's :attr:`__dict__` is pickled. .. method:: object.__setstate__(state) Upon unpickling, if the class also defines the method :meth:`__setstate__`, it is called with the unpickled state. If there is no :meth:`__setstate__` method, the pickled state must be a dictionary and its items are assigned to the new instance's dictionary. If a class defines both :meth:`__getstate__` and :meth:`__setstate__`, the state object needn't be a dictionary and these methods can do what they want. :mod:`jsonpickle.handlers` -- Custom Serialization Handlers ----------------------------------------------------------- The `jsonpickle.handlers` module allows plugging in custom serialization handlers at run-time. This feature is useful when jsonpickle is unable to serialize objects that are not under your direct control. .. automodule:: jsonpickle.handlers :members: :undoc-members: Low Level API ============= Typically this low level functionality is not needed by clients. Note that arguments like ``safe=True`` do not make it safe to load an untrusted jsonpickle string. :mod:`jsonpickle.pickler` -- Python to JSON-compatible dict ----------------------------------------------------------- .. automodule:: jsonpickle.pickler :members: :undoc-members: :mod:`jsonpickle.unpickler` -- JSON-compatible dict to Python ------------------------------------------------------------- .. automodule:: jsonpickle.unpickler :members: :undoc-members: :mod:`jsonpickle.backend` -- JSON Backend Management ---------------------------------------------------- .. automodule:: jsonpickle.backend :members: :mod:`jsonpickle.util` -- Helper functions ------------------------------------------ .. automodule:: jsonpickle.util :members: :undoc-members: jsonpickle-1.2/docs/source/changelog.rst0000664000372000037200000003257313471677136021321 0ustar travistravis00000000000000Change Log ========== Version 1.2 - May 15, 2019 -------------------------- * Simplified JSON representation for `__reduce__` values. (`#261 `_) * Improved Pandas support with new handlers for more Pandas data types. (`#256 `_) * Prevent stack overflows caused by bugs in user-defined `__getstate__` functions which cause infinite recursion. (`#260 `_) (`#259 `_) * Improved support for objects that contain dicts with Integer keys. Previously, jsonpickle could not restore objects that contained dicts with integer keys and provided getstate only. These objects are now handled robustly. (`#247 `_). * Support for encoding binary data in `base85`_ instead of base64 has been added on Python 3. Base85 produces payloads about 10% smaller than base64, albeit at the cost of lower throughput. For performance and backwards compatibility with Python 2 the pickler uses base64 by default, but it can be configured to use ``base85`` with the new ``use_base85`` argument. (`#251 `_). * Dynamic SQLAlchemy tables in SQLAlchemy >= 1.3 are now supported. (`#254 `_). .. _base85: https://en.wikipedia.org/wiki/Ascii85 Version 1.1 - January 22, 2019 ------------------------------ * Python 3.7 `collections.Iterator` deprecation warnings have been fixed. (`#229 `_). * Improved Pandas support for datetime and complex numbers. (`#245 `_) Version 1.0 - September 1, 2018 ------------------------------- * *NOTE* jsonpickle no longer supports Python2.6, or Python3 < 3.4. The officially supported Python versions are now 2.7 and 3.4+. * Improved Pandas and Numpy support. (`#227 `_) * Improved support for pickling iterators. (`#216 `_) * Better support for the stdlib `json` module when `simplejson` is not installed. (`#217 `_) * jsonpickle will now output python3-style module names when pickling builtins methods or functions. (`#223 `_) * jsonpickle will always flatten primitives, even when ``max_depth`` is reached, which avoids encoding unicode strings into their ``u'string'`` representation. (`#207 `_, `#180 `_, `#198 `_). * Nested classes are now supported on Python 3. (`#206 `_, `#176 `_). * Better support for older (pre-1.9) versions of numpy (`#195 `_). Version 0.9.6 - February 9, 2018 -------------------------------- * Better support for SQLAlchemy (`#180 `_). * Better support for NumPy and SciKit-Learn. (`#184 `_). * Better support for dict sub-classes (`#156 `_). Version 0.9.5 - July 16, 2017 ----------------------------- * Better support for objects that implement the reduce protocol. (`#170 `_). This backward-incompatible change removes the SimpleReduceHandler. Any projects registering that handler for a particular type should instead remove references to the handler and jsonpickle will now handle those types directly. Version 0.9.4 - January 10, 2017 -------------------------------- * Arbitrary byte streams are now better supported. (`#143 `_). * Better support for NumPy data types. The Python3 NumPy support is especially robust. * Fortran-ordered based NumPy arrays are now properly serialized. Version 0.9.3 - March 9, 2016 ----------------------------- * UUID objects can now be serialized (`#130 `_). * Added `set_decoder_options` method to allow decoder specific options equal to `set_encoder_options`. * Int keys can be encoded directly by e.g. demjson by passing `numeric_keys=True` and setting its backend options via `jsonpickle.set_encoder_options('demjson', strict=False)`. * Newer Numpy versions (v1.10+) are now supported. Version 0.9.2 - March 20, 2015 ------------------------------ * Fixes for serializing objects with custom handlers. * We now properly serialize deque objects constructed with a `maxlen` parameter. * Test suite fixes Version 0.9.1 - March 15, 2015 ------------------------------ * Support datetime objects with FixedOffsets. Version 0.9.0 - January 16, 2015 -------------------------------- * Support for Pickle Protocol v4. * We now support serializing defaultdict subclasses that use `self` as their default factory. * We now have a decorator syntax for registering custom handlers, and allow custom handlers to register themselves for all subclasses. (`#104 `_). * We now support serializing types with metaclasses and their instances (e.g., Python 3 `enum`). * We now support serializing bytestrings in both Python 2 and Python 3. In Python 2, the `str` type is decoded to UTF-8 whenever possible and serialized as a true bytestring elsewise; in Python 3, bytestrings are explicitly encoded/decoded as bytestrings. Unicode strings are always encoded as is in both Python 2 and Python 3. * Added support for serializing numpy arrays, dtypes and scalars (see `jsonpickle.ext.numpy` module). Version 0.8.0 - September 6, 2014 --------------------------------- * We now support serializing objects that contain references to module-level functions (`#77 `_). * Better Pickle Protocol v2 support (`#78 `_). * Support for string __slots__ and iterable __slots__ (`#67 `_) (`#68 `_). * `encode()` now has a `warn` option that makes jsonpickle emit warnings when encountering objects that cannot be pickled. * A Javascript implementation of jsonpickle is now included in the jsonpickleJS directory. Version 0.7.2 - August 6, 2014 ------------------------------ * We now properly serialize classes that inherit from classes that use `__slots__` and add additional slots in the derived class. * jsonpickle can now serialize objects that implement `__getstate__()` but not `__setstate__()`. The result of `__getstate__()` is returned as-is when doing a round-trip from Python objects to jsonpickle and back. * Better support for collections.defaultdict with custom factories. * Added support for `queue.Queue` objects. Version 0.7.1 - May 6, 2014 ------------------------------ * Added support for Python 3.4. * Added support for :class:`posix.stat_result`. Version 0.7.0 - March 15, 2014 ------------------------------ * Added ``handles`` decorator to :class:`jsonpickle.handlers.BaseHandler`, enabling simple declaration of a handler for a class. * `__getstate__()` and `__setstate__()` are now honored when pickling objects that subclass :class:`dict`. * jsonpickle can now serialize :class:`collections.Counter` objects. * Object references are properly handled when using integer keys. * Object references are now supported when using custom handlers. * Decimal objects are supported in Python 3. * jsonpickle's "fallthrough-on-error" behavior can now be disabled. * Simpler API for registering custom handlers. * A new "safe-mode" is provided which avoids eval(). Backwards-compatible deserialization of repr-serialized objects is disabled in this mode. e.g. `decode(string, safe=True)` Version 0.6.1 - August 25, 2013 ------------------------------- * Python 3.2 support, and additional fixes for Python 3. Version 0.6.0 - August 24, 2013 ------------------------------- * Python 3 support! * :class:`time.struct_time` is now serialized using the built-in :class:`jsonpickle.handlers.SimpleReduceHandler`. Version 0.5.0 - August 22, 2013 ------------------------------- * Non-string dictionary keys (e.g. ints, objects) are now supported by passing `keys=True` to :func:`jsonpickle.encode` and :func:`jsonpickle.decode`. * We now support namedtuple, deque, and defaultdict. * Datetimes with timezones are now fully supported. * Better support for complicated structures e.g. datetime inside dicts. * jsonpickle added support for references and cyclical data structures in 0.4.0. This can be disabled by passing `make_refs=False` to :func:`jsonpickle.encode`. Version 0.4.0 - June 21, 2011 ----------------------------- * Switch build from setuptools to distutils * Consistent dictionary key ordering * Fix areas with improper support for unpicklable=False * Added support for cyclical data structures (`#16 `_). * Experimental support for `jsonlib `_ and `py-yajl `_ backends. * New contributers David K. Hess and Alec Thomas .. warning:: To support cyclical data structures (`#16 `_), the storage format has been modified. Efforts have been made to ensure backwards-compatibility. jsonpickle 0.4.0 can read data encoded by jsonpickle 0.3.1, but earlier versions of jsonpickle may be unable to read data encoded by jsonpickle 0.4.0. Version 0.3.1 - December 12, 2009 --------------------------------- * Include tests and docs directories in sdist for distribution packages. Version 0.3.0 - December 11, 2009 --------------------------------- * Officially migrated to git from subversion. Project home now at ``_. Thanks to Michael Jone's `sphinx-to-github `_. * Fortified jsonpickle against common error conditions. * Added support for: * List and set subclasses. * Objects with module references. * Newstyle classes with `__slots__`. * Objects implementing `__setstate__()` and `__getstate__()` (follows the :mod:`pickle` protocol). * Improved support for Zope objects via pre-fetch. * Support for user-defined serialization handlers via the jsonpickle.handlers registry. * Removed cjson support per John Millikin's recommendation. * General improvements to style, including :pep:`257` compliance and refactored project layout. * Steps towards Python 2.3 and Python 3 support. * New contributors Dan Buch and Ian Schenck. * Thanks also to Kieran Darcy, Eoghan Murray, and Antonin Hildebrand for their assistance! Version 0.2.0 - January 10, 2009 -------------------------------- * Support for all major Python JSON backends (including json in Python 2.6, simplejson, cjson, and demjson) * Handle several datetime objects using the repr() of the objects (Thanks to Antonin Hildebrand). * Sphinx documentation * Added support for recursive data structures * Unicode dict-keys support * Support for Google App Engine and Django * Tons of additional testing and bug reports (Antonin Hildebrand, Sorin, Roberto Saccon, Faber Fedor, `FirePython `_, and `Joose `_) Version 0.1.0 - August 21, 2008 ------------------------------- * Added long as basic primitive (thanks Adam Fisk) * Prefer python-cjson to simplejson, if available * Major API change, use python-cjson's decode/encode instead of simplejson's load/loads/dump/dumps * Added benchmark.py to compare simplejson and python-cjson Version 0.0.5 - July 21, 2008 ----------------------------- * Changed prefix of special fields to conform with CouchDB requirements (Thanks Dean Landolt). Break backwards compatibility. * Moved to Google Code subversion * Fixed unit test imports Version 0.0.3 ------------- * Convert back to setup.py from pavement.py (issue found by spidaman) Version 0.0.2 ------------- * Handle feedparser's FeedParserDict * Converted project to Paver * Restructured directories * Increase test coverage Version 0.0.1 ------------- Initial release jsonpickle-1.2/docs/source/conf.py0000664000372000037200000001602413471677136020130 0ustar travistravis00000000000000# -*- coding: utf-8 -*- # # jsonpickle documentation build configuration file, created by # sphinx-quickstart on Fri Oct 16 23:44:35 2009. # # This file is execfile()d with the current directory set to its containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys, os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. sys.path.insert(0, os.path.abspath(os.path.join('..', '..'))) import jsonpickle sys.path.insert(1, os.path.abspath(os.path.join('..', '..', 'thirdparty', 'sphinx-to-github'))) try: import sphinxtogithub sphinxtogithub # import side-effects except ImportError, e: raise ImportError('Could not import sphinxtogithub\n' 'Is the git submodule populated at thirdparty/sphinx-to-github?\n' 'At the project root run:\n' '\tgit submodule init\n' '\tgit submodule update') # -- General configuration ----------------------------------------------------- # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.intersphinx', 'sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinxtogithub'] sphinx_to_github = True # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8' # The master toctree document. master_doc = 'index' # General information about the project. project = u'jsonpickle' copyright = u'2008-2011, John Paulett; 2009-2013, David Aguilar' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = jsonpickle.__version__ # The full version, including alpha/beta/rc tags. release = version # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of documents that shouldn't be included in the build. #unused_docs = [] # List of directories, relative to source directory, that shouldn't be searched # for source files. exclude_trees = [] # The reST default role (used for this markup: `text`) to use for all documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # -- Options for HTML output --------------------------------------------------- # The theme to use for HTML and HTML Help pages. Major themes that come with # Sphinx are currently 'default' and 'sphinxdoc'. html_theme = 'default' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = filter(os.path.exists, ['_static']) # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_use_modindex = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = '' # Output file base name for HTML help builder. htmlhelp_basename = 'jsonpickledoc' # -- Options for LaTeX output -------------------------------------------------- # The paper size ('letter' or 'a4'). #latex_paper_size = 'letter' # The font size ('10pt', '11pt' or '12pt'). #latex_font_size = '10pt' # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ ('index', 'jsonpickle.tex', u'jsonpickle Documentation', u'John Paulett', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # Additional stuff for the LaTeX preamble. #latex_preamble = '' # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_use_modindex = True # Example configuration for intersphinx: refer to the Python standard library. intersphinx_mapping = {'http://docs.python.org/': None} jsonpickle-1.2/docs/source/contrib.rst0000664000372000037200000000516113471677136021023 0ustar travistravis00000000000000========================== Contributing to jsonpickle ========================== We welcome contributions from everyone. Please fork jsonpickle on `github `_. Get the Code ============ .. _jsonpickle-contrib-checkout: :: git clone git://github.com/jsonpickle/jsonpickle.git Run the Test Suite ================== Before code is pulled into the master jsonpickle branch, all tests should pass. If you are contributing an addition or a change in behavior, we ask that you document the change in the form of test cases. The jsonpickle test suite uses several JSON encoding libraries as well as several libraries for sample objects. To simplify the process of setting up these libraries we recommend creating a virtualenv_ and using a pip_ requirements file to install the dependencies. In the base jsonpickle directory:: # create a virtualenv that is completely isolated from the # site-wide python install virtualenv --no-site-packages env # use pip to install development and testing dependencies ./env/bin/pip install --upgrade -r requirements-dev.txt To run the suite, simply invoke :file:`tests/runtests.py`:: $ ./env/bin/python tests/runtests.py test_None (util_tests.IsPrimitiveTestCase) ... ok test_bool (util_tests.IsPrimitiveTestCase) ... ok test_dict (util_tests.IsPrimitiveTestCase) ... ok test_float (util_tests.IsPrimitiveTestCase) ... ok ... .. _virtualenv: http://pypi.python.org/pypi/virtualenv .. _pip: http://pypi.python.org/pypi/pip Testing with Tox ================ jsonpickle supports many versions of Python. To make it easy to test mutiple versions of Python you should install the tox_ testing tool, e.g. on Debian:: $ sudo apt-get install tox Once tox_ is installed you can run the test suite against multiple Python interpreters:: $ make tox It is recommended that you install at least one Python2 and one Python3 interpreter for use by tox_. .. _tox: https://tox.readthedocs.io/ Generate Documentation ====================== You do not need to generate the documentation_ when contributing, though, if you are interested, you can generate the docs yourself. The following requires Sphinx_ (present if you follow the virtualenv instructions above):: # pull down the sphinx-to-github project git submodule init git submodule update cd docs make html If you wish to browse the documentation, use Python's :mod:`SimpleHTTPServer` to host them at http://localhost:8000:: cd build/html python -m SimpleHTTPServer .. _documentation: http://jsonpickle.github.com .. _Sphinx: http://sphinx.pocoo.org jsonpickle-1.2/docs/source/extensions.rst0000664000372000037200000000057613471677136021567 0ustar travistravis00000000000000===================== jsonpickle extensions ===================== NumPy ----- jsonpickle includes a built-in numpy extension. If would like to encode sklearn models, numpy arrays, and other numpy-based data then you must enable the numpy extension by registering its handlers:: >>> import jsonpickle.ext.numpy as jsonpickle_numpy >>> jsonpickle_numpy.register_handlers() jsonpickle-1.2/docs/source/index.rst0000664000372000037200000000456313471677136020477 0ustar travistravis00000000000000======================== jsonpickle Documentation ======================== `jsonpickle `_ is a Python library for serialization and deserialization of complex Python objects to and from JSON. The standard Python libraries for encoding Python into JSON, such as the stdlib's json, simplejson, and demjson, can only handle Python primitives that have a direct JSON equivalent (e.g. dicts, lists, strings, ints, etc.). jsonpickle builds on top of these libraries and allows more complex data structures to be serialized to JSON. jsonpickle is highly configurable and extendable--allowing the user to choose the JSON backend and add additional backends. .. contents:: jsonpickle Usage ================ .. automodule:: jsonpickle Download & Install ================== The easiest way to get jsonpickle is via PyPi_ with pip_:: $ pip install -U jsonpickle jsonpickle has no required dependencies (it uses the standard library's :mod:`json` module by default). You can also download or :ref:`checkout ` the latest code and install from source:: $ python setup.py install .. _PyPi: https://pypi.org/project/jsonpickle/ .. _pip: https://pypi.org/project/pip/ .. _download: https://pypi.org/project/jsonpickle/ API Reference ============= .. toctree:: api Extensions ========== .. toctree:: extensions Contributing ============ .. toctree:: contrib Contact ======= Please join our `mailing list `_. You can send email to *jsonpickle@googlegroups.com*. Check https://github.com/jsonpickle/jsonpickle for project updates. Authors ======= * John Paulett - john -at- paulett.org - https://github.com/johnpaulett * David Aguilar - davvid -at- gmail.com - https://github.com/davvid * Dan Buch - https://github.com/meatballhat * Ian Schenck - https://github.com/ianschenck * David K. Hess - https://github.com/davidkhess * Alec Thomas - https://github.com/alecthomas * jaraco - https://github.com/jaraco * Marcin Tustin - https://github.com/marcintustin Change Log ========== .. toctree:: :maxdepth: 2 changelog License ======= jsonpickle is provided under a `New BSD license `_, Copyright (C) 2008-2011 John Paulett (john -at- paulett.org) Copyright (C) 2009-2016 David Aguilar (davvid -at- gmail.com) jsonpickle-1.2/jsonpickle/0000775000372000037200000000000013471677155016540 5ustar travistravis00000000000000jsonpickle-1.2/jsonpickle/__init__.py0000664000372000037200000000503113471677136020647 0ustar travistravis00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2008 John Paulett (john -at- paulett.org) # Copyright (C) 2009, 2011, 2013 David Aguilar (davvid -at- gmail.com) # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. """Python library for serializing any arbitrary object graph into JSON. .. warning:: jsonpickle can execute arbitrary Python code. Do not load jsonpickles from untrusted / unauthenticated sources. jsonpickle can take almost any Python object and turn the object into JSON. Additionally, it can reconstitute the object back into Python. The object must be accessible globally via a module and must inherit from object (AKA new-style classes). Create an object:: class Thing(object): def __init__(self, name): self.name = name obj = Thing('Awesome') Use jsonpickle to transform the object into a JSON string:: import jsonpickle frozen = jsonpickle.encode(obj) Use jsonpickle to recreate a Python object from a JSON string:: thawed = jsonpickle.decode(frozen) The new object has the same type and data, but essentially is now a copy of the original. .. code-block:: python assert obj.name == thawed.name If you will never need to load (regenerate the Python class from JSON), you can pass in the keyword unpicklable=False to prevent extra information from being added to JSON:: oneway = jsonpickle.encode(obj, unpicklable=False) result = jsonpickle.decode(oneway) assert obj.name == result['name'] == 'Awesome' """ from __future__ import absolute_import, division, unicode_literals from .backend import json from .pickler import encode from .unpickler import decode # Export other names not in __all__ from .backend import JSONBackend # noqa: F401 from .version import __version__ # noqa: F401 from .handlers import register # noqa: F401 from .handlers import unregister # noqa: F401 from .pickler import Pickler # noqa: F401 from .unpickler import Unpickler # noqa: F401 __all__ = ('encode', 'decode') # register built-in handlers __import__('jsonpickle.handlers', level=0) # Export specific JSONPluginMgr methods into the jsonpickle namespace set_preferred_backend = json.set_preferred_backend set_decoder_options = json.set_decoder_options set_encoder_options = json.set_encoder_options load_backend = json.load_backend remove_backend = json.remove_backend enable_fallthrough = json.enable_fallthrough # json.load(), loads(), dump(), dumps() compatibility dumps = encode loads = decode jsonpickle-1.2/jsonpickle/backend.py0000664000372000037200000002305013471677136020500 0ustar travistravis00000000000000from __future__ import absolute_import, division, unicode_literals from .compat import string_types class JSONBackend(object): """Manages encoding and decoding using various backends. It tries these modules in this order: simplejson, json, demjson simplejson is a fast and popular backend and is tried first. json comes with Python and is tried second. demjson is the most permissive backend and is tried last. """ def __init__(self, fallthrough=True): # Whether we should fallthrough to the next backend self._fallthrough = fallthrough # The names of backends that have been successfully imported self._backend_names = [] # A dictionary mapping backend names to encode/decode functions self._encoders = {} self._decoders = {} # Options to pass to specific encoders json_opts = ((), {'sort_keys': True}) self._encoder_options = { 'json': json_opts, 'simplejson': json_opts, 'django.util.simplejson': json_opts, } # Options to pass to specific encoders self._decoder_options = {} # The exception class that is thrown when a decoding error occurs self._decoder_exceptions = {} # Whether we've loaded any backends successfully self._verified = False self.load_backend('simplejson') self.load_backend('json') self.load_backend('demjson', 'encode', 'decode', 'JSONDecodeError') self.load_backend('jsonlib', 'write', 'read', 'ReadError') self.load_backend('yajl') self.load_backend('ujson') def _verify(self): """Ensures that we've loaded at least one JSON backend.""" if self._verified: return raise AssertionError('jsonpickle requires at least one of the ' 'following:\n' ' python2.6, simplejson, or demjson') def enable_fallthrough(self, enable): """ Disable jsonpickle's fallthrough-on-error behavior By default, jsonpickle tries the next backend when decoding or encoding using a backend fails. This can make it difficult to force jsonpickle to use a specific backend, and catch errors, because the error will be suppressed and may not be raised by the subsequent backend. Calling `enable_backend(False)` will make jsonpickle immediately re-raise any exceptions raised by the backends. """ self._fallthrough = enable def load_backend(self, name, dumps='dumps', loads='loads', loads_exc=ValueError): """Load a JSON backend by name. This method loads a backend and sets up references to that backend's loads/dumps functions and exception classes. :param dumps: is the name of the backend's encode method. The method should take an object and return a string. Defaults to 'dumps'. :param loads: names the backend's method for the reverse operation -- returning a Python object from a string. :param loads_exc: can be either the name of the exception class used to denote decoding errors, or it can be a direct reference to the appropriate exception class itself. If it is a name, then the assumption is that an exception class of that name can be found in the backend module's namespace. :param load: names the backend's 'load' method. :param dump: names the backend's 'dump' method. :rtype bool: True on success, False if the backend could not be loaded. """ try: # Load the JSON backend mod = __import__(name) except ImportError: return False # Handle submodules, e.g. django.utils.simplejson try: for attr in name.split('.')[1:]: mod = getattr(mod, attr) except AttributeError: return False if (not self._store(self._encoders, name, mod, dumps) or not self._store(self._decoders, name, mod, loads)): return False if isinstance(loads_exc, string_types): # This backend's decoder exception is part of the backend if not self._store(self._decoder_exceptions, name, mod, loads_exc): return False else: # simplejson uses ValueError self._decoder_exceptions[name] = loads_exc # Setup the default args and kwargs for this encoder/decoder self._encoder_options.setdefault(name, ([], {})) self._decoder_options.setdefault(name, ([], {})) # Add this backend to the list of candidate backends self._backend_names.append(name) # Indicate that we successfully loaded a JSON backend self._verified = True return True def remove_backend(self, name): """Remove all entries for a particular backend.""" self._encoders.pop(name, None) self._decoders.pop(name, None) self._decoder_exceptions.pop(name, None) self._decoder_options.pop(name, None) self._encoder_options.pop(name, None) if name in self._backend_names: self._backend_names.remove(name) self._verified = bool(self._backend_names) def encode(self, obj): """ Attempt to encode an object into JSON. This tries the loaded backends in order and passes along the last exception if no backend is able to encode the object. """ self._verify() if not self._fallthrough: name = self._backend_names[0] return self.backend_encode(name, obj) for idx, name in enumerate(self._backend_names): try: return self.backend_encode(name, obj) except Exception as e: if idx == len(self._backend_names) - 1: raise e # def dumps dumps = encode def backend_encode(self, name, obj): optargs, optkwargs = self._encoder_options.get(name, ([], {})) encoder_kwargs = optkwargs.copy() encoder_args = (obj,) + tuple(optargs) return self._encoders[name](*encoder_args, **encoder_kwargs) def decode(self, string): """ Attempt to decode an object from a JSON string. This tries the loaded backends in order and passes along the last exception if no backends are able to decode the string. """ self._verify() if not self._fallthrough: name = self._backend_names[0] return self.backend_decode(name, string) for idx, name in enumerate(self._backend_names): try: return self.backend_decode(name, string) except self._decoder_exceptions[name] as e: if idx == len(self._backend_names) - 1: raise e else: pass # and try a more forgiving encoder, e.g. demjson # def loads loads = decode def backend_decode(self, name, string): optargs, optkwargs = self._decoder_options.get(name, ((), {})) decoder_kwargs = optkwargs.copy() return self._decoders[name](string, *optargs, **decoder_kwargs) def set_preferred_backend(self, name): """ Set the preferred json backend. If a preferred backend is set then jsonpickle tries to use it before any other backend. For example:: set_preferred_backend('simplejson') If the backend is not one of the built-in jsonpickle backends (json/simplejson, or demjson) then you must load the backend prior to calling set_preferred_backend. AssertionError is raised if the backend has not been loaded. """ if name in self._backend_names: self._backend_names.remove(name) self._backend_names.insert(0, name) else: errmsg = 'The "%s" backend has not been loaded.' % name raise AssertionError(errmsg) def set_encoder_options(self, name, *args, **kwargs): """ Associate encoder-specific options with an encoder. After calling set_encoder_options, any calls to jsonpickle's encode method will pass the supplied args and kwargs along to the appropriate backend's encode method. For example:: set_encoder_options('simplejson', sort_keys=True, indent=4) set_encoder_options('demjson', compactly=False) See the appropriate encoder's documentation for details about the supported arguments and keyword arguments. """ self._encoder_options[name] = (args, kwargs) def set_decoder_options(self, name, *args, **kwargs): """ Associate decoder-specific options with a decoder. After calling set_decoder_options, any calls to jsonpickle's decode method will pass the supplied args and kwargs along to the appropriate backend's decode method. For example:: set_decoder_options('simplejson', encoding='utf8', cls=JSONDecoder) set_decoder_options('demjson', strict=True) See the appropriate decoder's documentation for details about the supported arguments and keyword arguments. """ self._decoder_options[name] = (args, kwargs) def _store(self, dct, backend, obj, name): try: dct[backend] = getattr(obj, name) except AttributeError: self.remove_backend(backend) return False return True json = JSONBackend() jsonpickle-1.2/jsonpickle/compat.py0000664000372000037200000000167413471677136020404 0ustar travistravis00000000000000from __future__ import absolute_import, division, unicode_literals import sys import types import base64 PY_MAJOR = sys.version_info[0] PY2 = PY_MAJOR == 2 PY3 = PY_MAJOR == 3 class_types = (type,) iterator_types = (type(iter('')),) if PY3: import builtins import queue from base64 import encodebytes, decodebytes from collections.abc import Iterator as abc_iterator string_types = (str,) numeric_types = (int, float) ustr = str else: from collections import Iterator as abc_iterator # noqa builtins = __import__('__builtin__') class_types += (types.ClassType,) encodebytes = base64.encodestring decodebytes = base64.decodestring string_types = (builtins.basestring,) numeric_types = (int, float, builtins.long) queue = __import__('Queue') ustr = builtins.unicode def iterator(class_): if PY2 and hasattr(class_, '__next__'): class_.next = class_.__next__ return class_ jsonpickle-1.2/jsonpickle/ext/0000775000372000037200000000000013471677155017340 5ustar travistravis00000000000000jsonpickle-1.2/jsonpickle/ext/__init__.py0000664000372000037200000000000013471677136021436 0ustar travistravis00000000000000jsonpickle-1.2/jsonpickle/ext/numpy.py0000664000372000037200000003025213471677136021063 0ustar travistravis00000000000000from __future__ import absolute_import import ast import sys import zlib import warnings import json import numpy as np from ..handlers import BaseHandler, register, unregister from ..compat import numeric_types from ..util import b64decode, b64encode from .. import compat __all__ = ['register_handlers', 'unregister_handlers'] native_byteorder = '<' if sys.byteorder == 'little' else '>' def get_byteorder(arr): """translate equals sign to native order""" byteorder = arr.dtype.byteorder return native_byteorder if byteorder == '=' else byteorder class NumpyBaseHandler(BaseHandler): def flatten_dtype(self, dtype, data): if hasattr(dtype, 'tostring'): data['dtype'] = dtype.tostring() else: dtype = compat.ustr(dtype) prefix = '(numpy.record, ' if dtype.startswith(prefix): dtype = dtype[len(prefix):-1] data['dtype'] = dtype def restore_dtype(self, data): dtype = data['dtype'] if dtype.startswith(('{', '[')): dtype = ast.literal_eval(dtype) return np.dtype(dtype) class NumpyDTypeHandler(NumpyBaseHandler): def flatten(self, obj, data): self.flatten_dtype(obj, data) return data def restore(self, data): return self.restore_dtype(data) class NumpyGenericHandler(NumpyBaseHandler): def flatten(self, obj, data): self.flatten_dtype(obj.dtype.newbyteorder('N'), data) data['value'] = self.context.flatten(obj.tolist(), reset=False) return data def restore(self, data): value = self.context.restore(data['value'], reset=False) return self.restore_dtype(data).type(value) class NumpyNDArrayHandler(NumpyBaseHandler): """Stores arrays as text representation, without regard for views """ def flatten_flags(self, obj, data): if obj.flags.writeable is False: data['writeable'] = False def restore_flags(self, data, arr): if not data.get('writeable', True): arr.flags.writeable = False def flatten(self, obj, data): self.flatten_dtype(obj.dtype.newbyteorder('N'), data) self.flatten_flags(obj, data) data['values'] = self.context.flatten(obj.tolist(), reset=False) if 0 in obj.shape: # add shape information explicitly as it cannot be # inferred from an empty list data['shape'] = obj.shape return data def restore(self, data): values = self.context.restore(data['values'], reset=False) arr = np.array( values, dtype=self.restore_dtype(data), order=data.get('order', 'C') ) shape = data.get('shape', None) if shape is not None: arr = arr.reshape(shape) self.restore_flags(data, arr) return arr class NumpyNDArrayHandlerBinary(NumpyNDArrayHandler): """stores arrays with size greater than 'size_threshold' as (optionally) compressed base64 Notes ----- This would be easier to implement using np.save/np.load, but that would be less language-agnostic """ def __init__(self, size_threshold=16, compression=zlib): """ :param size_threshold: nonnegative int or None valid values for 'size_threshold' are all nonnegative integers and None if size_threshold is None, values are always stored as nested lists :param compression: a compression module or None valid values for 'compression' are {zlib, bz2, None} if compresion is None, no compression is applied """ self.size_threshold = size_threshold self.compression = compression def flatten_byteorder(self, obj, data): byteorder = obj.dtype.byteorder if byteorder != '|': data['byteorder'] = get_byteorder(obj) def restore_byteorder(self, data, arr): byteorder = data.get('byteorder', None) if byteorder: arr.dtype = arr.dtype.newbyteorder(byteorder) def flatten(self, obj, data): """encode numpy to json""" if self.size_threshold is None or self.size_threshold >= obj.size: # encode as text data = super(NumpyNDArrayHandlerBinary, self).flatten(obj, data) else: # encode as binary if obj.dtype == np.object: # There's a bug deep in the bowels of numpy that causes a # segfault when round-tripping an ndarray of dtype object. # E.g., the following will result in a segfault: # import numpy as np # arr = np.array([str(i) for i in range(3)], # dtype=np.object) # dtype = arr.dtype # shape = arr.shape # buf = arr.tobytes() # del arr # arr = np.ndarray(buffer=buf, dtype=dtype, # shape=shape).copy() # So, save as a binary-encoded list in this case buf = json.dumps(obj.tolist()).encode() elif hasattr(obj, 'tobytes'): # numpy docstring is lacking as of 1.11.2, # but this is the option we need buf = obj.tobytes(order='a') else: # numpy < 1.9 compatibility buf = obj.tostring(order='a') if self.compression: buf = self.compression.compress(buf) data['values'] = b64encode(buf) data['shape'] = obj.shape self.flatten_dtype(obj.dtype.newbyteorder('N'), data) self.flatten_byteorder(obj, data) self.flatten_flags(obj, data) if not obj.flags.c_contiguous: data['order'] = 'F' return data def restore(self, data): """decode numpy from json""" values = data['values'] if isinstance(values, list): # decode text representation arr = super(NumpyNDArrayHandlerBinary, self).restore(data) elif isinstance(values, numeric_types): # single-value array arr = np.array([values], dtype=self.restore_dtype(data)) else: # decode binary representation dtype = self.restore_dtype(data) buf = b64decode(values) if self.compression: buf = self.compression.decompress(buf) # See note above about segfault bug for numpy dtype object. Those # are saved as a list to work around that. if dtype == np.object: values = json.loads(buf.decode()) arr = np.array(values, dtype=dtype, order=data.get('order', 'C')) shape = data.get('shape', None) if shape is not None: arr = arr.reshape(shape) else: arr = np.ndarray( buffer=buf, dtype=dtype, shape=data.get('shape'), order=data.get('order', 'C') ).copy() # make a copy, to force the result to own the data self.restore_byteorder(data, arr) self.restore_flags(data, arr) return arr class NumpyNDArrayHandlerView(NumpyNDArrayHandlerBinary): """Pickles references inside ndarrays, or array-views Notes ----- The current implementation has some restrictions. 'base' arrays, or arrays which are viewed by other arrays, must be f-or-c-contiguous. This is not such a large restriction in practice, because all numpy array creation is c-contiguous by default. Relaxing this restriction would be nice though; especially if it can be done without bloating the design too much. Furthermore, ndarrays which are views of array-like objects implementing __array_interface__, but which are not themselves nd-arrays, are deepcopied with a warning (by default), as we cannot guarantee whatever custom logic such classes implement is correctly reproduced. """ def __init__(self, mode='warn', size_threshold=16, compression=zlib): """ :param mode: {'warn', 'raise', 'ignore'} How to react when encountering array-like objects whos references we cannot safely serialize :param size_threshold: nonnegative int or None valid values for 'size_threshold' are all nonnegative integers and None if size_threshold is None, values are always stored as nested lists :param compression: a compression module or None valid values for 'compression' are {zlib, bz2, None} if compresion is None, no compression is applied """ super(NumpyNDArrayHandlerView, self).__init__( size_threshold, compression) self.mode = mode def flatten(self, obj, data): """encode numpy to json""" base = obj.base if base is None and obj.flags.forc: # store by value data = super(NumpyNDArrayHandlerView, self).flatten(obj, data) # ensure that views on arrays stored as text # are interpreted correctly if not obj.flags.c_contiguous: data['order'] = 'F' elif isinstance(base, np.ndarray) and base.flags.forc: # store by reference data['base'] = self.context.flatten(base, reset=False) offset = obj.ctypes.data - base.ctypes.data if offset: data['offset'] = offset if not obj.flags.c_contiguous: data['strides'] = obj.strides data['shape'] = obj.shape self.flatten_dtype(obj.dtype.newbyteorder('N'), data) self.flatten_flags(obj, data) if get_byteorder(obj) != '|': byteorder = ( 'S' if get_byteorder(obj) != get_byteorder(base) else None) if byteorder: data['byteorder'] = byteorder if self.size_threshold is None or self.size_threshold >= obj.size: # not used in restore since base is present, but # include values for human-readability super(NumpyNDArrayHandlerBinary, self).flatten(obj, data) else: # store a deepcopy or fail if self.mode == 'warn': msg = ( "ndarray is defined by reference to an object " "we do not know how to serialize. " "A deep copy is serialized instead, breaking " "memory aliasing." ) warnings.warn(msg) elif self.mode == 'raise': msg = ( "ndarray is defined by reference to an object we do " "not know how to serialize." ) raise ValueError(msg) data = super(NumpyNDArrayHandlerView, self) \ .flatten(obj.copy(), data) return data def restore(self, data): """decode numpy from json""" base = data.get('base', None) if base is None: # decode array with owndata=True arr = super(NumpyNDArrayHandlerView, self).restore(data) else: # decode array view, which references the data of another array base = self.context.restore(base, reset=False) assert base.flags.forc, \ "Current implementation assumes base is C or F contiguous" arr = np.ndarray( buffer=base.data, dtype=self.restore_dtype(data).newbyteorder( data.get('byteorder', '|')), shape=data.get('shape'), offset=data.get('offset', 0), strides=data.get('strides', None) ) self.restore_flags(data, arr) return arr def register_handlers(): register(np.dtype, NumpyDTypeHandler, base=True) register(np.generic, NumpyGenericHandler, base=True) register(np.ndarray, NumpyNDArrayHandlerView(), base=True) def unregister_handlers(): unregister(np.dtype) unregister(np.generic) unregister(np.ndarray) jsonpickle-1.2/jsonpickle/ext/pandas.py0000664000372000037200000001536213471677136021166 0ustar travistravis00000000000000from __future__ import absolute_import import pandas as pd from io import StringIO import zlib from .. import encode, decode from ..handlers import BaseHandler, register, unregister from ..util import b64decode, b64encode from ..backend import json from .numpy import register_handlers as register_numpy_handlers, unregister_handlers as unregister_numpy_handlers __all__ = ['register_handlers', 'unregister_handlers'] class PandasProcessor(object): def __init__(self, size_threshold=500, compression=zlib): """ :param size_threshold: nonnegative int or None valid values for 'size_threshold' are all nonnegative integers and None. If size_threshold is None, dataframes are always stored as csv strings :param compression: a compression module or None valid values for 'compression' are {zlib, bz2, None} if compresion is None, no compression is applied """ self.size_threshold = size_threshold self.compression = compression def flatten_pandas(self, buf, data, meta=None): if self.size_threshold is not None and len(buf) > self.size_threshold: if self.compression: buf = self.compression.compress(buf.encode()) data['comp'] = True data['values'] = b64encode(buf) data['txt'] = False else: data['values'] = buf data['txt'] = True data['meta'] = meta return data def restore_pandas(self, data): if data.get('txt', True): # It's just text... buf = data['values'] else: buf = b64decode(data['values']) if data.get('comp', False): buf = self.compression.decompress(buf).decode() meta = data.get('meta', {}) return (buf, meta) def make_read_csv_params(meta): meta_dtypes = meta.get('dtypes', {}) parse_dates = [] converters = {} dtype = {} for k, v in meta_dtypes.items(): if v.startswith('datetime'): parse_dates.append(k) elif v.startswith('complex'): converters[k] = complex else: dtype[k] = v return dict(dtype=dtype, parse_dates=parse_dates, converters=converters) class PandasDfHandler(BaseHandler): pp = PandasProcessor() def flatten(self, obj, data): dtype = obj.dtypes.to_dict() meta = {'dtypes': {k: str(dtype[k]) for k in dtype}, 'index': encode(obj.index)} data = self.pp.flatten_pandas(obj.reset_index(drop=True).to_csv(index=False), data, meta) return data def restore(self, data): csv, meta = self.pp.restore_pandas(data) params = make_read_csv_params(meta) df = pd.read_csv(StringIO(csv), **params) df.set_index(decode(meta["index"]), inplace=True) return df class PandasSeriesHandler(BaseHandler): pp = PandasProcessor() def flatten(self, obj, data): dtypes = {k: str(pd.np.dtype(type(obj[k]))) for k in obj.keys()} meta = {'dtypes': dtypes, 'name': obj.name} # Save series as two rows rather than two cols to make preserving the # type easier. data = self.pp.flatten_pandas(obj.to_frame().T.to_csv(), data, meta) return data def restore(self, data): csv, meta = self.pp.restore_pandas(data) params = make_read_csv_params(meta) df = pd.read_csv(StringIO(csv), **params) ser = pd.Series(data=df.iloc[:, 1:].values[0], index=df.columns[1:].values, name=meta.get('name', None)) return ser class PandasIndexHandler(BaseHandler): pp = PandasProcessor() index_constructor = pd.Index name_bundler = lambda _, obj: {'name': obj.name} def flatten(self, obj, data): name_bundle = self.name_bundler(obj) meta = dict(dtype= str(obj.dtype), **name_bundle) buf = encode(obj.tolist()) data = self.pp.flatten_pandas(buf, data, meta) return data def restore(self, data): buf, meta = self.pp.restore_pandas(data) dtype = meta.get('dtype', None) name_bundle = {k: v for k, v in meta.items() if k in {'name', 'names'}} idx = self.index_constructor(decode(buf), dtype=dtype, **name_bundle) return idx class PandasPeriodIndexHandler(PandasIndexHandler): index_constructor = pd.PeriodIndex class PandasMultiIndexHandler(PandasIndexHandler): name_bundler = lambda _, obj: {'names': obj.names} class PandasTimestampHandler(BaseHandler): pp = PandasProcessor() def flatten(self, obj, data): meta = {"isoformat": obj.isoformat()} buf = "" data = self.pp.flatten_pandas(buf, data, meta) return data def restore(self, data): _, meta = self.pp.restore_pandas(data) isoformat = meta['isoformat'] obj = pd.Timestamp(isoformat) return obj class PandasPeriodHandler(BaseHandler): pp = PandasProcessor() def flatten(self, obj, data): meta = {"start_time": encode(obj.start_time), "freqstr": obj.freqstr} buf = "" data = self.pp.flatten_pandas(buf, data, meta) return data def restore(self, data): _, meta = self.pp.restore_pandas(data) start_time = decode(meta['start_time']) freqstr = meta['freqstr'] obj = pd.Period(start_time, freqstr) return obj class PandasIntervalHandler(BaseHandler): pp = PandasProcessor() def flatten(self, obj, data): meta = {"left": encode(obj.left), "right": encode(obj.right), "closed": obj.closed} buf = "" data = self.pp.flatten_pandas(buf, data, meta) return data def restore(self, data): _, meta = self.pp.restore_pandas(data) left = decode(meta['left']) right = decode(meta['right']) closed = str(meta['closed']) obj = pd.Interval(left, right, closed=closed) return obj def register_handlers(): register_numpy_handlers() register(pd.DataFrame, PandasDfHandler, base=True) register(pd.Series, PandasSeriesHandler, base=True) register(pd.Index, PandasIndexHandler, base=True) register(pd.PeriodIndex, PandasPeriodIndexHandler, base=True) register(pd.MultiIndex, PandasMultiIndexHandler, base=True) register(pd.Timestamp, PandasTimestampHandler, base=True) register(pd.Period, PandasPeriodHandler, base=True) register(pd.Interval, PandasIntervalHandler, base=True) def unregister_handlers(): unregister_numpy_handlers() unregister(pd.DataFrame) unregister(pd.Series) unregister(pd.Index) unregister(pd.PeriodIndex) unregister(pd.MultiIndex) unregister(pd.Timestamp) unregister(pd.Period) unregister(pd.Interval) jsonpickle-1.2/jsonpickle/handlers.py0000664000372000037200000001557413471677136020725 0ustar travistravis00000000000000""" Custom handlers may be created to handle other objects. Each custom handler must derive from :class:`jsonpickle.handlers.BaseHandler` and implement ``flatten`` and ``restore``. A handler can be bound to other types by calling :func:`jsonpickle.handlers.register`. """ from __future__ import absolute_import, division, unicode_literals import copy import datetime import re import uuid from . import compat from . import util class Registry(object): def __init__(self): self._handlers = {} self._base_handlers = {} def get(self, cls_or_name, default=None): """ :param cls_or_name: the type or its fully qualified name :param default: default value, if a matching handler is not found Looks up a handler by type reference or its fully qualified name. If a direct match is not found, the search is performed over all handlers registered with base=True. """ handler = self._handlers.get(cls_or_name) # attempt to find a base class if handler is None and util.is_type(cls_or_name): for cls, base_handler in self._base_handlers.items(): if issubclass(cls_or_name, cls): return base_handler return default if handler is None else handler def register(self, cls, handler=None, base=False): """Register the a custom handler for a class :param cls: The custom object class to handle :param handler: The custom handler class (if None, a decorator wrapper is returned) :param base: Indicates whether the handler should be registered for all subclasses This function can be also used as a decorator by omitting the `handler` argument:: @jsonpickle.handlers.register(Foo, base=True) class FooHandler(jsonpickle.handlers.BaseHandler): pass """ if handler is None: def _register(handler_cls): self.register(cls, handler=handler_cls, base=base) return handler_cls return _register if not util.is_type(cls): raise TypeError('{!r} is not a class/type'.format(cls)) # store both the name and the actual type for the ugly cases like # _sre.SRE_Pattern that cannot be loaded back directly self._handlers[util.importable_name(cls)] = \ self._handlers[cls] = handler if base: # only store the actual type for subclass checking self._base_handlers[cls] = handler def unregister(self, cls): self._handlers.pop(cls, None) self._handlers.pop(util.importable_name(cls), None) self._base_handlers.pop(cls, None) registry = Registry() register = registry.register unregister = registry.unregister get = registry.get class BaseHandler(object): def __init__(self, context): """ Initialize a new handler to handle a registered type. :Parameters: - `context`: reference to pickler/unpickler """ self.context = context def __call__(self, context): """This permits registering either Handler instances or classes :Parameters: - `context`: reference to pickler/unpickler """ self.context = context return self def flatten(self, obj, data): """ Flatten `obj` into a json-friendly form and write result to `data`. :param object obj: The object to be serialized. :param dict data: A partially filled dictionary which will contain the json-friendly representation of `obj` once this method has finished. """ raise NotImplementedError('You must implement flatten() in %s' % self.__class__) def restore(self, obj): """ Restore an object of the registered type from the json-friendly representation `obj` and return it. """ raise NotImplementedError('You must implement restore() in %s' % self.__class__) @classmethod def handles(self, cls): """ Register this handler for the given class. Suitable as a decorator, e.g.:: @MyCustomHandler.handles class MyCustomClass: def __reduce__(self): ... """ registry.register(cls, self) return cls class DatetimeHandler(BaseHandler): """Custom handler for datetime objects Datetime objects use __reduce__, and they generate binary strings encoding the payload. This handler encodes that payload to reconstruct the object. """ def flatten(self, obj, data): pickler = self.context if not pickler.unpicklable: return compat.ustr(obj) cls, args = obj.__reduce__() flatten = pickler.flatten payload = util.b64encode(args[0]) args = [payload] + [flatten(i, reset=False) for i in args[1:]] data['__reduce__'] = (flatten(cls, reset=False), args) return data def restore(self, data): cls, args = data['__reduce__'] unpickler = self.context restore = unpickler.restore cls = restore(cls, reset=False) value = util.b64decode(args[0]) params = (value,) + tuple([restore(i, reset=False) for i in args[1:]]) return cls.__new__(cls, *params) DatetimeHandler.handles(datetime.datetime) DatetimeHandler.handles(datetime.date) DatetimeHandler.handles(datetime.time) class RegexHandler(BaseHandler): """Flatten _sre.SRE_Pattern (compiled regex) objects""" def flatten(self, obj, data): data['pattern'] = obj.pattern return data def restore(self, data): return re.compile(data['pattern']) RegexHandler.handles(type(re.compile(''))) class QueueHandler(BaseHandler): """Opaquely serializes Queue objects Queues contains mutex and condition variables which cannot be serialized. Construct a new Queue instance when restoring. """ def flatten(self, obj, data): return data def restore(self, data): return compat.queue.Queue() QueueHandler.handles(compat.queue.Queue) class CloneFactory(object): """Serialization proxy for collections.defaultdict's default_factory""" def __init__(self, exemplar): self.exemplar = exemplar def __call__(self, clone=copy.copy): """Create new instances by making copies of the provided exemplar""" return clone(self.exemplar) def __repr__(self): return ( '' .format(id(self), self.exemplar)) class UUIDHandler(BaseHandler): """Serialize uuid.UUID objects""" def flatten(self, obj, data): data['hex'] = obj.hex return data def restore(self, data): return uuid.UUID(data['hex']) UUIDHandler.handles(uuid.UUID) jsonpickle-1.2/jsonpickle/pickler.py0000664000372000037200000005235213471677136020551 0ustar travistravis00000000000000# Copyright (C) 2008 John Paulett (john -at- paulett.org) # Copyright (C) 2009-2018 David Aguilar (davvid -at- gmail.com) # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. from __future__ import absolute_import, division, unicode_literals import decimal import warnings import sys import types from itertools import chain, islice from . import compat from . import util from . import tags from . import handlers from .backend import json from .compat import numeric_types, string_types, PY3, PY2 def encode(value, unpicklable=True, make_refs=True, keys=False, max_depth=None, reset=True, backend=None, warn=False, context=None, max_iter=None, use_decimal=False, numeric_keys=False, use_base85=False): """Return a JSON formatted representation of value, a Python object. :param unpicklable: If set to False then the output will not contain the information necessary to turn the JSON data back into Python objects, but a simpler JSON stream is produced. :param max_depth: If set to a non-negative integer then jsonpickle will not recurse deeper than 'max_depth' steps into the object. Anything deeper than 'max_depth' is represented using a Python repr() of the object. :param make_refs: If set to False jsonpickle's referencing support is disabled. Objects that are id()-identical won't be preserved across encode()/decode(), but the resulting JSON stream will be conceptually simpler. jsonpickle detects cyclical objects and will break the cycle by calling repr() instead of recursing when make_refs is set False. :param keys: If set to True then jsonpickle will encode non-string dictionary keys instead of coercing them into strings via `repr()`. :param warn: If set to True then jsonpickle will warn when it returns None for an object which it cannot pickle (e.g. file descriptors). :param max_iter: If set to a non-negative integer then jsonpickle will consume at most `max_iter` items when pickling iterators. :param use_decimal: If set to True jsonpickle will allow Decimal instances to pass-through, with the assumption that the simplejson backend will be used in `use_decimal` mode. In order to use this mode you will need to configure simplejson:: jsonpickle.set_encoder_options('simplejson', use_decimal=True, sort_keys=True) jsonpickle.set_decoder_options('simplejson', use_decimal=True) jsonpickle.set_preferred_backend('simplejson') NOTE: A side-effect of the above settings is that float values will be converted to Decimal when converting to json. >>> encode('my string') == '"my string"' True >>> encode(36) == '36' True >>> encode({'foo': True}) == '{"foo": true}' True >>> encode({'foo': [1, 2, [3, 4]]}, max_depth=1) '{"foo": "[1, 2, [3, 4]]"}' :param use_base85: If possible, use base85 to encode binary data. Base85 bloats binary data by 1/4 as opposed to base64, which expands it by 1/3. This argument is ignored on Python 2 because it doesn't support it. """ backend = backend or json context = context or Pickler( unpicklable=unpicklable, make_refs=make_refs, keys=keys, backend=backend, max_depth=max_depth, warn=warn, max_iter=max_iter, numeric_keys=numeric_keys, use_decimal=use_decimal, use_base85=use_base85) return backend.encode(context.flatten(value, reset=reset)) class Pickler(object): def __init__(self, unpicklable=True, make_refs=True, max_depth=None, backend=None, keys=False, warn=False, max_iter=None, numeric_keys=False, use_decimal=False, use_base85=False): self.unpicklable = unpicklable self.make_refs = make_refs self.backend = backend or json self.keys = keys self.warn = warn self.numeric_keys = numeric_keys self.use_base85 = use_base85 and (not PY2) # The current recursion depth self._depth = -1 # The maximal recursion depth self._max_depth = max_depth # Maps id(obj) to reference IDs self._objs = {} # Avoids garbage collection self._seen = [] # maximum amount of items to take from a pickled iterator self._max_iter = max_iter # Whether to allow decimals to pass-through self._use_decimal = use_decimal if self.use_base85: self._bytes_tag = tags.B85 self._bytes_encoder = util.b85encode else: self._bytes_tag = tags.B64 self._bytes_encoder = util.b64encode def reset(self): self._objs = {} self._depth = -1 self._seen = [] def _push(self): """Steps down one level in the namespace. """ self._depth += 1 def _pop(self, value): """Step up one level in the namespace and return the value. If we're at the root, reset the pickler's state. """ self._depth -= 1 if self._depth == -1: self.reset() return value def _log_ref(self, obj): """ Log a reference to an in-memory object. Return True if this object is new and was assigned a new ID. Otherwise return False. """ objid = id(obj) is_new = objid not in self._objs if is_new: new_id = len(self._objs) self._objs[objid] = new_id return is_new def _mkref(self, obj): """ Log a reference to an in-memory object, and return if that object should be considered newly logged. """ is_new = self._log_ref(obj) # Pretend the object is new pretend_new = not self.unpicklable or not self.make_refs return pretend_new or is_new def _getref(self, obj): return {tags.ID: self._objs.get(id(obj))} def flatten(self, obj, reset=True): """Takes an object and returns a JSON-safe representation of it. Simply returns any of the basic builtin datatypes >>> p = Pickler() >>> p.flatten('hello world') == 'hello world' True >>> p.flatten(49) 49 >>> p.flatten(350.0) 350.0 >>> p.flatten(True) True >>> p.flatten(False) False >>> r = p.flatten(None) >>> r is None True >>> p.flatten(False) False >>> p.flatten([1, 2, 3, 4]) [1, 2, 3, 4] >>> p.flatten((1,2,))[tags.TUPLE] [1, 2] >>> p.flatten({'key': 'value'}) == {'key': 'value'} True """ if reset: self.reset() return self._flatten(obj) def _flatten(self, obj): self._push() return self._pop(self._flatten_obj(obj)) def _flatten_obj(self, obj): self._seen.append(obj) max_reached = self._depth == self._max_depth in_cycle = ( max_reached or ( not self.make_refs and id(obj) in self._objs )) and not util.is_primitive(obj) if in_cycle: # break the cycle flatten_func = repr else: flatten_func = self._get_flattener(obj) if flatten_func is None: self._pickle_warning(obj) return None return flatten_func(obj) def _list_recurse(self, obj): return [self._flatten(v) for v in obj] def _get_flattener(self, obj): if PY2 and isinstance(obj, types.FileType): return self._flatten_file if util.is_bytes(obj): return self._flatten_bytestring if util.is_primitive(obj): return lambda obj: obj # Decimal is a primitive when use_decimal is True if self._use_decimal and isinstance(obj, decimal.Decimal): return lambda obj: obj list_recurse = self._list_recurse if util.is_list(obj): if self._mkref(obj): return list_recurse else: self._push() return self._getref # We handle tuples and sets by encoding them in a "(tuple|set)dict" if util.is_tuple(obj): if not self.unpicklable: return list_recurse return lambda obj: {tags.TUPLE: [self._flatten(v) for v in obj]} if util.is_set(obj): if not self.unpicklable: return list_recurse return lambda obj: {tags.SET: [self._flatten(v) for v in obj]} if util.is_dictionary(obj): return self._flatten_dict_obj if util.is_type(obj): return _mktyperef if util.is_object(obj): return self._ref_obj_instance if util.is_module_function(obj): return self._flatten_function # instance methods, lambdas, old style classes... self._pickle_warning(obj) return None def _ref_obj_instance(self, obj): """Reference an existing object or flatten if new """ if self._mkref(obj): # We've never seen this object so return its # json representation. return self._flatten_obj_instance(obj) # We've seen this object before so place an object # reference tag in the data. This avoids infinite recursion # when processing cyclical objects. return self._getref(obj) def _flatten_file(self, obj): """ Special case file objects """ assert not PY3 and isinstance(obj, types.FileType) return None def _flatten_bytestring(self, obj): if PY2: try: return obj.decode('utf-8') except UnicodeDecodeError: pass return {self._bytes_tag: self._bytes_encoder(obj)} def _flatten_obj_instance(self, obj): """Recursively flatten an instance and return a json-friendly dict """ data = {} has_class = hasattr(obj, '__class__') has_dict = hasattr(obj, '__dict__') has_slots = not has_dict and hasattr(obj, '__slots__') has_getnewargs = util.has_method(obj, '__getnewargs__') has_getnewargs_ex = util.has_method(obj, '__getnewargs_ex__') has_getinitargs = util.has_method(obj, '__getinitargs__') has_reduce, has_reduce_ex = util.has_reduce(obj) # Support objects with __getstate__(); this ensures that # both __setstate__() and __getstate__() are implemented has_getstate = hasattr(obj, '__getstate__') # not using has_method since __getstate__() is handled separately below if has_class: cls = obj.__class__ else: cls = type(obj) # Check for a custom handler class_name = util.importable_name(cls) handler = handlers.get(cls, handlers.get(class_name)) if handler is not None: if self.unpicklable: data[tags.OBJECT] = class_name return handler(self).flatten(obj, data) reduce_val = None if self.unpicklable: if has_reduce and not has_reduce_ex: try: reduce_val = obj.__reduce__() except TypeError: # A lot of builtin types have a reduce which # just raises a TypeError # we ignore those pass # test for a reduce implementation, and redirect before # doing anything else if that is what reduce requests elif has_reduce_ex: try: # we're implementing protocol 2 reduce_val = obj.__reduce_ex__(2) except TypeError: # A lot of builtin types have a reduce which # just raises a TypeError # we ignore those pass if reduce_val and isinstance(reduce_val, string_types): try: varpath = iter(reduce_val.split('.')) # curmod will be transformed by the # loop into the value to pickle curmod = sys.modules[next(varpath)] for modname in varpath: curmod = getattr(curmod, modname) # replace obj with value retrieved return self._flatten(curmod) except KeyError: # well, we can't do anything with that, so we ignore it pass elif reduce_val: # at this point, reduce_val should be some kind of iterable # pad out to len 5 rv_as_list = list(reduce_val) insufficiency = 5 - len(rv_as_list) if insufficiency: rv_as_list += [None] * insufficiency if getattr(rv_as_list[0], '__name__', '') == '__newobj__': rv_as_list[0] = tags.NEWOBJ f, args, state, listitems, dictitems = rv_as_list # check that getstate/setstate is sane if not (state and hasattr(obj, '__getstate__') and not hasattr(obj, '__setstate__') and not isinstance(obj, dict)): # turn iterators to iterables for convenient serialization if rv_as_list[3]: rv_as_list[3] = tuple(rv_as_list[3]) if rv_as_list[4]: rv_as_list[4] = tuple(rv_as_list[4]) reduce_args = list(map(self._flatten, rv_as_list)) last_index = len(reduce_args) - 1 while last_index >= 2 and reduce_args[last_index] is None: last_index -= 1 data[tags.REDUCE] = reduce_args[:last_index+1] return data if has_class and not util.is_module(obj): if self.unpicklable: data[tags.OBJECT] = class_name if has_getnewargs_ex: data[tags.NEWARGSEX] = list( map(self._flatten, obj.__getnewargs_ex__())) if has_getnewargs and not has_getnewargs_ex: data[tags.NEWARGS] = self._flatten(obj.__getnewargs__()) if has_getinitargs: data[tags.INITARGS] = self._flatten(obj.__getinitargs__()) if has_getstate: try: state = obj.__getstate__() except TypeError: # Has getstate but it cannot be called, e.g. file descriptors # in Python3 self._pickle_warning(obj) return None else: return self._getstate(state, data) if util.is_module(obj): if self.unpicklable: data[tags.REPR] = '{name}/{name}'.format(name=obj.__name__) else: data = compat.ustr(obj) return data if util.is_dictionary_subclass(obj): self._flatten_dict_obj(obj, data) return data if util.is_sequence_subclass(obj): return self._flatten_sequence_obj(obj, data) if util.is_iterator(obj): # force list in python 3 data[tags.ITERATOR] = list( map(self._flatten, islice(obj, self._max_iter))) return data if has_dict: # Support objects that subclasses list and set if util.is_sequence_subclass(obj): return self._flatten_sequence_obj(obj, data) # hack for zope persistent objects; this unghostifies the object getattr(obj, '_', None) return self._flatten_dict_obj(obj.__dict__, data) if has_slots: return self._flatten_newstyle_with_slots(obj, data) # catchall return for data created above without a return # (e.g. __getnewargs__ is not supposed to be the end of the story) if data: return data self._pickle_warning(obj) return None def _flatten_function(self, obj): if self.unpicklable: data = {tags.FUNCTION: util.importable_name(obj)} else: data = None return data def _flatten_dict_obj(self, obj, data=None): """Recursively call flatten() and return json-friendly dict """ if data is None: data = obj.__class__() flatten = self._flatten_key_value_pair for k, v in sorted(obj.items(), key=util.itemgetter): flatten(k, v, data) # the collections.defaultdict protocol if hasattr(obj, 'default_factory') and callable(obj.default_factory): factory = obj.default_factory if util.is_type(factory): # Reference the class/type value = _mktyperef(factory) else: # The factory is not a type and could reference e.g. functions # or even the object instance itself, which creates a cycle. if self._mkref(factory): # We've never seen this object before so pickle it in-place. # Create an instance from the factory and assume that the # resulting instance is a suitable examplar. value = self._flatten(handlers.CloneFactory(factory())) else: # We've seen this object before. # Break the cycle by emitting a reference. value = self._getref(factory) data['default_factory'] = value # Sub-classes of dict if hasattr(obj, '__dict__') and self.unpicklable: dict_data = {} self._flatten_dict_obj(obj.__dict__, dict_data) data['__dict__'] = dict_data return data def _flatten_obj_attrs(self, obj, attrs, data): flatten = self._flatten_key_value_pair ok = False for k in attrs: try: value = getattr(obj, k) flatten(k, value, data) except AttributeError: # The attribute may have been deleted continue ok = True return ok def _flatten_newstyle_with_slots(self, obj, data): """Return a json-friendly dict for new-style objects with __slots__. """ allslots = [_wrap_string_slot(getattr(cls, '__slots__', tuple())) for cls in obj.__class__.mro()] if not self._flatten_obj_attrs(obj, chain(*allslots), data): attrs = [x for x in dir(obj) if not x.startswith('__') and not x.endswith('__')] self._flatten_obj_attrs(obj, attrs, data) return data def _flatten_key_value_pair(self, k, v, data): """Flatten a key/value pair into the passed-in dictionary.""" if not util.is_picklable(k, v): return data if self.keys: if not isinstance(k, string_types) or k.startswith(tags.JSON_KEY): k = self._escape_key(k) else: if k is None: k = 'null' # for compatibility with common json encoders if self.numeric_keys and isinstance(k, numeric_types): pass elif not isinstance(k, string_types): try: k = repr(k) except Exception: k = compat.ustr(k) data[k] = self._flatten(v) return data def _flatten_sequence_obj(self, obj, data): """Return a json-friendly dict for a sequence subclass.""" if hasattr(obj, '__dict__'): self._flatten_dict_obj(obj.__dict__, data) value = [self._flatten(v) for v in obj] if self.unpicklable: data[tags.SEQ] = value else: return value return data def _escape_key(self, k): return tags.JSON_KEY + encode(k, reset=False, keys=True, context=self, backend=self.backend, make_refs=self.make_refs) def _getstate(self, obj, data): state = self._flatten(obj) if self.unpicklable: data[tags.STATE] = state else: data = state return data def _pickle_warning(self, obj): if self.warn: msg = 'jsonpickle cannot pickle %r: replaced with None' % obj warnings.warn(msg) def _mktyperef(obj): """Return a typeref dictionary >>> _mktyperef(AssertionError) == {'py/type': 'builtins.AssertionError'} True """ return {tags.TYPE: util.importable_name(obj)} def _wrap_string_slot(string): """Converts __slots__ = 'a' into __slots__ = ('a',) """ if isinstance(string, string_types): return (string,) return string jsonpickle-1.2/jsonpickle/tags.py0000664000372000037200000000173213471677136020052 0ustar travistravis00000000000000"""The jsonpickle.tags module provides the custom tags used for pickling and unpickling Python objects. These tags are keys into the flattened dictionaries created by the Pickler class. The Unpickler uses these custom key names to identify dictionaries that need to be specially handled. """ from __future__ import absolute_import, division, unicode_literals BYTES = 'py/bytes' B64 = 'py/b64' B85 = 'py/b85' FUNCTION = 'py/function' ID = 'py/id' INITARGS = 'py/initargs' ITERATOR = 'py/iterator' JSON_KEY = 'json://' NEWARGS = 'py/newargs' NEWARGSEX = 'py/newargsex' NEWOBJ = 'py/newobj' OBJECT = 'py/object' REDUCE = 'py/reduce' REF = 'py/ref' REPR = 'py/repr' SEQ = 'py/seq' SET = 'py/set' STATE = 'py/state' TUPLE = 'py/tuple' TYPE = 'py/type' # All reserved tag names RESERVED = { BYTES, FUNCTION, ID, INITARGS, ITERATOR, NEWARGS, NEWARGSEX, NEWOBJ, OBJECT, REDUCE, REF, REPR, SEQ, SET, STATE, TUPLE, TYPE, } jsonpickle-1.2/jsonpickle/unpickler.py0000664000372000037200000005613613471677136021120 0ustar travistravis00000000000000# Copyright (C) 2008 John Paulett (john -at- paulett.org) # Copyright (C) 2009-2018 David Aguilar (davvid -at- gmail.com) # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. from __future__ import absolute_import, division, unicode_literals import quopri import sys from . import compat from . import util from . import tags from . import handlers from .compat import numeric_types from .backend import json def decode(string, backend=None, context=None, keys=False, reset=True, safe=False, classes=None): """Convert a JSON string into a Python object. The keyword argument 'keys' defaults to False. If set to True then jsonpickle will decode non-string dictionary keys into python objects via the jsonpickle protocol. The keyword argument 'classes' defaults to None. If set to a single class, or a sequence (list, set, tuple) of classes, then the classes will be made available when constructing objects. This can be used to give jsonpickle access to local classes that are not available through the global module import scope. >>> decode('"my string"') == 'my string' True >>> decode('36') 36 """ backend = backend or json context = context or Unpickler(keys=keys, backend=backend, safe=safe) data = backend.decode(string) return context.restore(data, reset=reset, classes=classes) def _safe_hasattr(obj, attr): """Workaround unreliable hasattr() availability on sqlalchemy objects""" try: object.__getattribute__(obj, attr) return True except AttributeError: return False class _Proxy(object): """Proxies are dummy objects that are later replaced by real instances The `restore()` function has to solve a tricky problem when pickling objects with cyclical references -- the parent instance does not yet exist. The problem is that `__getnewargs__()`, `__getstate__()`, custom handlers, and cyclical objects graphs are allowed to reference the yet-to-be-created object via the referencing machinery. In other words, objects are allowed to depend on themselves for construction! We solve this problem by placing dummy Proxy objects into the referencing machinery so that we can construct the child objects before constructing the parent. Objects are initially created with Proxy attribute values instead of real references. We collect all objects that contain references to proxies and run a final sweep over them to swap in the real instance. This is done at the very end of the top-level `restore()`. The `instance` attribute below is replaced with the real instance after `__new__()` has been used to construct the object and is used when swapping proxies with real instances. """ def __init__(self): self.instance = None def get(self): return self.instance def reset(self, instance): self.instance = instance class _IDProxy(_Proxy): def __init__(self, objs, index): self._index = index self._objs = objs def get(self): return self._objs[self._index] def _obj_setattr(obj, attr, proxy): setattr(obj, attr, proxy.get()) def _obj_setvalue(obj, idx, proxy): obj[idx] = proxy.get() class Unpickler(object): def __init__(self, backend=None, keys=False, safe=False): self.backend = backend or json self.keys = keys self.safe = safe self.reset() def reset(self): """Resets the object's internal state. """ # Map reference names to object instances self._namedict = {} # The stack of names traversed for child objects self._namestack = [] # Map of objects to their index in the _objs list self._obj_to_idx = {} self._objs = [] self._proxies = [] # Extra local classes not accessible globally self._classes = {} def restore(self, obj, reset=True, classes=None): """Restores a flattened object to its original python state. Simply returns any of the basic builtin types >>> u = Unpickler() >>> u.restore('hello world') == 'hello world' True >>> u.restore({'key': 'value'}) == {'key': 'value'} True """ if reset: self.reset() if classes: self.register_classes(classes) value = self._restore(obj) if reset: self._swap_proxies() return value def register_classes(self, classes): """Register one or more classes :param classes: sequence of classes or a single class to register """ if isinstance(classes, (list, tuple, set)): for cls in classes: self.register_classes(cls) else: self._classes[util.importable_name(classes)] = classes def _swap_proxies(self): """Replace proxies with their corresponding instances""" for (obj, attr, proxy, method) in self._proxies: method(obj, attr, proxy) self._proxies = [] def _restore(self, obj): if has_tag(obj, tags.B64): restore = self._restore_base64 elif has_tag(obj, tags.B85): restore = self._restore_base85 elif has_tag(obj, tags.BYTES): # Backwards compatibility restore = self._restore_quopri elif has_tag(obj, tags.ID): restore = self._restore_id elif has_tag(obj, tags.REF): # Backwards compatibility restore = self._restore_ref elif has_tag(obj, tags.ITERATOR): restore = self._restore_iterator elif has_tag(obj, tags.TYPE): restore = self._restore_type elif has_tag(obj, tags.REPR): # Backwards compatibility restore = self._restore_repr elif has_tag(obj, tags.REDUCE): restore = self._restore_reduce elif has_tag(obj, tags.OBJECT): restore = self._restore_object elif has_tag(obj, tags.FUNCTION): restore = self._restore_function elif util.is_list(obj): restore = self._restore_list elif has_tag(obj, tags.TUPLE): restore = self._restore_tuple elif has_tag(obj, tags.SET): restore = self._restore_set elif util.is_dictionary(obj): restore = self._restore_dict else: def restore(x): return x return restore(obj) def _restore_base64(self, obj): return util.b64decode(obj[tags.B64].encode('utf-8')) def _restore_base85(self, obj): return util.b85decode(obj[tags.B85].encode('utf-8')) #: For backwards compatibility with bytes data produced by older versions def _restore_quopri(self, obj): return quopri.decodestring(obj[tags.BYTES].encode('utf-8')) def _restore_iterator(self, obj): return iter(self._restore_list(obj[tags.ITERATOR])) def _restore_reduce(self, obj): """ Supports restoring with all elements of __reduce__ as per pep 307. Assumes that iterator items (the last two) are represented as lists as per pickler implementation. """ proxy = _Proxy() self._mkref(proxy) reduce_val = list(map(self._restore, obj[tags.REDUCE])) if len(reduce_val) < 5: reduce_val.extend([None] * (5 - len(reduce_val))) f, args, state, listitems, dictitems = reduce_val if f == tags.NEWOBJ or getattr(f, '__name__', '') == '__newobj__': # mandated special case cls = args[0] if not isinstance(cls, type): cls = self._restore(cls) stage1 = cls.__new__(cls, *args[1:]) else: stage1 = f(*args) if state: try: stage1.__setstate__(state) except AttributeError: # it's fine - we'll try the prescribed default methods try: # we can't do a straight update here because we # need object identity of the state dict to be # preserved so that _swap_proxies works out for k, v in stage1.__dict__.items(): state.setdefault(k, v) stage1.__dict__ = state except AttributeError: # next prescribed default try: for k, v in state.items(): setattr(stage1, k, v) except Exception: dict_state, slots_state = state if dict_state: stage1.__dict__.update(dict_state) if slots_state: for k, v in slots_state.items(): setattr(stage1, k, v) if listitems: # should be lists if not None try: stage1.extend(listitems) except AttributeError: for x in listitems: stage1.append(x) if dictitems: for k, v in dictitems: stage1.__setitem__(k, v) proxy.reset(stage1) self._swapref(proxy, stage1) return stage1 def _restore_id(self, obj): try: idx = obj[tags.ID] return self._objs[idx] except IndexError: return _IDProxy(self._objs, idx) def _restore_ref(self, obj): return self._namedict.get(obj[tags.REF]) def _restore_type(self, obj): typeref = loadclass(obj[tags.TYPE], classes=self._classes) if typeref is None: return obj return typeref def _restore_repr(self, obj): if self.safe: # eval() is not allowed in safe mode return None obj = loadrepr(obj[tags.REPR]) return self._mkref(obj) def _restore_object(self, obj): class_name = obj[tags.OBJECT] cls = loadclass(class_name, classes=self._classes) handler = handlers.get(cls, handlers.get(class_name)) if handler is not None: # custom handler proxy = _Proxy() self._mkref(proxy) instance = handler(self).restore(obj) proxy.reset(instance) self._swapref(proxy, instance) return instance if cls is None: return self._mkref(obj) return self._restore_object_instance(obj, cls) def _restore_function(self, obj): return loadclass(obj[tags.FUNCTION], classes=self._classes) def _loadfactory(self, obj): try: default_factory = obj['default_factory'] except KeyError: return None del obj['default_factory'] return self._restore(default_factory) def _restore_object_instance(self, obj, cls): # This is a placeholder proxy object which allows child objects to # reference the parent object before it has been instantiated. proxy = _Proxy() self._mkref(proxy) # An object can install itself as its own factory, so load the factory # after the instance is available for referencing. factory = self._loadfactory(obj) if has_tag(obj, tags.NEWARGSEX): args, kwargs = obj[tags.NEWARGSEX] else: args = getargs(obj, classes=self._classes) kwargs = {} if args: args = self._restore(args) if kwargs: kwargs = self._restore(kwargs) is_oldstyle = ( not (isinstance(cls, type) or getattr(cls, '__meta__', None))) try: if (not is_oldstyle) and hasattr(cls, '__new__'): # new style classes if factory: instance = cls.__new__(cls, factory, *args, **kwargs) instance.default_factory = factory else: instance = cls.__new__(cls, *args, **kwargs) else: instance = object.__new__(cls) except TypeError: # old-style classes is_oldstyle = True if is_oldstyle: try: instance = cls(*args) except TypeError: # fail gracefully try: instance = make_blank_classic(cls) except Exception: # fail gracefully return self._mkref(obj) proxy.reset(instance) self._swapref(proxy, instance) if isinstance(instance, tuple): return instance if (_safe_hasattr(instance, 'default_factory') and isinstance(instance.default_factory, _Proxy)): instance.default_factory = instance.default_factory.get() return self._restore_object_instance_variables(obj, instance) def _restore_from_dict(self, obj, instance, ignorereserved=True): restore_key = self._restore_key_fn() method = _obj_setattr deferred = {} for k, v in sorted(obj.items(), key=util.itemgetter): # ignore the reserved attribute if ignorereserved and k in tags.RESERVED: continue if isinstance(k, numeric_types): str_k = k.__str__() else: str_k = k self._namestack.append(str_k) k = restore_key(k) # step into the namespace value = self._restore(v) if (util.is_noncomplex(instance) or util.is_dictionary_subclass(instance)): try: if k == '__dict__': setattr(instance, k, value) else: instance[k] = value except TypeError: # Immutable object, must be constructed in one shot if k != '__dict__': deferred[k] = value self._namestack.pop() continue else: setattr(instance, k, value) # This instance has an instance variable named `k` that is # currently a proxy and must be replaced if isinstance(value, _Proxy): self._proxies.append((instance, k, value, method)) # step out self._namestack.pop() if deferred: # SQLAlchemy Immutable mappings must be constructed in one shot instance = instance.__class__(deferred) return instance def _restore_object_instance_variables(self, obj, instance): instance = self._restore_from_dict(obj, instance) # Handle list and set subclasses if has_tag(obj, tags.SEQ): if hasattr(instance, 'append'): for v in obj[tags.SEQ]: instance.append(self._restore(v)) if hasattr(instance, 'add'): for v in obj[tags.SEQ]: instance.add(self._restore(v)) if has_tag(obj, tags.STATE): instance = self._restore_state(obj, instance) return instance def _restore_state(self, obj, instance): state = self._restore(obj[tags.STATE]) has_slots = (isinstance(state, tuple) and len(state) == 2 and isinstance(state[1], dict)) has_slots_and_dict = has_slots and isinstance(state[0], dict) if hasattr(instance, '__setstate__'): instance.__setstate__(state) elif isinstance(state, dict): # implements described default handling # of state for object with instance dict # and no slots instance = self._restore_from_dict( state, instance, ignorereserved=False) elif has_slots: instance = self._restore_from_dict( state[1], instance, ignorereserved=False) if has_slots_and_dict: instance = self._restore_from_dict( state[0], instance, ignorereserved=False) elif (not hasattr(instance, '__getnewargs__') and not hasattr(instance, '__getnewargs_ex__')): # __setstate__ is not implemented so that means that the best # we can do is return the result of __getstate__() rather than # return an empty shell of an object. # However, if there were newargs, it's not an empty shell instance = state return instance def _restore_list(self, obj): parent = [] self._mkref(parent) children = [self._restore(v) for v in obj] parent.extend(children) method = _obj_setvalue proxies = [ (parent, idx, value, method) for idx, value in enumerate(parent) if isinstance(value, _Proxy) ] self._proxies.extend(proxies) return parent def _restore_tuple(self, obj): return tuple([self._restore(v) for v in obj[tags.TUPLE]]) def _restore_set(self, obj): return {self._restore(v) for v in obj[tags.SET]} def _restore_dict(self, obj): data = {} restore_key = self._restore_key_fn() for k, v in sorted(obj.items(), key=util.itemgetter): if isinstance(k, numeric_types): str_k = k.__str__() else: str_k = k self._namestack.append(str_k) k = restore_key(k) data[k] = self._restore(v) # k is currently a proxy and must be replaced if isinstance(data[k], _Proxy): self._proxies.append((data, k, data[k], _obj_setvalue)) self._namestack.pop() return data def _restore_key_fn(self): """Return a callable that restores keys This function is responsible for restoring non-string keys when we are decoding with `keys=True`. """ # This function is called before entering a tight loop # where the returned function will be called. # We return a specific function after checking self.keys # instead of doing so in the body of the function to # avoid conditional branching inside a tight loop. if self.keys: restore_key = self._restore_pickled_key else: def restore_key(key): return key return restore_key def _restore_pickled_key(self, key): """Restore a possibly pickled key""" if (isinstance(key, compat.string_types) and key.startswith(tags.JSON_KEY)): key = decode(key[len(tags.JSON_KEY):], backend=self.backend, context=self, keys=True, reset=False) return key def _refname(self): """Calculates the name of the current location in the JSON stack. This is called as jsonpickle traverses the object structure to create references to previously-traversed objects. This allows cyclical data structures such as doubly-linked lists. jsonpickle ensures that duplicate python references to the same object results in only a single JSON object definition and special reference tags to represent each reference. >>> u = Unpickler() >>> u._namestack = [] >>> u._refname() == '/' True >>> u._namestack = ['a'] >>> u._refname() == '/a' True >>> u._namestack = ['a', 'b'] >>> u._refname() == '/a/b' True """ return '/' + '/'.join(self._namestack) def _mkref(self, obj): obj_id = id(obj) try: self._obj_to_idx[obj_id] except KeyError: self._obj_to_idx[obj_id] = len(self._objs) self._objs.append(obj) # Backwards compatibility: old versions of jsonpickle # produced "py/ref" references. self._namedict[self._refname()] = obj return obj def _swapref(self, proxy, instance): proxy_id = id(proxy) instance_id = id(instance) instance_index = self._obj_to_idx[proxy_id] self._obj_to_idx[instance_id] = instance_index del self._obj_to_idx[proxy_id] self._objs[instance_index] = instance self._namedict[self._refname()] = instance def loadclass(module_and_name, classes=None): """Loads the module and returns the class. >>> cls = loadclass('datetime.datetime') >>> cls.__name__ 'datetime' >>> loadclass('does.not.exist') >>> loadclass('builtins.int')() 0 """ # Check if the class exists in a caller-provided scope if classes: try: return classes[module_and_name] except KeyError: pass # Otherwise, load classes from globally-accessible imports names = module_and_name.split('.') # First assume that everything up to the last dot is the module name, # then try other splits to handle classes that are defined within # classes for up_to in range(len(names)-1, 0, -1): module = util.untranslate_module_name('.'.join(names[:up_to])) try: __import__(module) obj = sys.modules[module] for class_name in names[up_to:]: obj = getattr(obj, class_name) return obj except (AttributeError, ImportError, ValueError) as ex: continue return None def getargs(obj, classes=None): """Return arguments suitable for __new__()""" # Let saved newargs take precedence over everything if has_tag(obj, tags.NEWARGSEX): raise ValueError("__newargs_ex__ returns both args and kwargs") if has_tag(obj, tags.NEWARGS): return obj[tags.NEWARGS] if has_tag(obj, tags.INITARGS): return obj[tags.INITARGS] try: seq_list = obj[tags.SEQ] obj_dict = obj[tags.OBJECT] except KeyError: return [] typeref = loadclass(obj_dict, classes=classes) if not typeref: return [] if hasattr(typeref, '_fields'): if len(typeref._fields) == len(seq_list): return seq_list return [] class _trivialclassic: """ A trivial class that can be instantiated with no args """ def make_blank_classic(cls): """ Implement the mandated strategy for dealing with classic classes which cannot be instantiated without __getinitargs__ because they take parameters """ instance = _trivialclassic() instance.__class__ = cls return instance def loadrepr(reprstr): """Returns an instance of the object from the object's repr() string. It involves the dynamic specification of code. >>> obj = loadrepr('datetime/datetime.datetime.now()') >>> obj.__class__.__name__ 'datetime' """ module, evalstr = reprstr.split('/') mylocals = locals() localname = module if '.' in localname: localname = module.split('.', 1)[0] mylocals[localname] = __import__(module) return eval(evalstr) def has_tag(obj, tag): """Helper class that tests to see if the obj is a dictionary and contains a particular key/tag. >>> obj = {'test': 1} >>> has_tag(obj, 'test') True >>> has_tag(obj, 'fail') False >>> has_tag(42, 'fail') False """ return type(obj) is dict and tag in obj jsonpickle-1.2/jsonpickle/util.py0000664000372000037200000003272613471677136020100 0ustar travistravis00000000000000# Copyright (C) 2008 John Paulett (john -at- paulett.org) # Copyright (C) 2009-2018 David Aguilar (davvid -at- gmail.com) # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. """Helper functions for pickling and unpickling. Most functions assist in determining the type of an object. """ from __future__ import absolute_import, division, unicode_literals import base64 import collections import io import operator import time import types import inspect from . import tags from . import compat from .compat import numeric_types, PY2, PY3, class_types from .compat import abc_iterator, iterator_types if PY2: import __builtin__ SEQUENCES = (list, set, tuple) PRIMITIVES = (compat.ustr, bool, type(None)) + numeric_types def is_type(obj): """Returns True is obj is a reference to a type. >>> is_type(1) False >>> is_type(object) True >>> class Klass: pass >>> is_type(Klass) True """ # use "isinstance" and not "is" to allow for metaclasses return isinstance(obj, class_types) def has_method(obj, name): # false if attribute doesn't exist if not hasattr(obj, name): return False func = getattr(obj, name) # builtin descriptors like __getnewargs__ if isinstance(func, types.BuiltinMethodType): return True # note that FunctionType has a different meaning in py2/py3 if not isinstance(func, (types.MethodType, types.FunctionType)): return False # need to go through __dict__'s since in py3 # methods are essentially descriptors # __class__ for old-style classes base_type = obj if is_type(obj) else obj.__class__ original = None # there is no .mro() for old-style classes for subtype in inspect.getmro(base_type): original = vars(subtype).get(name) if original is not None: break # name not found in the mro if original is None: return False # static methods are always fine if isinstance(original, staticmethod): return True # at this point, the method has to be an instancemthod or a classmethod self_attr = '__self__' if PY3 else 'im_self' if not hasattr(func, self_attr): return False bound_to = getattr(func, self_attr) # class methods if isinstance(original, classmethod): return issubclass(base_type, bound_to) # bound methods return isinstance(obj, type(bound_to)) def is_object(obj): """Returns True is obj is a reference to an object instance. >>> is_object(1) True >>> is_object(object()) True >>> is_object(lambda x: 1) False """ return (isinstance(obj, object) and not isinstance(obj, (type, types.FunctionType, types.BuiltinFunctionType))) def is_primitive(obj): """Helper method to see if the object is a basic data type. Unicode strings, integers, longs, floats, booleans, and None are considered primitive and will return True when passed into *is_primitive()* >>> is_primitive(3) True >>> is_primitive([4,4]) False """ return type(obj) in PRIMITIVES def is_dictionary(obj): """Helper method for testing if the object is a dictionary. >>> is_dictionary({'key':'value'}) True """ return type(obj) is dict def is_sequence(obj): """Helper method to see if the object is a sequence (list, set, or tuple). >>> is_sequence([4]) True """ return type(obj) in SEQUENCES def is_list(obj): """Helper method to see if the object is a Python list. >>> is_list([4]) True """ return type(obj) is list def is_set(obj): """Helper method to see if the object is a Python set. >>> is_set(set()) True """ return type(obj) is set def is_bytes(obj): """Helper method to see if the object is a bytestring. >>> is_bytes(b'foo') True """ return type(obj) is bytes def is_unicode(obj): """Helper method to see if the object is a unicode string""" return type(obj) is compat.ustr def is_tuple(obj): """Helper method to see if the object is a Python tuple. >>> is_tuple((1,)) True """ return type(obj) is tuple def is_dictionary_subclass(obj): """Returns True if *obj* is a subclass of the dict type. *obj* must be a subclass and not the actual builtin dict. >>> class Temp(dict): pass >>> is_dictionary_subclass(Temp()) True """ # TODO: add UserDict return (hasattr(obj, '__class__') and issubclass(obj.__class__, dict) and type(obj) is not dict) def is_sequence_subclass(obj): """Returns True if *obj* is a subclass of list, set or tuple. *obj* must be a subclass and not the actual builtin, such as list, set, tuple, etc.. >>> class Temp(list): pass >>> is_sequence_subclass(Temp()) True """ return (hasattr(obj, '__class__') and ( issubclass(obj.__class__, SEQUENCES) or is_list_like(obj) ) and not is_sequence(obj)) def is_noncomplex(obj): """Returns True if *obj* is a special (weird) class, that is more complex than primitive data types, but is not a full object. Including: * :class:`~time.struct_time` """ if type(obj) is time.struct_time: return True return False def is_function(obj): """Returns true if passed a function >>> is_function(lambda x: 1) True >>> is_function(locals) True >>> def method(): pass >>> is_function(method) True >>> is_function(1) False """ function_types = ( types.FunctionType, types.MethodType, types.LambdaType, types.BuiltinFunctionType, types.BuiltinMethodType, ) return type(obj) in function_types def is_module_function(obj): """Return True if `obj` is a module-global function >>> import os >>> is_module_function(os.path.exists) True >>> is_module_function(lambda: None) False """ return (hasattr(obj, '__class__') and isinstance(obj, (types.FunctionType, types.BuiltinFunctionType)) and hasattr(obj, '__module__') and hasattr(obj, '__name__') and obj.__name__ != '') def is_module(obj): """Returns True if passed a module >>> import os >>> is_module(os) True """ return isinstance(obj, types.ModuleType) def is_picklable(name, value): """Return True if an object can be pickled >>> import os >>> is_picklable('os', os) True >>> def foo(): pass >>> is_picklable('foo', foo) True >>> is_picklable('foo', lambda: None) False """ if name in tags.RESERVED: return False return is_module_function(value) or not is_function(value) def is_installed(module): """Tests to see if ``module`` is available on the sys.path >>> is_installed('sys') True >>> is_installed('hopefullythisisnotarealmodule') False """ try: __import__(module) return True except ImportError: return False def is_list_like(obj): return hasattr(obj, '__getitem__') and hasattr(obj, 'append') def is_iterator(obj): is_file = PY2 and isinstance(obj, __builtin__.file) return (isinstance(obj, abc_iterator) and not isinstance(obj, io.IOBase) and not is_file) def is_collections(obj): try: return type(obj).__module__ == 'collections' except Exception: return False def is_reducible(obj): """ Returns false if of a type which have special casing, and should not have their __reduce__ methods used """ # defaultdicts may contain functions which we cannot serialise if is_collections(obj) and not isinstance(obj, collections.defaultdict): return True return (not (is_list(obj) or is_list_like(obj) or is_primitive(obj) or is_bytes(obj) or is_unicode(obj) or is_dictionary(obj) or is_sequence(obj) or is_set(obj) or is_tuple(obj) or is_dictionary_subclass(obj) or is_sequence_subclass(obj) or is_function(obj) or is_module(obj) or isinstance(getattr(obj, '__slots__', None), iterator_types) or type(obj) is object or obj is object or (is_type(obj) and obj.__module__ == 'datetime') )) def in_dict(obj, key, default=False): """ Returns true if key exists in obj.__dict__; false if not in. If obj.__dict__ is absent, return default """ return (key in obj.__dict__) if getattr(obj, '__dict__', None) else default def in_slots(obj, key, default=False): """ Returns true if key exists in obj.__slots__; false if not in. If obj.__slots__ is absent, return default """ return ( (key in obj.__slots__) if getattr(obj, '__slots__', None) else default ) def has_reduce(obj): """ Tests if __reduce__ or __reduce_ex__ exists in the object dict or in the class dicts of every class in the MRO *except object*. Returns a tuple of booleans (has_reduce, has_reduce_ex) """ if not is_reducible(obj) or is_type(obj): return (False, False) # in this case, reduce works and is desired # notwithstanding depending on default object # reduce if is_noncomplex(obj): return (False, True) has_reduce = False has_reduce_ex = False REDUCE = '__reduce__' REDUCE_EX = '__reduce_ex__' # For object instance has_reduce = in_dict(obj, REDUCE) or in_slots(obj, REDUCE) has_reduce_ex = in_dict(obj, REDUCE_EX) or in_slots(obj, REDUCE_EX) # turn to the MRO for base in type(obj).__mro__: if is_reducible(base): has_reduce = has_reduce or in_dict(base, REDUCE) has_reduce_ex = has_reduce_ex or in_dict(base, REDUCE_EX) if has_reduce and has_reduce_ex: return (has_reduce, has_reduce_ex) # for things that don't have a proper dict but can be # getattred (rare, but includes some builtins) cls = type(obj) object_reduce = getattr(object, REDUCE) object_reduce_ex = getattr(object, REDUCE_EX) if not has_reduce: has_reduce_cls = getattr(cls, REDUCE, False) if has_reduce_cls is not object_reduce: has_reduce = has_reduce_cls if not has_reduce_ex: has_reduce_ex_cls = getattr(cls, REDUCE_EX, False) if has_reduce_ex_cls is not object_reduce_ex: has_reduce_ex = has_reduce_ex_cls return (has_reduce, has_reduce_ex) def translate_module_name(module): """Rename builtin modules to a consistent module name. Prefer the more modern naming. This is used so that references to Python's `builtins` module can be loaded in both Python 2 and 3. We remap to the "__builtin__" name and unmap it when importing. Map the Python2 `exceptions` module to `builtins` because `builtins` is a superset and contains everything that is available in `exceptions`, which makes the translation simpler. See untranslate_module_name() for the reverse operation. """ lookup = dict(__builtin__='builtins', exceptions='builtins') return lookup.get(module, module) def untranslate_module_name(module): """Rename module names mention in JSON to names that we can import This reverses the translation applied by translate_module_name() to a module name available to the current version of Python. """ module = _0_9_6_compat_untranslate(module) lookup = dict(builtins='__builtin__') if PY2 else {} return lookup.get(module, module) def _0_9_6_compat_untranslate(module): """Provide compatibility for pickles created with jsonpickle 0.9.6 and earlier, remapping `exceptions` and `__builtin__` to `builtins`. """ lookup = dict(__builtin__='builtins', exceptions='builtins') return lookup.get(module, module) def importable_name(cls): """ >>> class Example(object): ... pass >>> ex = Example() >>> importable_name(ex.__class__) == 'jsonpickle.util.Example' True >>> importable_name(type(25)) == 'builtins.int' True >>> importable_name(None.__class__) == 'builtins.NoneType' True >>> importable_name(False.__class__) == 'builtins.bool' True >>> importable_name(AttributeError) == 'builtins.AttributeError' True """ # Use the fully-qualified name if available (Python >= 3.3) name = getattr(cls, '__qualname__', cls.__name__) module = translate_module_name(cls.__module__) return '{}.{}'.format(module, name) def b64encode(data): """ Encode binary data to ascii text in base64. Data must be bytes. """ return base64.b64encode(data).decode('ascii') def b64decode(payload): """ Decode payload - must be ascii text. """ return base64.b64decode(payload) def b85encode(data): """ Encode binary data to ascii text in base85. Data must be bytes. """ if PY2: raise NotImplementedError("Python 2 can't encode data in base85.") return base64.b85encode(data).decode('ascii') def b85decode(payload): """ Decode payload - must be ascii text. """ if PY2: raise NotImplementedError("Python 2 can't decode base85-encoded data.") return base64.b85decode(payload) def itemgetter(obj, getter=operator.itemgetter(0)): return compat.ustr(getter(obj)) jsonpickle-1.2/jsonpickle/version.py0000664000372000037200000000002413471677136020572 0ustar travistravis00000000000000__version__ = '1.2' jsonpickle-1.2/jsonpickle.egg-info/0000775000372000037200000000000013471677155020232 5ustar travistravis00000000000000jsonpickle-1.2/jsonpickle.egg-info/PKG-INFO0000664000372000037200000000215613471677154021332 0ustar travistravis00000000000000Metadata-Version: 1.1 Name: jsonpickle Version: 1.2 Summary: Python library for serializing any arbitrary object graph into JSON Home-page: https://jsonpickle.github.io/ Author: David Aguilar Author-email: davvid@gmail.com License: BSD Description: jsonpickle converts complex Python objects to and from JSON. Keywords: json pickle,json,pickle,marshal,serialization,JavaScript Object Notation Platform: POSIX Platform: Windows Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: Programming Language :: Python Classifier: Programming Language :: JavaScript jsonpickle-1.2/jsonpickle.egg-info/SOURCES.txt0000664000372000037200000000233113471677154022114 0ustar travistravis00000000000000COPYING MANIFEST.in README.rst requirements-dev.txt setup.cfg setup.py docs/source/api.rst docs/source/changelog.rst docs/source/conf.py docs/source/contrib.rst docs/source/extensions.rst docs/source/index.rst jsonpickle/__init__.py jsonpickle/backend.py jsonpickle/compat.py jsonpickle/handlers.py jsonpickle/pickler.py jsonpickle/tags.py jsonpickle/unpickler.py jsonpickle/util.py jsonpickle/version.py jsonpickle.egg-info/PKG-INFO jsonpickle.egg-info/SOURCES.txt jsonpickle.egg-info/dependency_links.txt jsonpickle.egg-info/top_level.txt jsonpickle/ext/__init__.py jsonpickle/ext/numpy.py jsonpickle/ext/pandas.py jsonpickleJS/.gitignore jsonpickleJS/LICENSE jsonpickleJS/README.md jsonpickleJS/handlers.js jsonpickleJS/main.js jsonpickleJS/pickler.js jsonpickleJS/tags.js jsonpickleJS/testPickle.html jsonpickleJS/testUnpickle.html jsonpickleJS/unpickler.js jsonpickleJS/util.js tests/backend_test.py tests/benchmark.py tests/bson_test.py tests/datetime_test.py tests/document_test.py tests/ecdsa_test.py tests/feedparser_test.py tests/handler_test.py tests/helper.py tests/jsonpickle_test.py tests/numpy_test.py tests/object_test.py tests/pandas_test.py tests/runtests.py tests/sqlalchemy_test.py tests/stdlib_test.py tests/util_test.pyjsonpickle-1.2/jsonpickle.egg-info/dependency_links.txt0000664000372000037200000000000113471677154024277 0ustar travistravis00000000000000 jsonpickle-1.2/jsonpickle.egg-info/top_level.txt0000664000372000037200000000001313471677154022755 0ustar travistravis00000000000000jsonpickle jsonpickle-1.2/jsonpickleJS/0000775000372000037200000000000013471677155016775 5ustar travistravis00000000000000jsonpickle-1.2/jsonpickleJS/.gitignore0000664000372000037200000000054213471677136020765 0ustar travistravis00000000000000*.py[cod] # OSX .DS_Store # C extensions *.so # Packages *.egg *.egg-info dist build eggs parts bin var sdist develop-eggs .installed.cfg lib lib64 __pycache__ # Installer logs pip-log.txt # Unit test / coverage reports .coverage .tox nosetests.xml # Translations *.mo # Mr Developer .mr.developer.cfg # Eclipse .project .pydevproject /.settings jsonpickle-1.2/jsonpickleJS/LICENSE0000664000372000037200000000301613471677136020001 0ustar travistravis00000000000000Copyright (c) 2013-14, Michael Scott Cuthbert and cuthbertLab All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of Michael Scott Cuthbert/cuthbertLab nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL Cuthbert OR cuthbertLab BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. jsonpickle-1.2/jsonpickleJS/README.md0000664000372000037200000001110213471677136020246 0ustar travistravis00000000000000jsonpickleJS ============ Javascript reinterpretation of Python jsonpickle to allow reading and (to a lesser extent) writing JSON objects Copyright (c) 2014 Michael Scott Cuthbert and cuthbertLab. Released under the BSD (3-clause) license. See LICENSE. Python to Javascript and Back ============================== Python has a remarkable number of ways (for a language that believes there's only one way to do it) to transfer data between itself and Javascript, most obviously with the ``json.dump()``/``json.dumps()`` calls, which work well on specifically created data, especially of primitive objects. It also has a good number of ways to store the contents of an object so that it could be reloaded later into the same Python interpreter/session. The most well known of these is the ``pickle`` module, which stores the Python data as binary data. The external ``jsonpickle`` module, which you'll need to have installed to make this module have any sense, combines the features of ``json`` and ``pickle``, storing Python data in a JSON (Javascript Object Notation) string, via ``jsonpickle.encode()`` that can be parsed back by ``jsonpickle.decode()`` to get nearly all data types restored to their previous state. (Highly recommended: jsonpickle v1.7 or higher. Older versions will not serialize recursive data structures as necessary for a lot of applications.) Since JSON is one of the most easily parsed formats in Javascript (the name should be a giveaway), this project, jsonpickleJS, exists to parse the output of Python's jsonpickle into objects in Javascript. The constructors of the objects need to have the same names and exist in the global namespace, such as ``window``, in the Javascript. For instance, if you have a class called ``myobject.Thing`` in Python, you'll need to have a Prototype constructor function called ``window.myobject.Thing`` in Javascript. The object, and any subobjects, will be created as closely as possible in Javascript. The reverse is also possible, with some caveats. Since Javascript doesn't (until ECMAScript 6) have the concept of named classes, each object will need to have a marker somewhere on it saying what Python object it should convert back to. The marker is ``o[jsonpickle.tags.PY_CLASS] = 'fully.qualified.ClassName'``. It may be possible in the future to use ``instanceof`` through the entire Global namespace to figure out what something is, but that seems rather dangerous and inefficient (A project for later). Limitations =========== Remember that Javascript does not have tuples, so all tuple objects are changed to lists. Namedtuples behave the same way, I believe. Dicts and Objects are identical in Javascript (both a blessing and a curse). Security ======== Pickle, jsonpickle, and jsonpickleJS all raise important security considerations you must be aware of. You will be loading data directly into Python or Javascript with no checks on what the data contains. Only load data you have personally produced if you want to be safe. In Javascript, malicious data may compromise your browser, browser history, functioning of the current web page, etc. That's pretty bad, but nothing compared to what can happen if you load jsonpickle data from Javascript into Python: a maliciously written set of jsonpickle data may be able to send the contents of any file back to the net or manipulate or delete the hard drive of the server. It may be that the parsed Python object have to be called in some way, but it may even be possible to have malicious code executed just on parsing; assume the worst. Your .html/.js may only produce safe data, but anyone with JS programming experience can inject other data into your server scripts. Be safe: be cautious going from Python to Javascript and NEVER accept Javascript-produced jsonpickle data from the net into your Python program in any sort of open environment. Usage ===== See the source code of: testUnpickle.html to see how to use it. JsonpickleJS follows the AMD moduleloader standard, so set the ``src=""`` attribute in the `` See Console for output... jsonpickle-1.2/jsonpickleJS/testUnpickle.html0000664000372000037200000000146513471677136022342 0ustar travistravis00000000000000 See Console for output... jsonpickle-1.2/jsonpickleJS/unpickler.js0000664000372000037200000002706413471677136021337 0ustar travistravis00000000000000/** * jsonPickle/javascript/unpickler -- Conversion from music21p jsonpickle streams * * Copyright (c) 2013-14, Michael Scott Cuthbert and cuthbertLab * Based on music21 (=music21p), Copyright (c) 2006–14, Michael Scott Cuthbert and cuthbertLab * * usage: * * js_obj = unpickler.decode(json_string); * * */ define(['./util', './handlers', './tags'], function(util, handlers, tags) { var unpickler = {}; unpickler.decode = function (string, user_handlers, options) { var params = { keys: false, safe: false, reset: true, backend: JSON, }; util.merge(params, options); var use_handlers = {}; util.merge(use_handlers, handlers); // don't screw up default handlers... util.merge(use_handlers, user_handlers); // backend does not do anything -- it is there for // compat with py-JSON-Pickle if (params.context === undefined) { var unpickler_options = { keys: params.keys, backend: params.backend, safe: params.safe, }; context = new unpickler.Unpickler(unpickler_options, use_handlers); } var jsonObj = params.backend.parse(string); return context.restore(jsonObj, params.reset); }; unpickler.Unpickler = function (options, handlers) { var params = { keys: false, safe: false, }; util.merge(params, options); this.keys = params.keys; this.safe = params.safe; this.handlers = handlers; //obsolete... //this._namedict = {}; // The namestack grows whenever we recurse into a child object this._namestack = []; // Maps objects to their index in the _objs list this._obj_to_idx = {}; this._objs = []; }; unpickler.Unpickler.prototype.reset = function () { //this._namedict = {}; this._namestack = []; this._obj_to_idx = {}; this._objs = []; }; /** * Restores a flattened object to a JavaScript representation * as close to the original python object as possible. * * Requires that javascript */ unpickler.Unpickler.prototype.restore = function (obj, reset) { if (reset) { this.reset(); } return this._restore(obj); }; unpickler.Unpickler.prototype._restore = function (obj) { var has_tag = unpickler.has_tag; var restore = undefined; if (has_tag(obj, tags.ID)) { restore = this._restore_id.bind(this); } else if (has_tag(obj, tags.REF)) { // backwards compat. not supported } else if (has_tag(obj, tags.TYPE)) { restore = this._restore_type.bind(this); } else if (has_tag(obj, tags.REPR)) { // backwards compat. not supported } else if (has_tag(obj, tags.OBJECT)) { restore = this._restore_object.bind(this); } else if (has_tag(obj, tags.TUPLE)) { restore = this._restore_tuple.bind(this); } else if (has_tag(obj, tags.SET)) { restore = this._restore_set.bind(this); } else if (util.is_list(obj)) { restore = this._restore_list.bind(this); } else if (util.is_dictionary(obj)) { restore = this._restore_dict.bind(this); } else { restore = function (obj) { return obj; }; } return restore(obj); }; unpickler.Unpickler.prototype._restore_id = function (obj) { return this._objs[obj[tags.ID]]; }; unpickler.Unpickler.prototype._restore_type = function (obj) { var typeref = unpickler.loadclass(obj[tags.TYPE]); if (typeref === undefined) { return obj; } else { return typeref; } }; unpickler.Unpickler.prototype._restore_object = function (obj) { var class_name = obj[tags.OBJECT]; var handler = this.handlers[class_name]; if (handler !== undefined && handler.restore !== undefined) { var instance = handler.restore(obj); instance[tags.PY_CLASS] = class_name; return this._mkref(instance); } else { var cls = unpickler.loadclass(class_name); if (cls === undefined) { obj[tags.PY_CLASS] = class_name; return this._mkref(obj); } var instance = this._restore_object_instance(obj, cls); instance[tags.PY_CLASS] = class_name; if (handler !== undefined && handler.post_restore !== undefined) { return handler.post_restore(instance); } else { return instance; } } }; unpickler.Unpickler.prototype._loadfactory = function (obj) { var default_factory = obj['default_factory']; if (default_factory === undefined) { return undefined; } else { obj['default_factory'] = undefined; return this._restore(default_factory); } }; unpickler.Unpickler.prototype._restore_object_instance = function (obj, cls) { //var factory = this._loadfactory(obj); var args = unpickler.getargs(obj); if (args.length > 0) { args = this._restore(args); } // not using factory... does not seem to apply to JS var instance = unpickler.construct(cls, args); this._mkref(instance); return this._restore_object_instance_variables(obj, instance); }; unpickler.Unpickler.prototype._restore_object_instance_variables = function (obj, instance) { var has_tag = unpickler.has_tag; var restore_key = this._restore_key_fn(); var keys = []; for (var k in obj) { if (obj.hasOwnProperty(k)) { keys.push(k); } } keys.sort(); for (var i = 0; i < keys.length; i++) { var k = keys[i]; if (tags.RESERVED.indexOf(k) != -1) { continue; } var v = obj[k]; this._namestack.push(k); k = restore_key(k); var value = undefined; if (v !== undefined && v !== null) { value = this._restore(v); } // no setattr checks... instance[k] = value; this._namestack.pop(); } if (has_tag(obj, tags.SEQ)) { if (instance.push !== undefined) { for (var v in obj[tags.SEQ]) { instance.push(this._restore(v)); } } // no .add ... } if (has_tag(obj, tags.STATE)) { instance = this._restore_state(obj, instance); } return instance; }; unpickler.Unpickler.prototype._restore_state = function (obj, instance) { // only if the JS object implements __setstate__ if (instance.__setstate__ !== undefined) { var state = this._restore(obj[tags.STATE]); instance.__setstate__(state); } else { instance = this._restore(obj[tags.STATE]); } return instance; }; unpickler.Unpickler.prototype._restore_list = function (obj) { var parent = []; this._mkref(parent); var children = []; for (var i = 0; i < obj.length; i++) { var v = obj[i]; var rest = this._restore(v); children.push(rest); } parent.push.apply(parent, children); return parent; }; unpickler.Unpickler.prototype._restore_tuple = function (obj) { // JS having no difference between list, tuple, set -- returns Array var children = []; var tupleContents = obj[tags.TUPLE]; for (var i = 0; i < tupleContents.length; i++) { children.push(this._restore(tupleContents[i])); } return children; }; unpickler.Unpickler.prototype._restore_set = function (obj) { // JS having no difference between list, tuple, set -- returns Array var children = []; var setContents = obj[tags.SET]; for (var i = 0; i < setContents.length; i++) { children.push(this._restore(setContents[i])); } return children; }; unpickler.Unpickler.prototype._restore_dict = function (obj) { var data = {}; //var restore_key = this._restore_key_fn(); var keys = []; for (var k in obj) { if (obj.hasOwnProperty(k)) { keys.push(k); } } keys.sort(); for (var i = 0; i < keys.length; i++) { var k = keys[i]; var v = obj[k]; this._namestack.push(k); data[k] = this._restore(v); // no setattr checks... this._namestack.pop(); } return data; }; unpickler.Unpickler.prototype._restore_key_fn = function () { if (this.keys) { return function (key) { if (key.indexOf(tags.JSON_KEY) == 0) { key = unpickler.decode(key.slice(tags.JSON_KEY.length), this.handlers, {context: this, keys: this.keys, reset: false} ); return key; } }; } else { return function (key) { return key; }; } }; // _refname not needed... unpickler.Unpickler.prototype._mkref = function (obj) { // does not use id(obj) in javascript this._objs.push(obj); return obj; }; unpickler.getargs = function (obj) { var seq_list = obj[tags.SEQ]; var obj_dict = obj[tags.OBJECT]; if (seq_list === undefined || obj_dict === undefined) { return []; } var typeref = unpickler.loadclass(obj_dict); if (typeref === undefined) { return []; } if (typeref['_fields'] !== undefined) { if (typeref['_fields'].length == seq_list.length) { return seq_list; } } return []; }; unpickler.loadclass = function (module_and_name) { var main_check = '__main__.'; if (module_and_name.indexOf(main_check) == 0) { module_and_name = module_and_name.slice(main_check.length); } var parent = window; var module_class_split = module_and_name.split('.'); for (var i = 0; i < module_class_split.length; i++) { var this_module_or_class = module_class_split[i]; parent = parent[this_module_or_class]; if (parent === undefined) { return parent; } } return parent; }; unpickler.has_tag = function (obj, tag) { if ((typeof obj == 'object') && (obj[tag] !== undefined)) { return true; } else { return false; } }; // http://stackoverflow.com/questions/1606797/use-of-apply-with-new-operator-is-this-possible unpickler.construct = function (constructor, args) { function F() { return constructor.apply(this, args); } F.prototype = constructor.prototype; return new F(); }; if (jsonpickle !== undefined) { jsonpickle.unpickler = unpickler; } return unpickler; });jsonpickle-1.2/jsonpickleJS/util.js0000664000372000037200000000416413471677136020314 0ustar travistravis00000000000000define(['./tags'], function(tags) { var util = {}; util.merge = function(destination, source) { if (source !== undefined) { for (var property in source) { destination[property] = source[property]; } } return destination; }; util.PRIMITIVES = ['string', 'number', 'boolean']; /* from jsonpickle.util */ // to_do... util.is_type = function (obj) { return false; }; // same as dictionary in JS... util.is_object = function (obj) { return util.is_dictionary(obj); }; util.is_primitive = function (obj) { if (obj === undefined || obj == null) { return true; } if (util.PRIMITIVES.indexOf(typeof obj) != -1) { return true; } return false; }; util.is_dictionary = function (obj) { return ((typeof obj == 'object') && (obj !== null)); }; util.is_sequence = function (obj) { return (util.is_list(obj) || util.is_set(obj) || util.is_tuple(obj)); }; util.is_list = function (obj) { return (obj instanceof Array); }; // to_do... util.is_set = function (obj) { return false; }; util.is_tuple = function (obj) { return false; }; util.is_dictionary_subclass = function (obj) { return false; }; util.is_sequence_subclass = function (obj) { return false; }; util.is_noncomplex = function (obj) { return false; }; util.is_function = function (obj) { return (typeof obj == 'function'); }; util.is_module = function (obj) { return false; }; util.is_picklable = function(name, value) { if (tags.RESERVED.indexOf(name) != -1) { return false; } if (util.is_function(value)) { return false; } else { return true; } }; util.is_installed = function (module) { return true; // ??? }; util.is_list_like = function (obj) { return util.is_list(obj); }; if (typeof jsonpickle != "undefined") { jsonpickle.util = util; } return util; });jsonpickle-1.2/requirements-dev.txt0000664000372000037200000000030713471677136020436 0ustar travistravis00000000000000bson coverage demjson ecdsa enum34; python_version < '3.4' feedparser jsonlib; python_version < '3.0' numpy pandas pymongo pytest pytest-cov simplejson sqlalchemy ujson yajl; sys_platform != 'win32' jsonpickle-1.2/setup.cfg0000664000372000037200000000022713471677155016221 0ustar travistravis00000000000000[aliases] dists = clean --all sdist bdist_wheel [bdist_wheel] universal = 1 [metadata] license_file = COPYING [egg_info] tag_build = tag_date = 0 jsonpickle-1.2/setup.py0000664000372000037200000000367713471677136016125 0ustar travistravis00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # # Copyright (C) 2008 John Paulett (john -at- paulett.org) # Copyright (C) 2009-2017 David Aguilar (davvid -at- gmail.com) # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. import os try: import setuptools as setup_mod except ImportError: import distutils.core as setup_mod here = os.path.dirname(__file__) version = os.path.join(here, 'jsonpickle', 'version.py') scope = {} exec(open(version).read(), scope) SETUP_ARGS = dict( name='jsonpickle', version=scope['__version__'], description=( 'Python library for serializing any arbitrary object graph into JSON'), long_description=( 'jsonpickle converts complex Python objects to and from JSON.'), author='David Aguilar', author_email='davvid@gmail.com', url='https://jsonpickle.github.io/', license='BSD', platforms=['POSIX', 'Windows'], keywords=['json pickle', 'json', 'pickle', 'marshal', 'serialization', 'JavaScript Object Notation'], classifiers=[ 'License :: OSI Approved :: BSD License', 'Operating System :: OS Independent', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.7', 'Topic :: Software Development :: Libraries :: Python Modules', 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', 'Programming Language :: Python', 'Programming Language :: JavaScript', ], options={'clean': {'all': 1}}, packages=['jsonpickle', 'jsonpickle.ext'], ) if __name__ == '__main__': setup_mod.setup(**SETUP_ARGS) jsonpickle-1.2/tests/0000775000372000037200000000000013471677155015541 5ustar travistravis00000000000000jsonpickle-1.2/tests/backend_test.py0000664000372000037200000001451613471677136020547 0ustar travistravis00000000000000from __future__ import absolute_import, division, unicode_literals import decimal import unittest from warnings import warn import jsonpickle from jsonpickle import compat from jsonpickle.compat import PY2 from jsonpickle.compat import PY3 from helper import SkippableTest class Thing(object): def __init__(self, name): self.name = name self.child = None SAMPLE_DATA = {'things': [Thing('data')]} class BackendBase(SkippableTest): def _is_installed(self, backend): if not jsonpickle.util.is_installed(backend): return self.skip('%s not available; please install' % backend) def set_backend(self, *args): backend = args[0] self._is_installed(backend) jsonpickle.load_backend(*args) jsonpickle.set_preferred_backend(backend) def set_preferred_backend(self, backend): self._is_installed(backend) jsonpickle.set_preferred_backend(backend) def tearDown(self): # always reset to default backend jsonpickle.set_preferred_backend('json') def assertEncodeDecode(self, json_input): expect = SAMPLE_DATA actual = jsonpickle.decode(json_input) self.assertEqual(expect['things'][0].name, actual['things'][0].name) self.assertEqual(expect['things'][0].child, actual['things'][0].child) pickled = jsonpickle.encode(SAMPLE_DATA) actual = jsonpickle.decode(pickled) self.assertEqual(expect['things'][0].name, actual['things'][0].name) self.assertEqual(expect['things'][0].child, actual['things'][0].child) def test_None_dict_key(self): """Ensure that backends produce the same result for None dict keys""" data = {None: None} expect = {'null': None} pickle = jsonpickle.encode(data) actual = jsonpickle.decode(pickle) self.assertEqual(expect, actual) class JsonTestCase(BackendBase): def setUp(self): self.set_preferred_backend('json') def test_backend(self): expected_pickled = ( '{"things": [{' '"py/object": "backend_test.Thing", ' '"name": "data", ' '"child": null} ' ']}') self.assertEncodeDecode(expected_pickled) class SimpleJsonTestCase(BackendBase): def setUp(self): self.set_preferred_backend('simplejson') def test_backend(self): expected_pickled = ( '{"things": [{' '"py/object": "backend_test.Thing", ' '"name": "data", ' '"child": null}' ']}') self.assertEncodeDecode(expected_pickled) def test_decimal(self): # Default behavior: Decimal is preserved obj = decimal.Decimal(0.5) as_json = jsonpickle.dumps(obj) clone = jsonpickle.loads(as_json) self.assertTrue(isinstance(clone, decimal.Decimal)) self.assertEqual(obj, clone) # Custom behavior: we want to use simplejson's Decimal support. jsonpickle.set_encoder_options('simplejson', use_decimal=True, sort_keys=True) jsonpickle.set_decoder_options('simplejson', use_decimal=True) # use_decimal mode allows Decimal objects to pass-through to simplejson. # The end result is we get a simple '0.5' value as our json string. as_json = jsonpickle.dumps(obj, unpicklable=True, use_decimal=True) self.assertEqual(as_json, '0.5') # But when loading we get back a Decimal. clone = jsonpickle.loads(as_json) self.assertTrue(isinstance(clone, decimal.Decimal)) # side-effect: floats become decimals too! obj = 0.5 as_json = jsonpickle.dumps(obj) clone = jsonpickle.loads(as_json) self.assertTrue(isinstance(clone, decimal.Decimal)) def has_module(module): try: __import__(module) except ImportError: warn(module + ' module not available for testing, ' 'consider installing') return False return True class DemjsonTestCase(BackendBase): def setUp(self): self.set_preferred_backend('demjson') def test_backend(self): expected_pickled = compat.ustr( '{"things":[{' '"child":null,' '"name":"data",' '"py/object":"backend_test.Thing"}' ']}') self.assertEncodeDecode(expected_pickled) def test_int_dict_keys_with_numeric_keys(self): jsonpickle.set_encoder_options('demjson', strict=False) int_dict = {1000: [1, 2]} pickle = jsonpickle.encode(int_dict, numeric_keys=True) actual = jsonpickle.decode(pickle) self.assertEqual(actual[1000], [1, 2]) class JsonlibTestCase(BackendBase): def setUp(self): if PY2: self.set_preferred_backend('jsonlib') def test_backend(self): if PY3: return self.skip('no jsonlib for python3') expected_pickled = ( '{"things":[{' r'"py\/object":"backend_test.Thing",' '"name":"data","child":null}' ']}') self.assertEncodeDecode(expected_pickled) class YajlTestCase(BackendBase): def setUp(self): self.set_preferred_backend('yajl') def test_backend(self): expected_pickled = ( '{"things":[{' '"py/object":"backend_test.Thing",' '"name":"data","child":null}' ']}') self.assertEncodeDecode(expected_pickled) class UJsonTestCase(BackendBase): def setUp(self): self.set_preferred_backend('ujson') def test_backend(self): expected_pickled = ( '{"things":[{' r'"py\/object":"backend_test.Thing",' '"name":"data","child":null}' ']}') self.assertEncodeDecode(expected_pickled) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(JsonTestCase)) suite.addTest(unittest.makeSuite(UJsonTestCase)) suite.addTest(unittest.makeSuite(SimpleJsonTestCase)) if has_module('demjson'): suite.addTest(unittest.makeSuite(DemjsonTestCase)) if has_module('yajl'): suite.addTest(unittest.makeSuite(YajlTestCase)) if PY2: if has_module('jsonlib'): suite.addTest(unittest.makeSuite(JsonlibTestCase)) return suite if __name__ == '__main__': unittest.main(defaultTest='suite') jsonpickle-1.2/tests/benchmark.py0000775000372000037200000000221013471677136020042 0ustar travistravis00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # # Copyright (C) 2008 John Paulett (john -at- paulett.org) # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. import argparse import os import sys import timeit json = """\ import feedparser import jsonpickle import feedparser_test as test doc = feedparser.parse(test.RSS_DOC) jsonpickle.set_preferred_backend('%s') pickled = jsonpickle.encode(doc) unpickled = jsonpickle.decode(pickled) if doc['feed']['title'] != unpickled['feed']['title']: print 'Not a match' """ if __name__ == '__main__': sys.path.insert(1, os.getcwd()) parser = argparse.ArgumentParser() parser.add_argument('-n', '--number', type=int, default=500, help='number of iterations to benchmark') parser.add_argument('mod', nargs='?', default='json', help='json module') args = parser.parse_args() mod = args.mod number = args.number print('Using %s' % mod) json_test = timeit.Timer(stmt=json % mod) print("%.9f sec/pass " % (json_test.timeit(number=number) / number)) jsonpickle-1.2/tests/bson_test.py0000664000372000037200000000441113471677136020112 0ustar travistravis00000000000000"""Test serializing pymongo bson structures""" import datetime import pickle import unittest import jsonpickle from helper import SkippableTest bson = None class Object(object): def __init__(self, offset): self.offset = datetime.timedelta(offset) def __getinitargs__(self): return self.offset, class BSONTestCase(SkippableTest): def setUp(self): global bson try: bson = __import__('bson.tz_util') self.should_skip = False except ImportError: self.should_skip = True def test_FixedOffsetSerializable(self): if self.should_skip: return self.skip('bson is not installed') fo = bson.tz_util.FixedOffset(-60*5, 'EST') serialized = jsonpickle.dumps(fo) restored = jsonpickle.loads(serialized) self.assertEqual(vars(restored), vars(fo)) def test_timedelta(self): if self.should_skip: return self.skip('bson is not installed') td = datetime.timedelta(-1, 68400) serialized = jsonpickle.dumps(td) restored = jsonpickle.loads(serialized) self.assertEqual(restored, td) def test_stdlib_pickle(self): if self.should_skip: return self.skip('bson is not installed') fo = bson.tz_util.FixedOffset(-60*5, 'EST') serialized = pickle.dumps(fo) restored = pickle.loads(serialized) self.assertEqual(vars(restored), vars(fo)) def test_nested_objects(self): if self.should_skip: return self.skip('bson is not installed') o = Object(99) serialized = jsonpickle.dumps(o) restored = jsonpickle.loads(serialized) self.assertEqual(restored.offset, datetime.timedelta(99)) def test_datetime_with_fixed_offset(self): if self.should_skip: return self.skip('bson is not installed') fo = bson.tz_util.FixedOffset(-60*5, 'EST') dt = datetime.datetime.now().replace(tzinfo=fo) serialized = jsonpickle.dumps(dt) restored = jsonpickle.loads(serialized) self.assertEqual(restored, dt) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(BSONTestCase, 'test')) return suite if __name__ == '__main__': unittest.main() jsonpickle-1.2/tests/datetime_test.py0000664000372000037200000001774613471677136020764 0ustar travistravis00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2013 Jason R. Coombs # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. import unittest import datetime import time import jsonpickle from jsonpickle import tags class ObjWithDate(object): def __init__(self): ts = datetime.datetime.now() self.data = dict(a='a', ts=ts) self.data_ref = dict(b='b', ts=ts) # UTC implementation from Python 2.7 docs class UTC(datetime.tzinfo): """UTC""" def utcoffset(self, dt): return datetime.timedelta() def tzname(self, dt): return 'UTC' def dst(self, dt): return datetime.timedelta() utc = UTC() class TimestampedVariable(object): def __init__(self, value=None): self._value = value self._dt_read = datetime.datetime.utcnow() self._dt_write = self._dt_read def get(self, default_value=None): if self._dt_read is None and self._dt_write is None: value = default_value self._value = value self._dt_write = datetime.datetime.utcnow() else: value = self._value self._dt_read = datetime.datetime.utcnow() return value def set(self, new_value): self._dt_write = datetime.datetime.utcnow() self._value = new_value def __repr__(self): dt_now = datetime.datetime.utcnow() td_read = dt_now - self._dt_read td_write = dt_now - self._dt_write s = '\n' s += ' value: ' + str(self._value) + '\n' s += ' dt_read: ' + str(self._dt_read) + ' (%s ago)' % td_read + '\n' s += ' dt_write: ' + str(self._dt_write) + ' (%s ago)' % td_write + '\n' return s def erasable(self, td=datetime.timedelta(seconds=1)): dt_now = datetime.datetime.utcnow() td_read = dt_now - self._dt_read td_write = dt_now - self._dt_write return td_read > td and td_write > td class PersistantVariables(object): def __init__(self): self._data = {} def __getitem__(self, key): return self._data.setdefault(key, TimestampedVariable(None)) def __setitem__(self, key, value): return self._data.setdefault(key, TimestampedVariable(value)) def __repr__(self): return str(self._data) class DateTimeInnerReferenceTestCase(unittest.TestCase): def test_object_with_inner_datetime_refs(self): pvars = PersistantVariables() pvars['z'] = 1 pvars['z2'] = 2 pickled = jsonpickle.encode(pvars) obj = jsonpickle.decode(pickled) # ensure the references are valid self.assertTrue(obj['z']._dt_read is obj['z']._dt_write) self.assertTrue(obj['z2']._dt_read is obj['z2']._dt_write) # ensure the values are valid self.assertEqual(obj['z'].get(), 1) self.assertEqual(obj['z2'].get(), 2) # ensure get() updates _dt_read self.assertTrue(obj['z']._dt_read is not obj['z']._dt_write) self.assertTrue(obj['z2']._dt_read is not obj['z2']._dt_write) class DateTimeSimpleTestCase(unittest.TestCase): def _roundtrip(self, obj): """ pickle and then unpickle object, then assert the new object is the same as the original. """ pickled = jsonpickle.encode(obj) unpickled = jsonpickle.decode(pickled) self.assertEqual(obj, unpickled) def test_datetime(self): """ jsonpickle should pickle a datetime object """ self._roundtrip(datetime.datetime.now()) def test_date(self): """ jsonpickle should pickle a date object """ self._roundtrip(datetime.datetime.today()) def test_time(self): """ jsonpickle should pickle a time object """ self._roundtrip(datetime.datetime.now().time()) def test_timedelta(self): """ jsonpickle should pickle a timedelta object """ self._roundtrip(datetime.timedelta(days=3)) def test_utc(self): """ jsonpickle should be able to encode and decode a datetime with a simple, pickleable UTC tzinfo. """ self._roundtrip(datetime.datetime.utcnow().replace(tzinfo=utc)) def test_unpickleable(self): """ If 'unpickleable' is set on the Pickler, the date objects should be simple, human-readable strings. """ obj = datetime.datetime.now() pickler = jsonpickle.pickler.Pickler(unpicklable=False) flattened = pickler.flatten(obj) self.assertEqual(str(obj), flattened) def test_object_with_datetime(self): test_obj = ObjWithDate() json = jsonpickle.encode(test_obj) test_obj_decoded = jsonpickle.decode(json) self.assertEqual(test_obj_decoded.data['ts'], test_obj_decoded.data_ref['ts']) class DateTimeAdvancedTestCase(unittest.TestCase): def setUp(self): self.pickler = jsonpickle.pickler.Pickler() self.unpickler = jsonpickle.unpickler.Unpickler() def tearDown(self): self.pickler.reset() self.unpickler.reset() def test_struct_time(self): expect = time.struct_time([1, 2, 3, 4, 5, 6, 7, 8, 9]) json = jsonpickle.encode(expect) actual = jsonpickle.decode(json) self.assertEqual(type(actual), time.struct_time) self.assertEqual(expect, actual) def test_struct_time_chars(self): expect = time.struct_time('123456789') flattened = self.pickler.flatten(expect) actual = self.unpickler.restore(flattened) self.assertEqual(expect, actual) def test_datetime_structure(self): obj = datetime.datetime.now() flattened = self.pickler.flatten(obj) self.assertTrue(tags.OBJECT in flattened) self.assertTrue('__reduce__' in flattened) inflated = self.unpickler.restore(flattened) self.assertEqual(obj, inflated) def test_datetime_inside_int_keys_defaults(self): t = datetime.time(hour=10) s = jsonpickle.encode({1: t, 2: t}) d = jsonpickle.decode(s) self.assertEqual(d["1"], d["2"]) self.assertTrue(d["1"] is d["2"]) self.assertTrue(isinstance(d["1"], datetime.time)) def test_datetime_inside_int_keys_with_keys_enabled(self): t = datetime.time(hour=10) s = jsonpickle.encode({1: t, 2: t}, keys=True) d = jsonpickle.decode(s, keys=True) self.assertEqual(d[1], d[2]) self.assertTrue(d[1] is d[2]) self.assertTrue(isinstance(d[1], datetime.time)) def test_datetime_repr_not_unpicklable(self): obj = datetime.datetime.now() pickler = jsonpickle.pickler.Pickler(unpicklable=False) flattened = pickler.flatten(obj) self.assertFalse(tags.REPR in flattened) self.assertFalse(tags.OBJECT in flattened) self.assertEqual(str(obj), flattened) def test_datetime_dict_keys_defaults(self): """Test that we handle datetime objects as keys.""" datetime_dict = {datetime.datetime(2008, 12, 31): True} pickled = jsonpickle.encode(datetime_dict) expect = {'datetime.datetime(2008, 12, 31, 0, 0)': True} actual = jsonpickle.decode(pickled) self.assertEqual(expect, actual) def test_datetime_dict_keys_with_keys_enabled(self): """Test that we handle datetime objects as keys.""" datetime_dict = {datetime.datetime(2008, 12, 31): True} pickled = jsonpickle.encode(datetime_dict, keys=True) expect = datetime_dict actual = jsonpickle.decode(pickled, keys=True) self.assertEqual(expect, actual) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(DateTimeSimpleTestCase)) suite.addTest(unittest.makeSuite(DateTimeAdvancedTestCase)) suite.addTest(unittest.makeSuite(DateTimeInnerReferenceTestCase)) return suite if __name__ == '__main__': unittest.main(defaultTest='suite') jsonpickle-1.2/tests/document_test.py0000664000372000037200000000506413471677136020774 0ustar travistravis00000000000000# -*- coding: utf-8 -*- import unittest import jsonpickle class Node(object): def __init__(self, name): self._name = name self._children = [] self._parent = None def add_child(self, child, index=-1): if index == -1: index = len(self._children) self._children.insert(index, child) child._parent = self class Document(Node): def __init__(self, name): Node.__init__(self, name) def __str__(self): ret_str = 'Document "%s"\n' % self._name for c in self._children: ret_str += repr(c) return ret_str def __repr__(self): return str(self) class Question(Node): def __init__(self, name): Node.__init__(self, name) def __str__(self): return ('Question "%s", parent: "%s"\n' % (self._name, self._parent._name)) def __repr__(self): return self.__str__() class Section(Node): def __init__(self, name): Node.__init__(self, name) def __str__(self): ret_str = ('Section "%s", parent: "%s"\n' % (self._name, self._parent._name)) for c in self._children: ret_str += repr(c) return ret_str def __repr__(self): return self.__str__() class DocumentTestCase(unittest.TestCase): def test_cyclical(self): """Test that we can pickle cyclical data structure This test is ensures that we can reference objects which first appear within a list (in other words, not a top-level object or attribute). Later children will reference that object through its "_parent" field. This makes sure that we handle this case correctly. """ document = Document('My Document') section1 = Section('Section 1') section2 = Section('Section 2') question1 = Question('Question 1') question2 = Question('Question 2') question3 = Question('Question 3') question4 = Question('Question 4') document.add_child(section1) document.add_child(section2) section1.add_child(question1) section1.add_child(question2) section2.add_child(question3) section2.add_child(question4) pickled = jsonpickle.encode(document) unpickled = jsonpickle.decode(pickled) self.assertEqual(str(document), str(unpickled)) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(DocumentTestCase)) return suite if __name__ == '__main__': unittest.main(defaultTest='suite') jsonpickle-1.2/tests/ecdsa_test.py0000664000372000037200000000214013471677136020225 0ustar travistravis00000000000000"""Test serializing ecdsa keys""" from __future__ import absolute_import, division, unicode_literals import unittest import jsonpickle from helper import SkippableTest class EcdsaTestCase(SkippableTest): def setUp(self): try: from ecdsa import NIST384p from ecdsa.keys import SigningKey self.NIST384p = NIST384p self.SigningKey = SigningKey self.should_skip = False except ImportError: self.should_skip = True def test_roundtrip(self): if self.should_skip: return self.skip('ecdsa module is not installed') message = 'test'.encode('utf-8') key_pair = self.SigningKey.generate(curve=self.NIST384p) sig = key_pair.sign(message) serialized = jsonpickle.dumps(key_pair.get_verifying_key()) restored = jsonpickle.loads(serialized) self.assertTrue(restored.verify(sig, message)) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(EcdsaTestCase, 'test')) return suite if __name__ == '__main__': unittest.main() jsonpickle-1.2/tests/feedparser_test.py0000664000372000037200000000612013471677136021270 0ustar travistravis00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2008 John Paulett (john -at- paulett.org) # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. import unittest import jsonpickle from helper import SkippableTest RSS_DOC = """ Sample Feed For documentation <em>only</em> <p>Copyright 2005, Mark Pilgrim</p>< Sample Toolkit tag:feedparser.org,2005-11-09:/docs/examples/atom10.xml 2005-11-09T11:56:34Z First entry title tag:feedparser.org,2005-11-09:/docs/examples/atom10.xml:3 2005-11-09T00:23:47Z 2005-11-09T11:56:34Z Mark Pilgrim http://diveintomark.org/ mark@example.org Joe http://example.org/joe/ joe@example.org Sam http://example.org/sam/ sam@example.org Watch out for nasty tricks
Watch out for nasty tricks
""" class FeedParserTestCase(SkippableTest): def setUp(self): try: import feedparser self.should_skip = False self.doc = feedparser.parse(RSS_DOC) except ImportError: self.should_skip = True return def test(self): if self.should_skip: return self.skip('feedparser module not available, please install') pickled = jsonpickle.encode(self.doc) unpickled = jsonpickle.decode(pickled) self.assertEqual(self.doc['feed']['title'], unpickled['feed']['title']) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(FeedParserTestCase, 'test')) return suite if __name__ == '__main__': unittest.main(defaultTest='suite') jsonpickle-1.2/tests/handler_test.py0000664000372000037200000001130213471677136020563 0ustar travistravis00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2013 Jason R. Coombs # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. import doctest import unittest import jsonpickle import jsonpickle.handlers class CustomObject(object): "A class to be serialized by a custom handler" def __init__(self, name=None, creator=None): self.name = name self.creator = creator def __eq__(self, other): return self.name == other.name class CustomA(CustomObject): pass class CustomB(CustomA): pass class NullHandler(jsonpickle.handlers.BaseHandler): def flatten(self, obj, data): data['name'] = obj.name return data def restore(self, obj): return CustomObject(obj['name'], creator=type(self)) class DecoratedBase(CustomObject): pass class DecoratedChild(DecoratedBase): pass @jsonpickle.handlers.register(DecoratedBase, base=True) class DecoratedHandler(NullHandler): pass class SithHandler(jsonpickle.handlers.BaseHandler): """(evil) serialization handler that rewrites the entire object""" def flatten(self, obj, data): data['name'] = obj.name data['py/object'] = 'sith.lord' return data class HandlerTestCase(unittest.TestCase): def setUp(self): jsonpickle.handlers.register(CustomObject, NullHandler) def tearDown(self): jsonpickle.handlers.unregister(CustomObject) def roundtrip(self, ob): encoded = jsonpickle.encode(ob) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded, ob) return decoded def test_custom_handler(self): """Ensure that the custom handler is indeed used""" expect = CustomObject('jarjar') encoded = jsonpickle.encode(expect) actual = jsonpickle.decode(encoded) self.assertEqual(expect.name, actual.name) self.assertTrue(expect.creator is None) self.assertTrue(actual.creator is NullHandler) def test_references(self): """ Ensure objects handled by a custom handler are properly dereferenced. """ ob = CustomObject() # create a dictionary which contains several references to ob subject = dict(a=ob, b=ob, c=ob) # ensure the subject can be roundtripped new_subject = self.roundtrip(subject) self.assertEqual(new_subject['a'], new_subject['b']) self.assertEqual(new_subject['b'], new_subject['c']) self.assertTrue(new_subject['a'] is new_subject['b']) self.assertTrue(new_subject['b'] is new_subject['c']) def test_invalid_class(self): self.assertRaises(TypeError, jsonpickle.handlers.register, 'foo', NullHandler) def test_base_handler(self): a = CustomA('a') self.assertTrue(a.creator is None) self.assertTrue(jsonpickle.decode(jsonpickle.encode(a)).creator is None) b = CustomB('b') self.assertTrue(b.creator is None) self.assertTrue(jsonpickle.decode(jsonpickle.encode(b)).creator is None) OtherHandler = type('OtherHandler', (NullHandler,), {}) jsonpickle.handlers.register(CustomA, OtherHandler, base=True) self.assertTrue(self.roundtrip(a).creator is OtherHandler) self.assertTrue(self.roundtrip(b).creator is OtherHandler) SpecializedHandler = type('SpecializedHandler', (NullHandler,), {}) jsonpickle.handlers.register(CustomB, SpecializedHandler) self.assertTrue(self.roundtrip(a).creator is OtherHandler) self.assertTrue(self.roundtrip(b).creator is SpecializedHandler) def test_decorated_register(self): db = DecoratedBase('db') dc = DecoratedChild('dc') self.assertTrue(self.roundtrip(db).creator is DecoratedHandler) self.assertTrue(self.roundtrip(dc).creator is DecoratedHandler) def test_custom_handler_can_rewrite_everything(self): """Test the low-level pickling structures""" jsonpickle.handlers.unregister(CustomObject) jsonpickle.handlers.register(CustomObject, SithHandler) obj = CustomObject('jarjar') # secret sith lord jarjar pickler = jsonpickle.pickler.Pickler() data = pickler.flatten(obj) self.assertTrue(isinstance(data, dict)) self.assertEqual(len(data), 2) self.assertEqual('jarjar', data['name']) self.assertEqual('sith.lord', data['py/object']) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(HandlerTestCase)) suite.addTest(doctest.DocTestSuite(jsonpickle.handlers)) return suite if __name__ == '__main__': unittest.main(defaultTest='suite') jsonpickle-1.2/tests/helper.py0000664000372000037200000000026413471677136017373 0ustar travistravis00000000000000import unittest class SkippableTest(unittest.TestCase): def skip(self, msg): if hasattr(self, 'skipTest'): return self.skipTest(msg) return None jsonpickle-1.2/tests/jsonpickle_test.py0000664000372000037200000015175213471677136021325 0ustar travistravis00000000000000# Copyright (C) 2008 John Paulett (john -at- paulett.org) # Copyright (C) 2009-2018 David Aguilar (davvid -at- gmail.com) # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. from __future__ import absolute_import, division, unicode_literals import doctest import os import unittest import collections import jsonpickle import jsonpickle.backend import jsonpickle.handlers from jsonpickle import compat, tags, util from jsonpickle.compat import PY2, PY3 from helper import SkippableTest class Thing(object): def __init__(self, name): self.name = name self.child = None def __repr__(self): return 'Thing("%s")' % self.name class Capture(object): def __init__(self, *args, **kwargs): self.args = args self.kwargs = kwargs class ThingWithProps(object): def __init__(self, name='', dogs='reliable', monkies='tricksy'): self.name = name self._critters = (('dogs', dogs), ('monkies', monkies)) def _get_identity(self): keys = [self.dogs, self.monkies, self.name] return hash('-'.join([str(key) for key in keys])) identity = property(_get_identity) def _get_dogs(self): return self._critters[0][1] dogs = property(_get_dogs) def _get_monkies(self): return self._critters[1][1] monkies = property(_get_monkies) def __getstate__(self): out = dict( __identity__=self.identity, nom=self.name, dogs=self.dogs, monkies=self.monkies, ) return out def __setstate__(self, state_dict): self._critters = (('dogs', state_dict.get('dogs')), ('monkies', state_dict.get('monkies'))) self.name = state_dict.get('nom', '') ident = state_dict.get('__identity__') if ident != self.identity: raise ValueError('expanded object does not match originial state!') def __eq__(self, other): return self.identity == other.identity class UserDict(dict): """A user class that inherits from :class:`dict`""" def __init__(self, **kwargs): dict.__init__(self, **kwargs) self.valid = False class Outer(object): class Middle(object): class Inner(object): pass class PicklingTestCase(unittest.TestCase): def setUp(self): self.pickler = jsonpickle.pickler.Pickler() self.unpickler = jsonpickle.unpickler.Unpickler() self.b85_pickler = jsonpickle.pickler.Pickler(use_base85=True) def tearDown(self): self.pickler.reset() self.unpickler.reset() @unittest.skipIf(not PY2, 'Python 2-specific base85 test') def test_base85_always_false_on_py2(self): self.assertFalse(self.b85_pickler.use_base85) @unittest.skipIf(PY2, 'Base85 not supported on Python 2') def test_base85_override_py3(self): """Ensure the Python 2 check still lets us set use_base85 on Python 3""" self.assertTrue(self.b85_pickler.use_base85) @unittest.skipIf(PY2, 'Base85 not supported on Python 2') def test_bytes_default_base85(self): data = os.urandom(16) encoded = util.b85encode(data) self.assertEqual({tags.B85: encoded}, self.b85_pickler.flatten(data)) @unittest.skipIf(PY2, 'Base85 not supported on Python 2') def test_py3_bytes_base64_default(self): data = os.urandom(16) encoded = util.b64encode(data) self.assertEqual({tags.B64: encoded}, self.pickler.flatten(data)) @unittest.skipIf(not PY2, 'Python 2-specific base64 test') def test_py2_default_base64(self): data = os.urandom(16) encoded = util.b64encode(data) self.assertEqual({tags.B64: encoded}, self.pickler.flatten(data)) @unittest.skipIf(PY2, 'Base85 not supported on Python 2') def test_decode_base85(self): pickled = {tags.B85: 'P{Y4;Xv4O{u^=-c'} expected = u'P\u00ffth\u00f6\u00f1 3!'.encode('utf-8') self.assertEqual(expected, self.unpickler.restore(pickled)) @unittest.skipIf(PY2, 'Base85 not supported on Python 2') def test_base85_still_handles_base64(self): pickled = {tags.B64: 'UMO/dGjDtsOxIDMh'} expected = u'P\u00ffth\u00f6\u00f1 3!'.encode('utf-8') self.assertEqual(expected, self.unpickler.restore(pickled)) @unittest.skipIf(not PY2, 'Python 2-specific base85 test') def test_base85_crashes_py2(self): with self.assertRaises(NotImplementedError): self.unpickler.restore({tags.B85: 'P{Y4;Xv4O{u^=-c'}) def test_string(self): self.assertEqual('a string', self.pickler.flatten('a string')) self.assertEqual('a string', self.unpickler.restore('a string')) def test_unicode(self): self.assertEqual('a string', self.pickler.flatten('a string')) self.assertEqual('a string', self.unpickler.restore('a string')) def test_int(self): self.assertEqual(3, self.pickler.flatten(3)) self.assertEqual(3, self.unpickler.restore(3)) def test_float(self): self.assertEqual(3.5, self.pickler.flatten(3.5)) self.assertEqual(3.5, self.unpickler.restore(3.5)) def test_boolean(self): self.assertTrue(self.pickler.flatten(True)) self.assertFalse(self.pickler.flatten(False)) self.assertTrue(self.unpickler.restore(True)) self.assertFalse(self.unpickler.restore(False)) def test_none(self): self.assertTrue(self.pickler.flatten(None) is None) self.assertTrue(self.unpickler.restore(None) is None) def test_list(self): # multiple types of values listA = [1, 35.0, 'value'] self.assertEqual(listA, self.pickler.flatten(listA)) self.assertEqual(listA, self.unpickler.restore(listA)) # nested list listB = [40, 40, listA, 6] self.assertEqual(listB, self.pickler.flatten(listB)) self.assertEqual(listB, self.unpickler.restore(listB)) # 2D list listC = [[1, 2], [3, 4]] self.assertEqual(listC, self.pickler.flatten(listC)) self.assertEqual(listC, self.unpickler.restore(listC)) # empty list listD = [] self.assertEqual(listD, self.pickler.flatten(listD)) self.assertEqual(listD, self.unpickler.restore(listD)) def test_set(self): setlist = ['orange', 'apple', 'grape'] setA = set(setlist) flattened = self.pickler.flatten(setA) for s in setlist: self.assertTrue(s in flattened[tags.SET]) setA_pickle = {tags.SET: setlist} self.assertEqual(setA, self.unpickler.restore(setA_pickle)) def test_dict(self): dictA = {'key1': 1.0, 'key2': 20, 'key3': 'thirty', tags.JSON_KEY + '6': 6} self.assertEqual(dictA, self.pickler.flatten(dictA)) self.assertEqual(dictA, self.unpickler.restore(dictA)) dictB = {} self.assertEqual(dictB, self.pickler.flatten(dictB)) self.assertEqual(dictB, self.unpickler.restore(dictB)) def test_tuple(self): # currently all collections are converted to lists tupleA = (4, 16, 32) tupleA_pickle = {tags.TUPLE: [4, 16, 32]} self.assertEqual(tupleA_pickle, self.pickler.flatten(tupleA)) self.assertEqual(tupleA, self.unpickler.restore(tupleA_pickle)) tupleB = (4,) tupleB_pickle = {tags.TUPLE: [4]} self.assertEqual(tupleB_pickle, self.pickler.flatten(tupleB)) self.assertEqual(tupleB, self.unpickler.restore(tupleB_pickle)) def test_tuple_roundtrip(self): data = (1, 2, 3) newdata = jsonpickle.decode(jsonpickle.encode(data)) self.assertEqual(data, newdata) def test_set_roundtrip(self): data = {1, 2, 3} newdata = jsonpickle.decode(jsonpickle.encode(data)) self.assertEqual(data, newdata) def test_list_roundtrip(self): data = [1, 2, 3] newdata = jsonpickle.decode(jsonpickle.encode(data)) self.assertEqual(data, newdata) def test_class(self): inst = Thing('test name') inst.child = Thing('child name') flattened = self.pickler.flatten(inst) self.assertEqual('test name', flattened['name']) child = flattened['child'] self.assertEqual('child name', child['name']) inflated = self.unpickler.restore(flattened) self.assertEqual('test name', inflated.name) self.assertTrue(type(inflated) is Thing) self.assertEqual('child name', inflated.child.name) self.assertTrue(type(inflated.child) is Thing) def test_classlist(self): array = [Thing('one'), Thing('two'), 'a string'] flattened = self.pickler.flatten(array) self.assertEqual('one', flattened[0]['name']) self.assertEqual('two', flattened[1]['name']) self.assertEqual('a string', flattened[2]) inflated = self.unpickler.restore(flattened) self.assertEqual('one', inflated[0].name) self.assertTrue(type(inflated[0]) is Thing) self.assertEqual('two', inflated[1].name) self.assertTrue(type(inflated[1]) is Thing) self.assertEqual('a string', inflated[2]) def test_classdict(self): dict = {'k1': Thing('one'), 'k2': Thing('two'), 'k3': 3} flattened = self.pickler.flatten(dict) self.assertEqual('one', flattened['k1']['name']) self.assertEqual('two', flattened['k2']['name']) self.assertEqual(3, flattened['k3']) inflated = self.unpickler.restore(flattened) self.assertEqual('one', inflated['k1'].name) self.assertTrue(type(inflated['k1']) is Thing) self.assertEqual('two', inflated['k2'].name) self.assertTrue(type(inflated['k2']) is Thing) self.assertEqual(3, inflated['k3']) def test_recursive(self): """create a recursive structure and test that we can handle it """ parent = Thing('parent') child = Thing('child') child.sibling = Thing('sibling') parent.self = parent parent.child = child parent.child.twin = child parent.child.parent = parent parent.child.sibling.parent = parent cloned = jsonpickle.decode(jsonpickle.encode(parent)) self.assertEqual(parent.name, cloned.name) self.assertEqual(parent.child.name, cloned.child.name) self.assertEqual(parent.child.sibling.name, cloned.child.sibling.name) self.assertEqual(cloned, cloned.child.parent) self.assertEqual(cloned, cloned.child.sibling.parent) self.assertEqual(cloned, cloned.child.twin.parent) self.assertEqual(cloned.child, cloned.child.twin) def test_tuple_notunpicklable(self): self.pickler.unpicklable = False flattened = self.pickler.flatten(('one', 2, 3)) self.assertEqual(flattened, ['one', 2, 3]) def test_set_not_unpicklable(self): self.pickler.unpicklable = False flattened = self.pickler.flatten({'one', 2, 3}) self.assertTrue('one' in flattened) self.assertTrue(2 in flattened) self.assertTrue(3 in flattened) self.assertTrue(isinstance(flattened, list)) def test_thing_with_module(self): obj = Thing('with-module') obj.themodule = os flattened = self.pickler.flatten(obj) inflated = self.unpickler.restore(flattened) self.assertEqual(inflated.themodule, os) def test_thing_with_module_safe(self): obj = Thing('with-module') obj.themodule = os flattened = self.pickler.flatten(obj) self.unpickler.safe = True inflated = self.unpickler.restore(flattened) self.assertEqual(inflated.themodule, None) def test_thing_with_submodule(self): from distutils import sysconfig obj = Thing('with-submodule') obj.submodule = sysconfig flattened = self.pickler.flatten(obj) inflated = self.unpickler.restore(flattened) self.assertEqual(inflated.submodule, sysconfig) def test_type_reference(self): """This test ensures that users can store references to types. """ obj = Thing('object-with-type-reference') # reference the built-in 'object' type obj.typeref = object flattened = self.pickler.flatten(obj) self.assertEqual(flattened['typeref'], {tags.TYPE: 'builtins.object'}) inflated = self.unpickler.restore(flattened) self.assertEqual(inflated.typeref, object) def test_class_reference(self): """This test ensures that users can store references to classes. """ obj = Thing('object-with-class-reference') # reference the 'Thing' class (not an instance of the class) obj.classref = Thing flattened = self.pickler.flatten(obj) self.assertEqual(flattened['classref'], {tags.TYPE: 'jsonpickle_test.Thing'}) inflated = self.unpickler.restore(flattened) self.assertEqual(inflated.classref, Thing) def test_supports_getstate_setstate(self): obj = ThingWithProps('object-which-defines-getstate-setstate') flattened = self.pickler.flatten(obj) self.assertTrue(flattened[tags.STATE].get('__identity__')) self.assertTrue(flattened[tags.STATE].get('nom')) inflated = self.unpickler.restore(flattened) self.assertEqual(obj, inflated) def test_references(self): obj_a = Thing('foo') obj_b = Thing('bar') coll = [obj_a, obj_b, obj_b] flattened = self.pickler.flatten(coll) inflated = self.unpickler.restore(flattened) self.assertEqual(len(inflated), len(coll)) for x in range(len(coll)): self.assertEqual(repr(coll[x]), repr(inflated[x])) def test_references_in_number_keyed_dict(self): """ Make sure a dictionary with numbers as keys and objects as values can make the round trip. Because JSON must coerce integers to strings in dict keys, the sort order may have a tendency to change between pickling and unpickling, and this could affect the object references. """ one = Thing('one') two = Thing('two') twelve = Thing('twelve') two.child = twelve obj = { 1: one, 2: two, 12: twelve, } self.assertNotEqual(list(sorted(obj.keys())), list(map(int, sorted(map(str, obj.keys()))))) flattened = self.pickler.flatten(obj) inflated = self.unpickler.restore(flattened) self.assertEqual(len(inflated), 3) self.assertEqual(inflated['12'].name, 'twelve') def test_builtin_error(self): expect = AssertionError json = jsonpickle.encode(expect) actual = jsonpickle.decode(json) self.assertEqual(expect, actual) self.assertTrue(expect is actual) def test_builtin_function(self): expect = dir json = jsonpickle.encode(expect) actual = jsonpickle.decode(json) self.assertEqual(expect, actual) self.assertTrue(expect is actual) def test_restore_legacy_builtins(self): """ jsonpickle 0.9.6 and earlier used the Python 2 `__builtin__` naming for builtins. Ensure those can be loaded until they're no longer supported. """ ae = jsonpickle.decode('{"py/type": "__builtin__.AssertionError"}') assert ae is AssertionError ae = jsonpickle.decode('{"py/type": "exceptions.AssertionError"}') assert ae is AssertionError cls = jsonpickle.decode('{"py/type": "__builtin__.int"}') assert cls is int class JSONPickleTestCase(SkippableTest): def setUp(self): self.obj = Thing('A name') self.expected_json = ( '{"%s": "jsonpickle_test.Thing", "name": "A name", "child": null}' % tags.OBJECT ) def test_API_names(self): """ Enforce expected names in main module """ names = list(vars(jsonpickle)) self.assertIn('pickler', names) self.assertIn('unpickler', names) self.assertIn('JSONBackend', names) self.assertIn('__version__', names) self.assertIn('register', names) self.assertIn('unregister', names) self.assertIn('Pickler', names) self.assertIn('Unpickler', names) self.assertIn('encode', names) self.assertIn('decode', names) def test_encode(self): expect = self.obj pickle = jsonpickle.encode(self.obj) actual = jsonpickle.decode(pickle) self.assertEqual(expect.name, actual.name) self.assertEqual(expect.child, actual.child) def test_encode_notunpicklable(self): expect = {'name': 'A name', 'child': None} pickle = jsonpickle.encode(self.obj, unpicklable=False) actual = jsonpickle.decode(pickle) self.assertEqual(expect['name'], actual['name']) def test_decode(self): actual = jsonpickle.decode(self.expected_json) self.assertEqual(self.obj.name, actual.name) self.assertEqual(type(self.obj), type(actual)) def test_json(self): expect = self.obj pickle = jsonpickle.encode(self.obj) actual = jsonpickle.decode(pickle) self.assertEqual(actual.name, expect.name) self.assertEqual(actual.child, expect.child) actual = jsonpickle.decode(self.expected_json) self.assertEqual(self.obj.name, actual.name) self.assertEqual(type(self.obj), type(actual)) def test_unicode_dict_keys(self): if PY2: uni = unichr(0x1234) # noqa else: uni = chr(0x1234) pickle = jsonpickle.encode({uni: uni}) actual = jsonpickle.decode(pickle) self.assertTrue(uni in actual) self.assertEqual(actual[uni], uni) def test_tuple_dict_keys_default(self): """Test that we handle dictionaries with tuples as keys.""" tuple_dict = {(1, 2): 3, (4, 5): {(7, 8): 9}} pickle = jsonpickle.encode(tuple_dict) expect = {'(1, 2)': 3, '(4, 5)': {'(7, 8)': 9}} actual = jsonpickle.decode(pickle) self.assertEqual(expect, actual) tuple_dict = {(1, 2): [1, 2]} pickle = jsonpickle.encode(tuple_dict) actual = jsonpickle.decode(pickle) self.assertEqual(actual['(1, 2)'], [1, 2]) def test_tuple_dict_keys_with_keys_enabled(self): """Test that we handle dictionaries with tuples as keys.""" tuple_dict = {(1, 2): 3, (4, 5): {(7, 8): 9}} pickle = jsonpickle.encode(tuple_dict, keys=True) expect = tuple_dict actual = jsonpickle.decode(pickle, keys=True) self.assertEqual(expect, actual) tuple_dict = {(1, 2): [1, 2]} pickle = jsonpickle.encode(tuple_dict, keys=True) actual = jsonpickle.decode(pickle, keys=True) self.assertEqual(actual[(1, 2)], [1, 2]) def test_None_dict_key_default(self): # We do string coercion for non-string keys so None becomes 'None' expect = {'null': None} obj = {None: None} pickle = jsonpickle.encode(obj) actual = jsonpickle.decode(pickle) self.assertEqual(expect, actual) def test_None_dict_key_with_keys_enabled(self): expect = {None: None} obj = {None: None} pickle = jsonpickle.encode(obj, keys=True) actual = jsonpickle.decode(pickle, keys=True) self.assertEqual(expect, actual) def test_object_dict_keys(self): """Test that we handle random objects as keys. """ thing = Thing('random') pickle = jsonpickle.encode({thing: True}) actual = jsonpickle.decode(pickle) self.assertEqual(actual, {'Thing("random")': True}) def test_int_dict_keys_defaults(self): int_dict = {1000: [1, 2]} pickle = jsonpickle.encode(int_dict) actual = jsonpickle.decode(pickle) self.assertEqual(actual['1000'], [1, 2]) def test_int_dict_keys_with_keys_enabled(self): int_dict = {1000: [1, 2]} pickle = jsonpickle.encode(int_dict, keys=True) actual = jsonpickle.decode(pickle, keys=True) self.assertEqual(actual[1000], [1, 2]) def test_string_key_requiring_escape_dict_keys_with_keys_enabled(self): json_key_dict = {tags.JSON_KEY + '6': [1, 2]} pickled = jsonpickle.encode(json_key_dict, keys=True) unpickled = jsonpickle.decode(pickled, keys=True) self.assertEqual(unpickled[tags.JSON_KEY + '6'], [1, 2]) def test_string_key_not_requiring_escape_dict_keys_with_keys_enabled(self): """test that string keys that do not require escaping are not escaped""" str_dict = {'name': [1, 2]} pickled = jsonpickle.encode(str_dict, keys=True) unpickled = jsonpickle.decode(pickled) self.assertTrue('name' in unpickled) def test_dict_subclass(self): obj = UserDict() obj.valid = True obj.s = 'string' obj.d = 'd_string' obj['d'] = {} obj['s'] = 'test' pickle = jsonpickle.encode(obj) actual = jsonpickle.decode(pickle) self.assertEqual(type(actual), UserDict) self.assertTrue('d' in actual) self.assertTrue('s' in actual) self.assertTrue(hasattr(actual, 'd')) self.assertTrue(hasattr(actual, 's')) self.assertTrue(hasattr(actual, 'valid')) self.assertEqual(obj['d'], actual['d']) self.assertEqual(obj['s'], actual['s']) self.assertEqual(obj.d, actual.d) self.assertEqual(obj.s, actual.s) self.assertEqual(obj.valid, actual.valid) def test_list_of_objects(self): """Test that objects in lists are referenced correctly""" a = Thing('a') b = Thing('b') pickle = jsonpickle.encode([a, b, b]) actual = jsonpickle.decode(pickle) self.assertEqual(actual[1], actual[2]) self.assertEqual(type(actual[0]), Thing) self.assertEqual(actual[0].name, 'a') self.assertEqual(actual[1].name, 'b') self.assertEqual(actual[2].name, 'b') def test_refs_keys_values(self): """Test that objects in dict keys are referenced correctly """ j = Thing('random') object_dict = {j: j} pickle = jsonpickle.encode(object_dict, keys=True) actual = jsonpickle.decode(pickle, keys=True) self.assertEqual(list(actual.keys()), list(actual.values())) def test_object_keys_to_list(self): """Test that objects in dict values are referenced correctly """ j = Thing('random') object_dict = {j: [j, j]} pickle = jsonpickle.encode(object_dict, keys=True) actual = jsonpickle.decode(pickle, keys=True) obj = list(actual.keys())[0] self.assertEqual(j.name, obj.name) self.assertTrue(obj is actual[obj][0]) self.assertTrue(obj is actual[obj][1]) def test_refs_in_objects(self): """Test that objects in lists are referenced correctly""" a = Thing('a') b = Thing('b') pickle = jsonpickle.encode([a, b, b]) actual = jsonpickle.decode(pickle) self.assertNotEqual(actual[0], actual[1]) self.assertEqual(actual[1], actual[2]) self.assertTrue(actual[1] is actual[2]) def test_refs_recursive(self): """Test that complicated recursive refs work""" a = Thing('a') a.self_list = [Thing('0'), Thing('1'), Thing('2')] a.first = a.self_list[0] a.stuff = {a.first: a.first} a.morestuff = {a.self_list[1]: a.stuff} pickle = jsonpickle.encode(a, keys=True) b = jsonpickle.decode(pickle, keys=True) item = b.self_list[0] self.assertEqual(b.first, item) self.assertEqual(b.stuff[b.first], item) self.assertEqual(b.morestuff[b.self_list[1]][b.first], item) def test_load_backend(self): """Test that we can call jsonpickle.load_backend() """ jsonpickle.load_backend('simplejson', 'dumps', 'loads', ValueError) self.assertTrue(True) def test_set_preferred_backend_allows_magic(self): """Tests that we can use the pluggable backends magically """ backend = 'os.path' jsonpickle.load_backend(backend, 'split', 'join', AttributeError) jsonpickle.set_preferred_backend(backend) slash_hello, world = jsonpickle.encode('/hello/world') jsonpickle.remove_backend(backend) self.assertEqual(slash_hello, '/hello') self.assertEqual(world, 'world') def test_load_backend_submodule(self): """Test that we can load a submodule as a backend """ jsonpickle.load_backend('os.path', 'split', 'join', AttributeError) self.assertTrue('os.path' in jsonpickle.json._backend_names and 'os.path' in jsonpickle.json._encoders and 'os.path' in jsonpickle.json._decoders and 'os.path' in jsonpickle.json._encoder_options and 'os.path' in jsonpickle.json._decoder_exceptions) def _backend_is_partially_loaded(self, backend): """Return True if the specified backend is incomplete""" return (backend in jsonpickle.json._backend_names or backend in jsonpickle.json._encoders or backend in jsonpickle.json._decoders or backend in jsonpickle.json._encoder_options or backend in jsonpickle.json._decoder_exceptions) def test_load_backend_handles_bad_encode(self): """Test that we ignore bad encoders""" load_backend = jsonpickle.load_backend self.assertFalse(load_backend('os.path', 'bad!', 'split', AttributeError)) self.assertFalse(self._backend_is_partially_loaded('os.path')) def test_load_backend_raises_on_bad_decode(self): """Test that we ignore bad decoders""" load_backend = jsonpickle.load_backend self.assertFalse(load_backend('os.path', 'join', 'bad!', AttributeError)) self.assertFalse(self._backend_is_partially_loaded('os.path')) def test_load_backend_handles_bad_loads_exc(self): """Test that we ignore bad decoder exceptions""" load_backend = jsonpickle.load_backend self.assertFalse(load_backend('os.path', 'join', 'split', 'bad!')) self.assertFalse(self._backend_is_partially_loaded('os.path')) def test_list_item_reference(self): thing = Thing('parent') thing.child = Thing('child') thing.child.refs = [thing] encoded = jsonpickle.encode(thing) decoded = jsonpickle.decode(encoded) self.assertEqual(id(decoded.child.refs[0]), id(decoded)) def test_reference_to_list(self): thing = Thing('parent') thing.a = [1] thing.b = thing.a thing.b.append(thing.a) thing.b.append([thing.a]) encoded = jsonpickle.encode(thing) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.a[0], 1) self.assertEqual(decoded.b[0], 1) self.assertEqual(id(decoded.a), id(decoded.b)) self.assertEqual(id(decoded.a), id(decoded.a[1])) self.assertEqual(id(decoded.a), id(decoded.a[2][0])) def test_make_refs_disabled_list(self): obj_a = Thing('foo') obj_b = Thing('bar') coll = [obj_a, obj_b, obj_b] encoded = jsonpickle.encode(coll, make_refs=False) decoded = jsonpickle.decode(encoded) self.assertEqual(len(decoded), 3) self.assertTrue(decoded[0] is not decoded[1]) self.assertTrue(decoded[1] is not decoded[2]) def test_make_refs_disabled_reference_to_list(self): thing = Thing('parent') thing.a = [1] thing.b = thing.a thing.b.append(thing.a) thing.b.append([thing.a]) encoded = jsonpickle.encode(thing, make_refs=False) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.a[0], 1) self.assertEqual(decoded.b[0:3], '[1,') self.assertEqual(decoded.a[1][0:3], '[1,') self.assertEqual(decoded.a[2][0][0:3], '[1,') def test_can_serialize_inner_classes(self): class InnerScope(object): """Private class visible to this method only""" def __init__(self, name): self.name = name obj = InnerScope('test') encoded = jsonpickle.encode(obj) # Single class decoded = jsonpickle.decode(encoded, classes=InnerScope) self._test_inner_class(InnerScope, obj, decoded) # List of classes decoded = jsonpickle.decode(encoded, classes=[InnerScope]) self._test_inner_class(InnerScope, obj, decoded) # Tuple of classes decoded = jsonpickle.decode(encoded, classes=(InnerScope,)) self._test_inner_class(InnerScope, obj, decoded) # Set of classes decoded = jsonpickle.decode(encoded, classes={InnerScope}) self._test_inner_class(InnerScope, obj, decoded) def _test_inner_class(self, InnerScope, obj, decoded): self.assertTrue(isinstance(obj, InnerScope)) self.assertEqual(decoded.name, obj.name) def test_can_serialize_nested_classes(self): if PY2: return self.skip('Serialization of nested classes requires ' 'Python >= 3') middle = Outer.Middle inner = Outer.Middle.Inner encoded_middle = jsonpickle.encode(middle) encoded_inner = jsonpickle.encode(inner) decoded_middle = jsonpickle.decode(encoded_middle) decoded_inner = jsonpickle.decode(encoded_inner) self.assertTrue(isinstance(decoded_middle, type)) self.assertTrue(isinstance(decoded_inner, type)) self.assertEqual(decoded_middle, middle) self.assertEqual(decoded_inner, inner) def test_can_serialize_nested_class_objects(self): if PY2: return self.skip('Serialization of nested classes requires ' 'Python >= 3') middle_obj = Outer.Middle() middle_obj.attribute = 5 inner_obj = Outer.Middle.Inner() inner_obj.attribute = 6 encoded_middle_obj = jsonpickle.encode(middle_obj) encoded_inner_obj = jsonpickle.encode(inner_obj) decoded_middle_obj = jsonpickle.decode(encoded_middle_obj) decoded_inner_obj = jsonpickle.decode(encoded_inner_obj) self.assertTrue(isinstance(decoded_middle_obj, Outer.Middle)) self.assertTrue(isinstance(decoded_inner_obj, Outer.Middle.Inner)) self.assertEqual(decoded_middle_obj.attribute, middle_obj.attribute) self.assertEqual(decoded_inner_obj.attribute, inner_obj.attribute) class PicklableNamedTuple(object): """ A picklable namedtuple wrapper, to demonstrate the need for protocol 2 compatibility. Yes, this is contrived in its use of new, but it demonstrates the issue. """ def __new__(cls, propnames, vals): # it's necessary to use the correct class name for class resolution # classes that fake their own names may never be unpicklable ntuple = collections.namedtuple(cls.__name__, propnames) ntuple.__getnewargs__ = (lambda self: (propnames, vals)) instance = ntuple.__new__(ntuple, *vals) return instance class PicklableNamedTupleEx(object): """ A picklable namedtuple wrapper, to demonstrate the need for protocol 4 compatibility. Yes, this is contrived in its use of new, but it demonstrates the issue. """ def __getnewargs__(self): raise NotImplementedError("This class needs __getnewargs_ex__") def __new__(cls, newargs=__getnewargs__, **kwargs): # it's necessary to use the correct class name for class resolution # classes that fake their own names may never be unpicklable ntuple = collections.namedtuple(cls.__name__, sorted(kwargs.keys())) ntuple.__getnewargs_ex__ = (lambda self: ((), kwargs)) ntuple.__getnewargs__ = newargs instance = ntuple.__new__(ntuple, *[b for a, b in sorted(kwargs.items())]) return instance class PickleProtocol2Thing(object): def __init__(self, *args): self.args = args def __getnewargs__(self): return self.args def __eq__(self, other): """ Make PickleProtocol2Thing('slotmagic') == PickleProtocol2Thing('slotmagic') """ if self.__dict__ == other.__dict__ and dir(self) == dir(other): for prop in dir(self): selfprop = getattr(self, prop) if not callable(selfprop) and prop[0] != '_': if selfprop != getattr(other, prop): return False return True else: return False # these two instances are used below and in tests slotmagic = PickleProtocol2Thing('slotmagic') dictmagic = PickleProtocol2Thing('dictmagic') class PickleProtocol2GetState(PickleProtocol2Thing): def __new__(cls, *args): instance = super(PickleProtocol2GetState, cls).__new__(cls) instance.newargs = args return instance def __getstate__(self): return 'I am magic' class PickleProtocol2GetStateDict(PickleProtocol2Thing): def __getstate__(self): return {'magic': True} class PickleProtocol2GetStateSlots(PickleProtocol2Thing): def __getstate__(self): return (None, {'slotmagic': slotmagic}) class PickleProtocol2GetStateSlotsDict(PickleProtocol2Thing): def __getstate__(self): return ({'dictmagic': dictmagic}, {'slotmagic': slotmagic}) class PickleProtocol2GetSetState(PickleProtocol2GetState): def __setstate__(self, state): """ Contrived example, easy to test """ if state == "I am magic": self.magic = True else: self.magic = False class PickleProtocol2ChildThing(object): def __init__(self, child): self.child = child def __getnewargs__(self): return ([self.child],) class PickleProtocol2ReduceString(object): def __reduce__(self): return __name__+'.slotmagic' class PickleProtocol2ReduceExString(object): def __reduce__(self): assert False, "Should not be here" def __reduce_ex__(self, n): return __name__+'.slotmagic' class PickleProtocol2ReduceTuple(object): def __init__(self, argval, optional=None): self.argval = argval self.optional = optional def __reduce__(self): return (PickleProtocol2ReduceTuple, # callable ('yam', 1), # args None, # state iter([]), # listitems iter([]), # dictitems ) @compat.iterator class ReducibleIterator(object): def __iter__(self): return self def __next__(self): raise StopIteration() def __reduce__(self): return ReducibleIterator, () def protocol_2_reduce_tuple_func(*args): return PickleProtocol2ReduceTupleFunc(*args) class PickleProtocol2ReduceTupleFunc(object): def __init__(self, argval, optional=None): self.argval = argval self.optional = optional def __reduce__(self): return (protocol_2_reduce_tuple_func, # callable ('yam', 1), # args None, # state iter([]), # listitems iter([]), # dictitems ) def __newobj__(lol, fail): """ newobj is special-cased, such that it is not actually called """ class PickleProtocol2ReduceNewobj(PickleProtocol2ReduceTupleFunc): def __new__(cls, *args): inst = super(cls, cls).__new__(cls) inst.newargs = args return inst def __reduce__(self): return (__newobj__, # callable (PickleProtocol2ReduceNewobj, 'yam', 1), # args None, # state iter([]), # listitems iter([]), # dictitems ) class PickleProtocol2ReduceTupleState(PickleProtocol2ReduceTuple): def __reduce__(self): return (PickleProtocol2ReduceTuple, # callable ('yam', 1), # args {'foo': 1}, # state iter([]), # listitems iter([]), # dictitems ) class PickleProtocol2ReduceTupleSetState(PickleProtocol2ReduceTuple): def __setstate__(self, state): self.bar = state['foo'] def __reduce__(self): return (type(self), # callable ('yam', 1), # args {'foo': 1}, # state iter([]), # listitems iter([]), # dictitems ) class PickleProtocol2ReduceTupleStateSlots(object): __slots__ = ('argval', 'optional', 'foo') def __init__(self, argval, optional=None): self.argval = argval self.optional = optional def __reduce__(self): return (PickleProtocol2ReduceTuple, # callable ('yam', 1), # args {'foo': 1}, # state iter([]), # listitems iter([]), # dictitems ) class PickleProtocol2ReduceListitemsAppend(object): def __init__(self): self.inner = [] def __reduce__(self): return (PickleProtocol2ReduceListitemsAppend, # callable (), # args {}, # state iter(['foo', 'bar']), # listitems iter([]), # dictitems ) def append(self, item): self.inner.append(item) class PickleProtocol2ReduceListitemsExtend(object): def __init__(self): self.inner = [] def __reduce__(self): return (PickleProtocol2ReduceListitemsAppend, # callable (), # args {}, # state iter(['foo', 'bar']), # listitems iter([]), # dictitems ) def extend(self, items): self.inner.exend(items) class PickleProtocol2ReduceDictitems(object): def __init__(self): self.inner = {} def __reduce__(self): return (PickleProtocol2ReduceDictitems, # callable (), # args {}, # state [], # listitems iter(zip(['foo', 'bar'], ['foo', 'bar'])), # dictitems ) def __setitem__(self, k, v): return self.inner.__setitem__(k, v) class PickleProtocol2Classic: def __init__(self, foo): self.foo = foo class PickleProtocol2ClassicInitargs: def __init__(self, foo, bar=None): self.foo = foo if bar: self.bar = bar def __getinitargs__(self): return ('choo', 'choo') class PicklingProtocol4TestCase(unittest.TestCase): def test_pickle_newargs_ex(self): """ Ensure we can pickle and unpickle an object whose class needs arguments to __new__ and get back the same typle """ instance = PicklableNamedTupleEx(**{'a': 'b', 'n': 2}) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(instance, decoded) def test_validate_reconstruct_by_newargs_ex(self): """ Ensure that the exemplar tuple's __getnewargs_ex__ works This is necessary to know whether the breakage exists in jsonpickle or not """ instance = PicklableNamedTupleEx(**{'a': 'b', 'n': 2}) args, kwargs = instance.__getnewargs_ex__() newinstance = PicklableNamedTupleEx.__new__(PicklableNamedTupleEx, *args, **kwargs) self.assertEqual(instance, newinstance) def test_references(self): shared = Thing('shared') instance = PicklableNamedTupleEx(**{'a': shared, 'n': shared}) child = Thing('child') shared.child = child child.child = instance encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded[0], decoded[1]) self.assertTrue(decoded[0] is decoded[1]) self.assertTrue(decoded.a is decoded.n) self.assertEqual(decoded.a.name, 'shared') self.assertEqual(decoded.a.child.name, 'child') self.assertTrue(decoded.a.child.child is decoded) self.assertTrue(decoded.n.child.child is decoded) self.assertTrue(decoded.a.child is decoded.n.child) self.assertEqual(decoded.__class__.__name__, PicklableNamedTupleEx.__name__) # TODO the class itself looks just like the real class, but it's # actually a reconstruction; PicklableNamedTupleEx is not type(decoded). self.assertFalse(decoded.__class__ is PicklableNamedTupleEx) class PicklingProtocol2TestCase(SkippableTest): def test_classic_init_has_args(self): """ Test unpickling a classic instance whose init takes args, has no __getinitargs__ Because classic only exists under 2, skipped if PY3 """ if PY3: return self.skip('No classic classes in PY3') instance = PickleProtocol2Classic(3) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.foo, 3) def test_getinitargs(self): """ Test __getinitargs__ with classic instance Because classic only exists under 2, skipped if PY3 """ if PY3: return self.skip('No classic classes in PY3') instance = PickleProtocol2ClassicInitargs(3) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.bar, 'choo') def test_reduce_complex_num(self): instance = 5j encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded, instance) def test_reduce_complex_zero(self): instance = 0j encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded, instance) def test_reduce_dictitems(self): 'Test reduce with dictitems set (as a generator)' instance = PickleProtocol2ReduceDictitems() encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.inner, {'foo': 'foo', 'bar': 'bar'}) def test_reduce_listitems_extend(self): 'Test reduce with listitems set (as a generator), yielding single items' instance = PickleProtocol2ReduceListitemsExtend() encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.inner, ['foo', 'bar']) def test_reduce_listitems_append(self): 'Test reduce with listitems set (as a generator), yielding single items' instance = PickleProtocol2ReduceListitemsAppend() encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.inner, ['foo', 'bar']) def test_reduce_state_setstate(self): 'Test reduce with the optional state argument set, on an object with '\ 'a __setstate__' instance = PickleProtocol2ReduceTupleSetState(5) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.argval, 'yam') self.assertEqual(decoded.optional, 1) self.assertEqual(decoded.bar, 1) self.assertFalse(hasattr(decoded, 'foo')) def test_reduce_state_no_dict(self): 'Test reduce with the optional state argument set, on an object with '\ 'no __dict__, and no __setstate__' instance = PickleProtocol2ReduceTupleStateSlots(5) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.argval, 'yam') self.assertEqual(decoded.optional, 1) self.assertEqual(decoded.foo, 1) def test_reduce_state_dict(self): 'Test reduce with the optional state argument set, on an object with '\ 'a __dict__, and no __setstate__' instance = PickleProtocol2ReduceTupleState(5) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.argval, 'yam') self.assertEqual(decoded.optional, 1) self.assertEqual(decoded.foo, 1) def test_reduce_basic(self): """Test reduce with only callable and args""" instance = PickleProtocol2ReduceTuple(5) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.argval, 'yam') self.assertEqual(decoded.optional, 1) def test_reduce_basic_func(self): """Test reduce with only callable and args callable is a module-level function """ instance = PickleProtocol2ReduceTupleFunc(5) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.argval, 'yam') self.assertEqual(decoded.optional, 1) def test_reduce_newobj(self): """Test reduce with callable called __newobj__ ensures special-case behaviour """ instance = PickleProtocol2ReduceNewobj(5) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.newargs, ('yam', 1)) def test_reduce_iter(self): instance = iter('123') self.assertTrue(util.is_iterator(instance)) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(next(decoded), '1') self.assertEqual(next(decoded), '2') self.assertEqual(next(decoded), '3') def test_reduce_iterable(self): """ An reducible object should be pickleable even if it is also iterable. """ instance = ReducibleIterator() self.assertTrue(util.is_iterator(instance)) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertIsInstance(decoded, ReducibleIterator) def test_reduce_string(self): """ Ensure json pickle will accept the redirection to another object when __reduce__ returns a string """ instance = PickleProtocol2ReduceString() encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded, slotmagic) def test_reduce_ex_string(self): """ Ensure json pickle will accept the redirection to another object when __reduce_ex__ returns a string ALSO tests that __reduce_ex__ is called in preference to __reduce__ """ instance = PickleProtocol2ReduceExString() encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded, slotmagic) def test_pickle_newargs(self): """ Ensure we can pickle and unpickle an object whose class needs arguments to __new__ and get back the same typle """ instance = PicklableNamedTuple(('a', 'b'), (1, 2)) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(instance, decoded) def test_validate_reconstruct_by_newargs(self): """ Ensure that the exemplar tuple's __getnewargs__ works This is necessary to know whether the breakage exists in jsonpickle or not """ instance = PicklableNamedTuple(('a', 'b'), (1, 2)) newinstance = PicklableNamedTuple.__new__(PicklableNamedTuple, *(instance.__getnewargs__())) self.assertEqual(instance, newinstance) def test_getnewargs_priority(self): """ Ensure newargs are used before py/state when decoding (As per PEP 307, classes are not supposed to implement all three magic methods) """ instance = PickleProtocol2GetState('whatevs') encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.newargs, ('whatevs',)) def test_restore_dict_state(self): """ Ensure that if getstate returns a dict, and there is no custom __setstate__, the dict is used as a source of variables to restore """ instance = PickleProtocol2GetStateDict('whatevs') encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertTrue(decoded.magic) def test_restore_slots_state(self): """ Ensure that if getstate returns a 2-tuple with a dict in the second position, and there is no custom __setstate__, the dict is used as a source of variables to restore """ instance = PickleProtocol2GetStateSlots('whatevs') encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(decoded.slotmagic.__dict__, slotmagic.__dict__) self.assertEqual(decoded.slotmagic, slotmagic) def test_restore_slots_dict_state(self): """ Ensure that if getstate returns a 2-tuple with a dict in both positions, and there is no custom __setstate__, the dicts are used as a source of variables to restore """ instance = PickleProtocol2GetStateSlotsDict('whatevs') encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(PickleProtocol2Thing('slotmagic'), PickleProtocol2Thing('slotmagic')) self.assertEqual(decoded.slotmagic.__dict__, slotmagic.__dict__) self.assertEqual(decoded.slotmagic, slotmagic) self.assertEqual(decoded.dictmagic, dictmagic) def test_setstate(self): """ Ensure output of getstate is passed to setstate """ instance = PickleProtocol2GetSetState('whatevs') encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertTrue(decoded.magic) def test_handles_nested_objects(self): child = PickleProtocol2Thing(None) instance = PickleProtocol2Thing(child, child) encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertEqual(PickleProtocol2Thing, decoded.__class__) self.assertEqual(PickleProtocol2Thing, decoded.args[0].__class__) self.assertEqual(PickleProtocol2Thing, decoded.args[1].__class__) self.assertTrue(decoded.args[0] is decoded.args[1]) def test_handles_cyclical_objects(self): child = Capture(None) instance = Capture(child, child) child.args = (instance,) # create a cycle # TODO we do not properly restore references inside of lists. # Change the above tuple into a list to show the breakage. encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) # Ensure the right objects were constructed self.assertEqual(Capture, decoded.__class__) self.assertEqual(Capture, decoded.args[0].__class__) self.assertEqual(Capture, decoded.args[1].__class__) self.assertEqual(Capture, decoded.args[0].args[0].__class__) self.assertEqual(Capture, decoded.args[1].args[0].__class__) # It's turtles all the way down self.assertEqual(Capture, decoded.args[0].args[0] .args[0].args[0] .args[0].args[0] .args[0].args[0] .args[0].args[0] .args[0].args[0] .args[0].args[0] .args[0].__class__) # Ensure that references are properly constructed self.assertTrue(decoded.args[0] is decoded.args[1]) self.assertTrue(decoded is decoded.args[0].args[0]) self.assertTrue(decoded is decoded.args[1].args[0]) self.assertTrue(decoded.args[0] is decoded.args[0].args[0].args[0]) self.assertTrue(decoded.args[0] is decoded.args[1].args[0].args[0]) def test_handles_cyclical_objects_in_lists(self): child = PickleProtocol2ChildThing(None) instance = PickleProtocol2ChildThing([child, child]) child.child = instance # create a cycle encoded = jsonpickle.encode(instance) decoded = jsonpickle.decode(encoded) self.assertTrue(decoded is decoded.child[0].child) self.assertTrue(decoded is decoded.child[1].child) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(JSONPickleTestCase)) suite.addTest(unittest.makeSuite(PicklingTestCase)) suite.addTest(unittest.makeSuite(PicklingProtocol2TestCase)) suite.addTest(unittest.makeSuite(PicklingProtocol4TestCase)) suite.addTest(doctest.DocTestSuite(jsonpickle)) suite.addTest(doctest.DocTestSuite(jsonpickle.pickler)) suite.addTest(doctest.DocTestSuite(jsonpickle.unpickler)) return suite if __name__ == '__main__': unittest.main(defaultTest='suite') jsonpickle-1.2/tests/numpy_test.py0000664000372000037200000003060213471677136020322 0ustar travistravis00000000000000from __future__ import absolute_import, division, unicode_literals import unittest import datetime import warnings import jsonpickle from jsonpickle import handlers from jsonpickle.compat import PY2 from helper import SkippableTest try: import numpy as np import numpy.testing as npt from numpy.compat import asbytes from numpy.testing import assert_equal from jsonpickle.ext import numpy as numpy_ext except ImportError: np = None class NumpyTestCase(SkippableTest): def setUp(self): if np is None: self.should_skip = True return self.should_skip = False import jsonpickle.ext.numpy jsonpickle.ext.numpy.register_handlers() def tearDown(self): if self.should_skip: return import jsonpickle.ext.numpy jsonpickle.ext.numpy.unregister_handlers() def roundtrip(self, obj): return jsonpickle.decode(jsonpickle.encode(obj)) def test_dtype_roundtrip(self): if self.should_skip: return self.skip('numpy is not importable') dtypes = [ np.int, np.float, np.complex, np.int32, np.str, np.object, np.unicode, np.dtype('f4,i4,f2,i1'), np.dtype(('f4', 'i4'), ('f2', 'i1')), np.dtype('1i4', align=True), np.dtype('M8[7D]'), np.dtype({'names': ['f0', 'f1', 'f2'], 'formats': ['f4', (64, 64)), (1,)), ('rtile', '>f4', (64, 36))], (3,)), ('bottom', [('bleft', ('>f4', (8, 64)), (1,)), ('bright', '>f4', (8, 36))])]), ]) for dtype in dtypes: self.assertEqual(self.roundtrip(dtype), dtype) def test_generic_roundtrip(self): if self.should_skip: return self.skip('numpy is not importable') values = [ np.int_(1), np.int32(-2), np.float_(2.5), np.nan, -np.inf, np.inf, np.datetime64('2014-01-01'), np.str_('foo'), np.unicode_('bar'), np.object_({'a': 'b'}), np.complex_(1 - 2j) ] for value in values: decoded = self.roundtrip(value) assert_equal(decoded, value) self.assertTrue(isinstance(decoded, type(value))) def test_ndarray_roundtrip(self): if self.should_skip: return self.skip('numpy is not importable') arrays = [ np.random.random((10, 20)), np.array([[True, False, True]]), np.array(['foo', 'bar']), np.array(['baz'.encode('utf-8')]), np.array(['2010', 'NaT', '2030']).astype('M'), np.rec.array(asbytes('abcdefg') * 100, formats='i2,a3,i4', shape=3), np.rec.array([(1, 11, 'a'), (2, 22, 'b'), (3, 33, 'c'), (4, 44, 'd'), (5, 55, 'ex'), (6, 66, 'f'), (7, 77, 'g')], formats='u1,f4,a1'), np.array(['1960-03-12', datetime.date(1960, 3, 12)], dtype='M8[D]'), np.array([0, 1, -1, np.inf, -np.inf, np.nan], dtype='f2'), ] if not PY2: arrays.extend([ np.rec.array([ ('NGC1001', 11), ('NGC1002', 1.), ('NGC1003', 1.)], dtype=[('target', 'S20'), ('V_mag', 'f4')]) ]) for array in arrays: decoded = self.roundtrip(array) assert_equal(decoded, array) self.assertEqual(decoded.dtype, array.dtype) def test_shapes_containing_zeroes(self): """Test shapes which cannot be represented as nested lists""" if self.should_skip: return self.skip('numpy is not importable') a = np.eye(3)[3:] _a = self.roundtrip(a) npt.assert_array_equal(a, _a) def test_accuracy(self): """Test the accuracy of the string representation""" if self.should_skip: return self.skip('numpy is not importable') rand = np.random.randn(3, 3) _rand = self.roundtrip(rand) npt.assert_array_equal(rand, _rand) def test_b64(self): """Test the binary encoding""" if self.should_skip: return self.skip('numpy is not importable') a = np.random.rand(10, 10) # array of substantial size is stored as b64 _a = self.roundtrip(a) npt.assert_array_equal(a, _a) def test_views(self): """Test views under serialization""" if self.should_skip: return self.skip('numpy is not importable') rng = np.arange(20) # a range of an array view = rng[10:] # a view referencing a portion of an array data = [rng, view] _data = self.roundtrip(data) _data[0][15] = -1 self.assertEqual(_data[1][5], -1) def test_strides(self): """Test non-standard strides and offsets""" if self.should_skip: return self.skip('numpy is not importable') arr = np.eye(3) view = arr[1:, 1:] self.assertTrue(view.base is arr) data = [arr, view] _data = self.roundtrip(data) # test that the deserialized arrays indeed view the same memory _arr, _view = _data _arr[1, 2] = -1 self.assertEqual(_view[0, 1], -1) self.assertTrue(_view.base is _arr) def test_weird_arrays(self): """Test references to arrays that do not effectively own their memory""" if self.should_skip: return self.skip('numpy is not importable') a = np.arange(9) b = a[5:] a.strides = 1 # this is kinda fishy; a has overlapping memory, _a does not if PY2: warn_count = 0 else: warn_count = 1 with warnings.catch_warnings(record=True) as w: _a = self.roundtrip(a) self.assertEqual(len(w), warn_count) npt.assert_array_equal(a, _a) # this also requires a deepcopy to work with warnings.catch_warnings(record=True) as w: _a, _b = self.roundtrip([a, b]) self.assertEqual(len(w), warn_count) npt.assert_array_equal(a, _a) npt.assert_array_equal(b, _b) def test_transpose(self): """test handling of non-c-contiguous memory layout""" if self.should_skip: return self.skip('numpy is not importable') # simple case; view a c-contiguous array a = np.arange(9).reshape(3, 3) b = a[1:, 1:] self.assertTrue(b.base is a.base) _a, _b = self.roundtrip([a, b]) self.assertTrue(_b.base is _a.base) npt.assert_array_equal(a, _a) npt.assert_array_equal(b, _b) # a and b both view the same contiguous array a = np.arange(9).reshape(3, 3).T b = a[1:, 1:] self.assertTrue(b.base is a.base) _a, _b = self.roundtrip([a, b]) self.assertTrue(_b.base is _a.base) npt.assert_array_equal(a, _a) npt.assert_array_equal(b, _b) # view an f-contiguous array a = a.copy() a.strides = a.strides[::-1] b = a[1:, 1:] self.assertTrue(b.base is a) _a, _b = self.roundtrip([a, b]) self.assertTrue(_b.base is _a) npt.assert_array_equal(a, _a) npt.assert_array_equal(b, _b) # now a.data.contiguous is False; we have to make a deepcopy to make # this work note that this is a pretty contrived example though! a = np.arange(8).reshape(2, 2, 2).copy() a.strides = a.strides[0], a.strides[2], a.strides[1] b = a[1:, 1:] self.assertTrue(b.base is a) if PY2: warn_count = 0 else: warn_count = 1 with warnings.catch_warnings(record=True) as w: _a, _b = self.roundtrip([a, b]) self.assertEqual(len(w), warn_count) npt.assert_array_equal(a, _a) npt.assert_array_equal(b, _b) def test_fortran_base(self): """Test a base array in fortran order""" if self.should_skip: return self.skip('numpy is not importable') a = np.asfortranarray(np.arange(100).reshape((10, 10))) _a = self.roundtrip(a) npt.assert_array_equal(a, _a) def test_buffer(self): """test behavior with memoryviews which are not ndarrays""" if self.should_skip: return self.skip('numpy is not importable') bstring = 'abcdefgh'.encode('utf-8') a = np.frombuffer(bstring, dtype=np.byte) if PY2: warn_count = 0 else: warn_count = 1 with warnings.catch_warnings(record=True) as w: _a = self.roundtrip(a) npt.assert_array_equal(a, _a) self.assertEqual(len(w), warn_count) def test_as_strided(self): """Test the result of as_strided() as_strided() returns an object that implements the array interface but is not an ndarray. """ if self.should_skip: return self.skip('numpy is not importable') a = np.arange(10) b = np.lib.stride_tricks.as_strided(a, shape=(5,), strides=(a.dtype.itemsize * 2,)) data = [a, b] with warnings.catch_warnings(record=True) as w: # as_strided returns a DummyArray object, which we can not # currently serialize correctly FIXME: would be neat to add # support for all objects implementing the __array_interface__ _data = self.roundtrip(data) self.assertEqual(len(w), 1) # as we were warned, deserialized result is no longer a view _data[0][0] = -1 self.assertEqual(_data[1][0], 0) def test_immutable(self): """test that immutability flag is copied correctly""" if self.should_skip: return self.skip('numpy is not importable') a = np.arange(10) a.flags.writeable = False _a = self.roundtrip(a) try: _a[0] = 0 self.assertTrue(False, 'item assignment must raise') except ValueError: self.assertTrue(True) def test_byteorder(self): """Test the byteorder for text and binary encodings""" if self.should_skip: return self.skip('numpy is not importable') # small arr is stored as text a = np.arange(10).newbyteorder() b = a[:].newbyteorder() _a, _b = self.roundtrip([a, b]) npt.assert_array_equal(a, _a) npt.assert_array_equal(b, _b) # bigger arr is stored as binary a = np.arange(100).newbyteorder() b = a[:].newbyteorder() _a, _b = self.roundtrip([a, b]) npt.assert_array_equal(a, _a) npt.assert_array_equal(b, _b) def test_zero_dimensional_array(self): if self.should_skip: return self.skip('numpy is not importable') self.roundtrip(np.array(0.0)) def test_nested_data_list_of_dict_with_list_keys(self): """Ensure we can handle numpy arrays within a nested structure""" obj = [{'key': [np.array(0)]}] self.roundtrip(obj) obj = [{'key': [np.array([1.0])]}] self.roundtrip(obj) def test_size_threshold_None(self): handler = numpy_ext.NumpyNDArrayHandlerView(size_threshold=None) handlers.registry.unregister(np.ndarray) handlers.registry.register(np.ndarray, handler, base=True) self.roundtrip(np.array([0, 1])) def test_ndarray_dtype_object(self): a = np.array(['F'+str(i) for i in range(30)], dtype=np.object) buf = jsonpickle.encode(a) # This is critical for reproducing the numpy segfault issue when # restoring ndarray of dtype object del a _a = jsonpickle.decode(buf) a = np.array(['F'+str(i) for i in range(30)], dtype=np.object) npt.assert_array_equal(a, _a) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(NumpyTestCase, 'test')) return suite if __name__ == '__main__': unittest.main() jsonpickle-1.2/tests/object_test.py0000664000372000037200000007135013471677136020425 0ustar travistravis00000000000000from __future__ import absolute_import, division, unicode_literals import enum import collections import decimal import re import unittest import datetime import jsonpickle from jsonpickle import compat from jsonpickle import handlers from jsonpickle import tags from jsonpickle import util from jsonpickle.compat import queue, PY2 from helper import SkippableTest class Thing(object): def __init__(self, name): self.name = name self.child = None class DictSubclass(dict): name = 'Test' class ListSubclass(list): pass class BrokenReprThing(Thing): def __repr__(self): raise Exception('%s has a broken repr' % self.name) def __str__(self): return '' % self.name class GetstateDict(dict): def __init__(self, name, **kwargs): dict.__init__(self, **kwargs) self.name = name self.active = False def __getstate__(self): return (self.name, dict(self.items())) def __setstate__(self, state): self.name, vals = state self.update(vals) self.active = True class GetstateOnly(object): def __init__(self, a=1, b=2): self.a = a self.b = b def __getstate__(self): return [self.a, self.b] class GetstateReturnsList(object): def __init__(self, x, y): self.x = x self.y = y def __getstate__(self): return [self.x, self.y] def __setstate__(self, state): self.x, self.y = state[0], state[1] class GetstateRecursesInfintely(object): def __getstate__(self): return GetstateRecursesInfintely() class ListSubclassWithInit(list): def __init__(self, attr): self.attr = attr super(ListSubclassWithInit, self).__init__() NamedTuple = collections.namedtuple('NamedTuple', 'a, b, c') class ObjWithJsonPickleRepr(object): def __init__(self): self.data = {'a': self} def __repr__(self): return jsonpickle.encode(self) class OldStyleClass: pass class SetSubclass(set): pass class ThingWithFunctionRefs(object): def __init__(self): self.fn = func def func(x): return x class ThingWithQueue(object): def __init__(self): self.child_1 = queue.Queue() self.child_2 = queue.Queue() self.childref_1 = self.child_1 self.childref_2 = self.child_2 class ThingWithSlots(object): __slots__ = ('a', 'b') def __init__(self, a, b): self.a = a self.b = b class ThingWithInheritedSlots(ThingWithSlots): __slots__ = ('c',) def __init__(self, a, b, c): ThingWithSlots.__init__(self, a, b) self.c = c class ThingWithIterableSlots(object): __slots__ = iter('ab') def __init__(self, a, b): self.a = a self.b = b class ThingWithStringSlots(object): __slots__ = 'ab' def __init__(self, a, b): self.ab = a + b class ThingWithSelfAsDefaultFactory(collections.defaultdict): """defaultdict subclass that uses itself as its default factory""" def __init__(self): self.default_factory = self def __call__(self): return self.__class__() class ThingWithClassAsDefaultFactory(collections.defaultdict): """defaultdict subclass that uses its class as its default factory""" def __init__(self): self.default_factory = self.__class__ def __call__(self): return self.__class__() class IntEnumTest(enum.IntEnum): X = 1 Y = 2 class StringEnumTest(enum.Enum): A = 'a' B = 'b' class SubEnum(enum.Enum): a = 1 b = 2 class EnumClass(object): def __init__(self): self.enum_a = SubEnum.a self.enum_b = SubEnum.b class MessageTypes(enum.Enum): STATUS = 'STATUS', CONTROL = 'CONTROL' class MessageStatus(enum.Enum): OK = 'OK', ERROR = 'ERROR' class MessageCommands(enum.Enum): STATUS_ALL = 'STATUS_ALL' class Message(object): def __init__(self, message_type, command, status=None, body=None): self.message_type = MessageTypes(message_type) if command: self.command = MessageCommands(command) if status: self.status = MessageStatus(status) if body: self.body = body class ThingWithTimedeltaAttribute(object): def __init__(self, offset): self.offset = datetime.timedelta(offset) def __getinitargs__(self): return self.offset, class IntKeysObject(object): def __init__(self): self.data = {0: 0} def __getstate__(self): return self.__dict__ class AdvancedObjectsTestCase(SkippableTest): def setUp(self): self.pickler = jsonpickle.pickler.Pickler() self.unpickler = jsonpickle.unpickler.Unpickler() def tearDown(self): self.pickler.reset() self.unpickler.reset() def test_defaultdict_roundtrip(self): """Make sure we can handle collections.defaultdict(list)""" # setup defaultdict = collections.defaultdict defdict = defaultdict(list) defdict['a'] = 1 defdict['b'].append(2) defdict['c'] = defaultdict(dict) # jsonpickle work your magic encoded = jsonpickle.encode(defdict) newdefdict = jsonpickle.decode(encoded) # jsonpickle never fails self.assertEqual(newdefdict['a'], 1) self.assertEqual(newdefdict['b'], [2]) self.assertEqual(type(newdefdict['c']), defaultdict) self.assertEqual(defdict.default_factory, list) self.assertEqual(newdefdict.default_factory, list) def test_defaultdict_roundtrip_simple_lambda(self): """Make sure we can handle defaultdict(lambda: defaultdict(int))""" # setup a sparse collections.defaultdict with simple lambdas defaultdict = collections.defaultdict defdict = defaultdict(lambda: defaultdict(int)) defdict[0] = 'zero' defdict[1] = defaultdict(lambda: defaultdict(dict)) defdict[1][0] = 'zero' # roundtrip encoded = jsonpickle.encode(defdict, keys=True) newdefdict = jsonpickle.decode(encoded, keys=True) self.assertEqual(newdefdict[0], 'zero') self.assertEqual(type(newdefdict[1]), defaultdict) self.assertEqual(newdefdict[1][0], 'zero') self.assertEqual(newdefdict[1][1], {}) # inner defaultdict self.assertEqual(newdefdict[2][0], 0) # outer defaultdict self.assertEqual(type(newdefdict[3]), defaultdict) # outer-most defaultdict self.assertEqual(newdefdict[3].default_factory, int) def test_defaultdict_subclass_with_self_as_default_factory(self): cls = ThingWithSelfAsDefaultFactory tree = cls() newtree = self._test_defaultdict_tree(tree, cls) self.assertEqual(type(newtree['A'].default_factory), cls) self.assertTrue(newtree.default_factory is newtree) self.assertTrue(newtree['A'].default_factory is newtree['A']) self.assertTrue(newtree['Z'].default_factory is newtree['Z']) def test_defaultdict_subclass_with_class_as_default_factory(self): cls = ThingWithClassAsDefaultFactory tree = cls() newtree = self._test_defaultdict_tree(tree, cls) self.assertTrue(newtree.default_factory is cls) self.assertTrue(newtree['A'].default_factory is cls) self.assertTrue(newtree['Z'].default_factory is cls) def _test_defaultdict_tree(self, tree, cls): tree['A']['B'] = 1 tree['A']['C'] = 2 # roundtrip encoded = jsonpickle.encode(tree) newtree = jsonpickle.decode(encoded) # make sure we didn't lose anything self.assertEqual(type(newtree), cls) self.assertEqual(type(newtree['A']), cls) self.assertEqual(newtree['A']['B'], 1) self.assertEqual(newtree['A']['C'], 2) # ensure that the resulting default_factory is callable and creates # a new instance of cls. self.assertEqual(type(newtree['A'].default_factory()), cls) # we've never seen 'D' before so the reconstructed defaultdict tree # should create an instance of cls. self.assertEqual(type(newtree['A']['D']), cls) # ensure that proxies do not escape into user code self.assertNotEqual(type(newtree.default_factory), jsonpickle.unpickler._Proxy) self.assertNotEqual(type(newtree['A'].default_factory), jsonpickle.unpickler._Proxy) self.assertNotEqual(type(newtree['A']['Z'].default_factory), jsonpickle.unpickler._Proxy) return newtree def test_deque_roundtrip(self): """Make sure we can handle collections.deque""" old_deque = collections.deque([0, 1, 2], maxlen=5) encoded = jsonpickle.encode(old_deque) new_deque = jsonpickle.decode(encoded) self.assertNotEqual(encoded, 'nil') self.assertEqual(old_deque[0], 0) self.assertEqual(new_deque[0], 0) self.assertEqual(old_deque[1], 1) self.assertEqual(new_deque[1], 1) self.assertEqual(old_deque[2], 2) self.assertEqual(new_deque[2], 2) self.assertEqual(old_deque.maxlen, 5) self.assertEqual(new_deque.maxlen, 5) def test_namedtuple_roundtrip(self): old_nt = NamedTuple(0, 1, 2) encoded = jsonpickle.encode(old_nt) new_nt = jsonpickle.decode(encoded) self.assertEqual(type(old_nt), type(new_nt)) self.assertTrue(old_nt is not new_nt) self.assertEqual(old_nt.a, new_nt.a) self.assertEqual(old_nt.b, new_nt.b) self.assertEqual(old_nt.c, new_nt.c) self.assertEqual(old_nt[0], new_nt[0]) self.assertEqual(old_nt[1], new_nt[1]) self.assertEqual(old_nt[2], new_nt[2]) def test_counter_roundtrip(self): counter = collections.Counter({1: 2}) encoded = jsonpickle.encode(counter) decoded = jsonpickle.decode(encoded) self.assertTrue(type(decoded) is collections.Counter) # the integer key becomes a string when keys=False self.assertEqual(decoded.get('1'), 2) def test_counter_roundtrip_with_keys(self): counter = collections.Counter({1: 2}) encoded = jsonpickle.encode(counter, keys=True) decoded = jsonpickle.decode(encoded, keys=True) self.assertTrue(type(decoded) is collections.Counter) self.assertEqual(decoded.get(1), 2) def test_list_with_fd(self): fd = open(__file__, 'r') fd.close() obj = [fd] jsonstr = jsonpickle.encode(obj) newobj = jsonpickle.decode(jsonstr) self.assertEqual([None], newobj) def test_thing_with_fd(self): fd = open(__file__, 'r') fd.close() obj = Thing(fd) jsonstr = jsonpickle.encode(obj) newobj = jsonpickle.decode(jsonstr) self.assertEqual(None, newobj.name) def test_dict_with_fd(self): fd = open(__file__, 'r') fd.close() obj = {'fd': fd} jsonstr = jsonpickle.encode(obj) newobj = jsonpickle.decode(jsonstr) self.assertEqual(None, newobj['fd']) def test_thing_with_lamda(self): obj = Thing(lambda: True) jsonstr = jsonpickle.encode(obj) newobj = jsonpickle.decode(jsonstr) self.assertFalse(hasattr(newobj, 'name')) def test_newstyleslots(self): obj = ThingWithSlots(True, False) jsonstr = jsonpickle.encode(obj) newobj = jsonpickle.decode(jsonstr) self.assertTrue(newobj.a) self.assertFalse(newobj.b) def test_newstyleslots_inherited(self): obj = ThingWithInheritedSlots(True, False, None) jsonstr = jsonpickle.encode(obj) newobj = jsonpickle.decode(jsonstr) self.assertTrue(newobj.a) self.assertFalse(newobj.b) self.assertEqual(newobj.c, None) def test_newstyleslots_inherited_deleted_attr(self): obj = ThingWithInheritedSlots(True, False, None) del obj.c jsonstr = jsonpickle.encode(obj) newobj = jsonpickle.decode(jsonstr) self.assertTrue(newobj.a) self.assertFalse(newobj.b) self.assertFalse(hasattr(newobj, 'c')) def test_newstyleslots_with_children(self): obj = ThingWithSlots(Thing('a'), Thing('b')) jsonstr = jsonpickle.encode(obj) newobj = jsonpickle.decode(jsonstr) self.assertEqual(newobj.a.name, 'a') self.assertEqual(newobj.b.name, 'b') def test_newstyleslots_with_children_inherited(self): obj = ThingWithInheritedSlots(Thing('a'), Thing('b'), Thing('c')) jsonstr = jsonpickle.encode(obj) newobj = jsonpickle.decode(jsonstr) self.assertEqual(newobj.a.name, 'a') self.assertEqual(newobj.b.name, 'b') self.assertEqual(newobj.c.name, 'c') def test_newstyleslots_iterable(self): obj = ThingWithIterableSlots('alpha', 'bravo') jsonstr = jsonpickle.encode(obj) newobj = jsonpickle.decode(jsonstr) self.assertEqual(newobj.a, 'alpha') self.assertEqual(newobj.b, 'bravo') def test_newstyleslots_string_slot(self): obj = ThingWithStringSlots('a', 'b') jsonstr = jsonpickle.encode(obj) newobj = jsonpickle.decode(jsonstr) self.assertEqual(newobj.ab, 'ab') def test_list_subclass(self): obj = ListSubclass() obj.extend([1, 2, 3]) flattened = self.pickler.flatten(obj) self.assertTrue(tags.OBJECT in flattened) self.assertTrue(tags.SEQ in flattened) self.assertEqual(len(flattened[tags.SEQ]), 3) for v in obj: self.assertTrue(v in flattened[tags.SEQ]) restored = self.unpickler.restore(flattened) self.assertEqual(type(restored), ListSubclass) self.assertEqual(restored, obj) def test_list_subclass_with_init(self): obj = ListSubclassWithInit('foo') self.assertEqual(obj.attr, 'foo') flattened = self.pickler.flatten(obj) inflated = self.unpickler.restore(flattened) self.assertEqual(type(inflated), ListSubclassWithInit) def test_list_subclass_with_data(self): obj = ListSubclass() obj.extend([1, 2, 3]) data = SetSubclass([1, 2, 3]) obj.data = data flattened = self.pickler.flatten(obj) restored = self.unpickler.restore(flattened) self.assertEqual(restored, obj) self.assertEqual(type(restored.data), SetSubclass) self.assertEqual(restored.data, data) def test_set_subclass(self): obj = SetSubclass([1, 2, 3]) flattened = self.pickler.flatten(obj) self.assertTrue(tags.OBJECT in flattened) self.assertTrue(tags.SEQ in flattened) self.assertEqual(len(flattened[tags.SEQ]), 3) for v in obj: self.assertTrue(v in flattened[tags.SEQ]) restored = self.unpickler.restore(flattened) self.assertEqual(type(restored), SetSubclass) self.assertEqual(restored, obj) def test_set_subclass_with_data(self): obj = SetSubclass([1, 2, 3]) data = ListSubclass() data.extend([1, 2, 3]) obj.data = data flattened = self.pickler.flatten(obj) restored = self.unpickler.restore(flattened) self.assertEqual(restored.data.__class__, ListSubclass) self.assertEqual(restored.data, data) def test_decimal(self): obj = decimal.Decimal('0.5') flattened = self.pickler.flatten(obj) inflated = self.unpickler.restore(flattened) self.assertTrue(isinstance(inflated, decimal.Decimal)) def test_repr_using_jsonpickle(self): thing = ObjWithJsonPickleRepr() thing.child = ObjWithJsonPickleRepr() thing.child.parent = thing encoded = jsonpickle.encode(thing) decoded = jsonpickle.decode(encoded) self.assertEqual(id(decoded), id(decoded.child.parent)) def test_broken_repr_dict_key(self): """Tests that we can pickle dictionaries with keys that have broken __repr__ implementations. """ br = BrokenReprThing('test') obj = {br: True} pickler = jsonpickle.pickler.Pickler() flattened = pickler.flatten(obj) self.assertTrue('' in flattened) self.assertTrue(flattened['']) def test_ordered_dict(self): d = collections.OrderedDict([('c', 3), ('a', 1), ('b', 2)]) encoded = jsonpickle.encode(d) decoded = jsonpickle.decode(encoded) self.assertEqual(d, decoded) def test_ordered_dict_unpicklable(self): d = collections.OrderedDict([('c', 3), ('a', 1), ('b', 2)]) encoded = jsonpickle.encode(d, unpicklable=False) decoded = jsonpickle.decode(encoded) self.assertEqual(d, decoded) def test_ordered_dict_reduces(self): d = collections.OrderedDict([('c', 3), ('a', 1), ('b', 2)]) has_reduce, has_reduce_ex = util.has_reduce(d) self.assertTrue(util.is_reducible(d)) self.assertTrue(has_reduce or has_reduce_ex) def test_int_keys_in_object_with_getstate_only(self): obj = IntKeysObject() encoded = jsonpickle.encode(obj, keys=True) decoded = jsonpickle.decode(encoded, keys=True) self.assertEqual(obj.data, decoded.data) def test_ordered_dict_int_keys(self): d = { 1: collections.OrderedDict([(2, -2), (3, -3)]), 4: collections.OrderedDict([(5, -5), (6, -6)]), } encoded = jsonpickle.encode(d, keys=True) decoded = jsonpickle.decode(encoded, keys=True) self.assertEqual(collections.OrderedDict, type(decoded[1])) self.assertEqual(collections.OrderedDict, type(decoded[4])) self.assertEqual(-2, decoded[1][2]) self.assertEqual(-3, decoded[1][3]) self.assertEqual(-5, decoded[4][5]) self.assertEqual(-6, decoded[4][6]) self.assertEqual(d, decoded) def test_ordered_dict_nested(self): bottom = collections.OrderedDict([('z', 1), ('a', 2)]) middle = collections.OrderedDict([('c', bottom)]) top = collections.OrderedDict([('b', middle)]) encoded = jsonpickle.encode(top) decoded = jsonpickle.decode(encoded) self.assertEqual(top, decoded) # test unpicklable=False encoded = jsonpickle.encode(top, unpicklable=False) decoded = jsonpickle.decode(encoded) self.assertEqual(top, decoded) def test_posix_stat_result(self): try: import posix except ImportError: return expect = posix.stat(__file__) encoded = jsonpickle.encode(expect) actual = jsonpickle.decode(encoded) self.assertEqual(expect, actual) def test_oldstyleclass(self): obj = OldStyleClass() obj.value = 1234 flattened = self.pickler.flatten(obj) self.assertEqual(1234, flattened['value']) inflated = self.unpickler.restore(flattened) self.assertEqual(1234, inflated.value) def test_dictsubclass(self): obj = DictSubclass() obj['key1'] = 1 expect = { tags.OBJECT: 'object_test.DictSubclass', 'key1': 1, '__dict__': {}, } flattened = self.pickler.flatten(obj) self.assertEqual(expect, flattened) inflated = self.unpickler.restore(flattened) self.assertEqual(type(inflated), DictSubclass) self.assertEqual(1, inflated['key1']) self.assertEqual(inflated.name, 'Test') def test_dictsubclass_notunpickable(self): self.pickler.unpicklable = False obj = DictSubclass() obj['key1'] = 1 flattened = self.pickler.flatten(obj) self.assertEqual(1, flattened['key1']) self.assertFalse(tags.OBJECT in flattened) inflated = self.unpickler.restore(flattened) self.assertEqual(1, inflated['key1']) def test_getstate_dict_subclass_structure(self): obj = GetstateDict('test') obj['key1'] = 1 flattened = self.pickler.flatten(obj) self.assertTrue(tags.OBJECT in flattened) self.assertEqual('object_test.GetstateDict', flattened[tags.OBJECT]) self.assertTrue(tags.STATE in flattened) self.assertTrue(tags.TUPLE in flattened[tags.STATE]) self.assertEqual(['test', {'key1': 1}], flattened[tags.STATE][tags.TUPLE]) def test_getstate_dict_subclass_roundtrip_simple(self): obj = GetstateDict('test') obj['key1'] = 1 flattened = self.pickler.flatten(obj) inflated = self.unpickler.restore(flattened) self.assertEqual(1, inflated['key1']) self.assertEqual(inflated.name, 'test') def test_getstate_dict_subclass_roundtrip_cyclical(self): obj = GetstateDict('test') obj['key1'] = 1 # The "name" field of obj2 points to obj (reference) obj2 = GetstateDict(obj) # The "obj2" key in obj points to obj2 (cyclical reference) obj['obj2'] = obj2 flattened = self.pickler.flatten(obj) inflated = self.unpickler.restore(flattened) # The dict must be preserved self.assertEqual(1, inflated['key1']) # __getstate__/__setstate__ must have been run self.assertEqual(inflated.name, 'test') self.assertEqual(inflated.active, True) self.assertEqual(inflated['obj2'].active, True) # The reference must be preserved self.assertTrue(inflated is inflated['obj2'].name) def test_getstate_list_simple(self): obj = GetstateReturnsList(1, 2) flattened = self.pickler.flatten(obj) inflated = self.unpickler.restore(flattened) self.assertEqual(inflated.x, 1) self.assertEqual(inflated.y, 2) def test_getstate_list_inside_list(self): obj1 = GetstateReturnsList(1, 2) obj2 = GetstateReturnsList(3, 4) obj = [obj1, obj2] flattened = self.pickler.flatten(obj) inflated = self.unpickler.restore(flattened) self.assertEqual(inflated[0].x, 1) self.assertEqual(inflated[0].y, 2) self.assertEqual(inflated[1].x, 3) self.assertEqual(inflated[1].y, 4) def test_getstate_with_getstate_only(self): obj = GetstateOnly() a = obj.a = 'this object implements' b = obj.b = '__getstate__ but not __setstate__' expect = [a, b] flat = self.pickler.flatten(obj) actual = flat[tags.STATE] self.assertEqual(expect, actual) restored = self.unpickler.restore(flat) self.assertEqual(expect, restored) def test_getstate_does_not_recurse_infinitely(self): obj = GetstateRecursesInfintely() pickler = jsonpickle.pickler.Pickler(max_depth=5) pickler.flatten(obj) def test_thing_with_queue(self): obj = ThingWithQueue() flattened = self.pickler.flatten(obj) restored = self.unpickler.restore(flattened) self.assertEqual(type(restored.child_1), type(queue.Queue())) self.assertEqual(type(restored.child_2), type(queue.Queue())) # Check references self.assertTrue(restored.child_1 is restored.childref_1) self.assertTrue(restored.child_2 is restored.childref_2) def test_thing_with_func(self): obj = ThingWithFunctionRefs() obj.ref = obj flattened = self.pickler.flatten(obj) restored = self.unpickler.restore(flattened) self.assertTrue(restored.fn is obj.fn) expect = 'success' actual1 = restored.fn(expect) self.assertEqual(expect, actual1) self.assertTrue(restored is restored.ref) def test_thing_with_compiled_regex(self): rgx = re.compile(r'(.*)(cat)') obj = Thing(rgx) flattened = self.pickler.flatten(obj) restored = self.unpickler.restore(flattened) match = restored.name.match('fatcat') self.assertEqual('fat', match.group(1)) self.assertEqual('cat', match.group(2)) def test_base_object_roundrip(self): roundtrip = self.unpickler.restore(self.pickler.flatten(object())) self.assertEqual(type(roundtrip), object) def test_enum34(self): restore = self.unpickler.restore flatten = self.pickler.flatten def roundtrip(obj): return restore(flatten(obj)) self.assertTrue(roundtrip(IntEnumTest.X) is IntEnumTest.X) self.assertTrue(roundtrip(IntEnumTest) is IntEnumTest) self.assertTrue(roundtrip(StringEnumTest.A) is StringEnumTest.A) self.assertTrue(roundtrip(StringEnumTest) is StringEnumTest) def test_enum34_nested(self): ec = EnumClass() encoded = jsonpickle.encode(ec) decoded = jsonpickle.decode(encoded) self.assertEqual(ec.enum_a, decoded.enum_a) self.assertEqual(ec.enum_b, decoded.enum_b) def test_enum_references(self): a = IntEnumTest.X b = IntEnumTest.X enums_list = [a, b] encoded = jsonpickle.encode(enums_list) decoded = jsonpickle.decode(encoded) self.assertEqual(enums_list, decoded) def test_enum_unpicklable(self): obj = Message(MessageTypes.STATUS, MessageCommands.STATUS_ALL) encoded = jsonpickle.encode(obj, unpicklable=False) decoded = jsonpickle.decode(encoded) self.assertTrue('message_type' in decoded) self.assertTrue('command' in decoded) def test_bytes_unicode(self): b1 = b'foo' b2 = b'foo\xff' u1 = 'foo' # unicode strings get encoded/decoded as is encoded = self.pickler.flatten(u1) self.assertTrue(encoded == u1) self.assertTrue(isinstance(encoded, compat.ustr)) decoded = self.unpickler.restore(encoded) self.assertTrue(decoded == u1) self.assertTrue(isinstance(decoded, compat.ustr)) # bytestrings are wrapped in PY3 but in PY2 we try to decode first encoded = self.pickler.flatten(b1) if PY2: self.assertEqual(encoded, u1) self.assertTrue(isinstance(encoded, compat.ustr)) else: self.assertNotEqual(encoded, u1) encoded_ustr = util.b64encode(b'foo') self.assertEqual({tags.B64: encoded_ustr}, encoded) self.assertTrue(isinstance(encoded[tags.B64], compat.ustr)) decoded = self.unpickler.restore(encoded) self.assertTrue(decoded == b1) if PY2: self.assertTrue(isinstance(decoded, compat.ustr)) else: self.assertTrue(isinstance(decoded, bytes)) # bytestrings that we can't decode to UTF-8 will always be wrapped encoded = self.pickler.flatten(b2) self.assertNotEqual(encoded, b2) encoded_ustr = util.b64encode(b'foo\xff') self.assertEqual({tags.B64: encoded_ustr}, encoded) self.assertTrue(isinstance(encoded[tags.B64], compat.ustr)) decoded = self.unpickler.restore(encoded) self.assertEqual(decoded, b2) self.assertTrue(isinstance(decoded, bytes)) def test_backcompat_bytes_quoted_printable(self): """Test decoding bytes objects from older jsonpickle versions""" b1 = b'foo' b2 = b'foo\xff' # older versions of jsonpickle used a quoted-printable encoding expect = b1 actual = self.unpickler.restore({tags.BYTES: 'foo'}) self.assertEqual(expect, actual) expect = b2 actual = self.unpickler.restore({tags.BYTES: 'foo=FF'}) self.assertEqual(expect, actual) def test_nested_objects(self): obj = ThingWithTimedeltaAttribute(99) flattened = self.pickler.flatten(obj) restored = self.unpickler.restore(flattened) self.assertEqual(restored.offset, datetime.timedelta(99)) # Test classes for ExternalHandlerTestCase class Mixin(object): def ok(self): return True class UnicodeMixin(str, Mixin): def __add__(self, rhs): obj = super(UnicodeMixin, self).__add__(rhs) return UnicodeMixin(obj) class UnicodeMixinHandler(handlers.BaseHandler): def flatten(self, obj, data): data['value'] = obj return data def restore(self, obj): return UnicodeMixin(obj['value']) handlers.register(UnicodeMixin, UnicodeMixinHandler) class ExternalHandlerTestCase(unittest.TestCase): def test_unicode_mixin(self): obj = UnicodeMixin('test') self.assertTrue(isinstance(obj, UnicodeMixin)) self.assertEqual(obj, 'test') # Encode into JSON content = jsonpickle.encode(obj) # Resurrect from JSON new_obj = jsonpickle.decode(content) new_obj += ' passed' self.assertEqual(new_obj, 'test passed') self.assertTrue(isinstance(new_obj, UnicodeMixin)) self.assertTrue(new_obj.ok()) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(AdvancedObjectsTestCase)) suite.addTest(unittest.makeSuite(ExternalHandlerTestCase)) return suite if __name__ == '__main__': unittest.main(defaultTest='suite') jsonpickle-1.2/tests/pandas_test.py0000664000372000037200000001742413471677136020427 0ustar travistravis00000000000000from __future__ import absolute_import, division, unicode_literals import unittest import jsonpickle from helper import SkippableTest try: import pandas as pd import numpy as np from pandas.testing import assert_series_equal, assert_frame_equal, assert_index_equal except ImportError: np = None class PandasTestCase(SkippableTest): def setUp(self): if np is None: self.should_skip = True return self.should_skip = False import jsonpickle.ext.pandas jsonpickle.ext.pandas.register_handlers() def tearDown(self): if self.should_skip: return import jsonpickle.ext.pandas jsonpickle.ext.pandas.unregister_handlers() def roundtrip(self, obj): return jsonpickle.decode(jsonpickle.encode(obj)) def test_series_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') ser = pd.Series({ 'an_int': np.int_(1), 'a_float': np.float_(2.5), 'a_nan': np.nan, 'a_minus_inf': -np.inf, 'an_inf': np.inf, 'a_str': np.str_('foo'), 'a_unicode': np.unicode_('bar'), 'date': np.datetime64('2014-01-01'), 'complex': np.complex_(1 - 2j), # TODO: the following dtypes are not currently supported. # 'object': np.object_({'a': 'b'}), }) decoded_ser = self.roundtrip(ser) assert_series_equal(decoded_ser, ser) def test_dataframe_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') df = pd.DataFrame({ 'an_int': np.int_([1, 2, 3]), 'a_float': np.float_([2.5, 3.5, 4.5]), 'a_nan': np.array([np.nan] * 3), 'a_minus_inf': np.array([-np.inf] * 3), 'an_inf': np.array([np.inf] * 3), 'a_str': np.str_('foo'), 'a_unicode': np.unicode_('bar'), 'date': np.array([np.datetime64('2014-01-01')] * 3), 'complex': np.complex_([1 - 2j, 2 - 1.2j, 3 - 1.3j]), # TODO: the following dtypes are not currently supported. # 'object': np.object_([{'a': 'b'}]*3), }) decoded_df = self.roundtrip(df) assert_frame_equal(decoded_df, df) def test_multindex_dataframe_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') df = pd.DataFrame({ 'idx_lvl0': ['a', 'b', 'c'], 'idx_lvl1': np.int_([1, 1, 2]), 'an_int': np.int_([1, 2, 3]), 'a_float': np.float_([2.5, 3.5, 4.5]), 'a_nan': np.array([np.nan] * 3), 'a_minus_inf': np.array([-np.inf] * 3), 'an_inf': np.array([np.inf] * 3), 'a_str': np.str_('foo'), 'a_unicode': np.unicode_('bar'), }) df = df.set_index(['idx_lvl0', 'idx_lvl1', ]) decoded_df = self.roundtrip(df) assert_frame_equal(decoded_df, df) def test_dataframe_with_interval_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') df = pd.DataFrame({"a": [1, 2], "b": [3, 4]}, index=pd.IntervalIndex.from_breaks([1,2,4])) decoded_df = self.roundtrip(df) assert_frame_equal(decoded_df, df) def test_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') idx = pd.Index(range(5, 10)) decoded_idx = self.roundtrip(idx) assert_index_equal(decoded_idx, idx) def test_datetime_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') idx = pd.date_range(start='2019-01-01', end='2019-02-01', freq='D') decoded_idx = self.roundtrip(idx) assert_index_equal(decoded_idx, idx) def test_ragged_datetime_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') idx = pd.DatetimeIndex(['2019-01-01', '2019-01-02', '2019-01-05',]) decoded_idx = self.roundtrip(idx) assert_index_equal(decoded_idx, idx) def test_timedelta_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') idx = pd.timedelta_range(start='1 day', periods=4, closed='right') decoded_idx = self.roundtrip(idx) assert_index_equal(decoded_idx, idx) def test_period_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') idx = pd.period_range(start='2017-01-01', end='2018-01-01', freq='M') decoded_idx = self.roundtrip(idx) assert_index_equal(decoded_idx, idx) def test_int64_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') idx = pd.Int64Index([-1, 0, 3, 4]) decoded_idx = self.roundtrip(idx) assert_index_equal(decoded_idx, idx) def test_uint64_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') idx = pd.UInt64Index([0, 3, 4]) decoded_idx = self.roundtrip(idx) assert_index_equal(decoded_idx, idx) def test_float64_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') idx = pd.Float64Index([0.1, 3.7, 4.2]) decoded_idx = self.roundtrip(idx) assert_index_equal(decoded_idx, idx) def test_interval_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') idx = pd.IntervalIndex.from_breaks(range(5)) decoded_idx = self.roundtrip(idx) assert_index_equal(decoded_idx, idx) def test_datetime_interval_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') idx = pd.IntervalIndex.from_breaks(pd.date_range('2019-01-01', '2019-01-10')) decoded_idx = self.roundtrip(idx) assert_index_equal(decoded_idx, idx) def test_multi_index_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') idx = pd.MultiIndex.from_product(((1,2,3), ("a", "b"))) decoded_idx = self.roundtrip(idx) assert_index_equal(decoded_idx, idx) def test_timestamp_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') obj = pd.Timestamp('2019-01-01') decoded_obj = self.roundtrip(obj) assert decoded_obj == obj def test_period_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') obj = pd.Timestamp('2019-01-01') decoded_obj = self.roundtrip(obj) assert decoded_obj == obj def test_interval_roundtrip(self): if self.should_skip: return self.skip('pandas is not importable') obj = pd.Interval(2, 4, closed=str('left')) decoded_obj = self.roundtrip(obj) assert decoded_obj == obj def test_b64(self): """Test the binary encoding""" if self.should_skip: return self.skip('pandas is not importable') # array of substantial size is stored as b64 a = np.random.rand(20, 10) index = ['Row' + str(i) for i in range(1, a.shape[0] + 1)] columns = ['Col' + str(i) for i in range(1, a.shape[1] + 1)] df = pd.DataFrame(a, index=index, columns=columns) decoded_df = self.roundtrip(df) assert_frame_equal(decoded_df, df) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(PandasTestCase, 'test')) return suite if __name__ == '__main__': unittest.main() jsonpickle-1.2/tests/runtests.py0000775000372000037200000000250313471677136020004 0ustar travistravis00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # # Copyright (C) 2008 John Paulett (john -at- paulett.org) # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. import os import sys testdir = os.path.dirname(os.path.abspath(__file__)) # noqa: E402 sys.path.insert(1, os.path.dirname(testdir)) # noqa: E402 import unittest import backend_test import datetime_test import document_test import handler_test import jsonpickle_test import object_test import stdlib_test import util_test import feedparser_test import bson_test import numpy_test import pandas_test def suite(): suite = unittest.TestSuite() suite.addTest(util_test.suite()) suite.addTest(handler_test.suite()) suite.addTest(backend_test.suite()) suite.addTest(jsonpickle_test.suite()) suite.addTest(datetime_test.suite()) suite.addTest(document_test.suite()) suite.addTest(object_test.suite()) suite.addTest(stdlib_test.suite()) suite.addTest(feedparser_test.suite()) suite.addTest(numpy_test.suite()) suite.addTest(bson_test.suite()) suite.addTest(pandas_test.suite()) return suite def main(): return unittest.TextTestRunner(verbosity=2).run(suite()) if __name__ == '__main__': sys.exit(not main().wasSuccessful()) jsonpickle-1.2/tests/sqlalchemy_test.py0000664000372000037200000001014013471677136021307 0ustar travistravis00000000000000"""Test serializing sqlalchemy models""" import unittest import jsonpickle from helper import SkippableTest try: import sqlalchemy as sqa from sqlalchemy.ext import declarative from sqlalchemy.orm import Session HAS_SQA = True except ImportError: HAS_SQA = False if HAS_SQA: Base = declarative.declarative_base() class Table(Base): __tablename__ = 'table' id = sqa.Column(sqa.Integer, primary_key=True) name = sqa.Column(sqa.Text) value = sqa.Column(sqa.Float) class SQLAlchemyTestCase(SkippableTest): def setUp(self): """Create a new sqlalchemy engine for the test""" if HAS_SQA: url = 'sqlite:///:memory:' self.engine = sqa.create_engine(url) Base.metadata.drop_all(self.engine) Base.metadata.create_all(self.engine) self.should_skip = False else: self.should_skip = True def test_sqlalchemy_roundtrip_with_detached_session(self): """Test cloned SQLAlchemy objects detached from any session""" if self.should_skip: return self.skip('sqlalchemy is not installed') expect = Table(name='coolness', value=11.0) session = Session(bind=self.engine, expire_on_commit=False) session.add(expect) session.commit() jsonstr = jsonpickle.dumps(expect) actual = jsonpickle.loads(jsonstr) # actual is a shadow object; it cannot be added to the same # session otherwise sqlalchemy will detect an identity conflict. # To make this work we use expire_on_commit=True so that sqlalchemy # allows us to do read-only operations detached from any session. self.assertEqual(expect.id, actual.id) self.assertEqual(expect.name, actual.name) self.assertEqual(expect.value, actual.value) def test_sqlalchemy_roundtrip_with_two_sessions(self): """Test cloned SQLAlchemy objects attached to a secondary session""" if self.should_skip: return self.skip('sqlalchemy is not installed') expect = Table(name='coolness', value=11.0) session = Session(bind=self.engine, expire_on_commit=False) session.add(expect) session.commit() jsonstr = jsonpickle.dumps(expect) actual = jsonpickle.loads(jsonstr) # actual is a shadow object; it cannot be added to the same # session otherwise sqlalchemy will detect an identity conflict. # To make this work we use expire_on_commit=True so that sqlalchemy # allows us to do read-only operations detached from any session. self.assertEqual(expect.id, actual.id) self.assertEqual(expect.name, actual.name) self.assertEqual(expect.value, actual.value) def test_sqlalchemy_with_dynamic_table(self): """Test creating a table dynamically, per #180""" if self.should_skip: return self.skip('sqlalchemy is not installed') meta = sqa.MetaData() expect = sqa.Table( 'test', meta, sqa.Column('id', sqa.Integer()), sqa.Column('text', sqa.Text())) jsonstr = jsonpickle.dumps(expect) actual = jsonpickle.loads(jsonstr) self.assertEqual(expect.__class__, actual.__class__) self.assertEqual(expect.name, actual.name) # These must be unique instances self.assertNotEqual(expect.metadata, actual.metadata) # Columns names must exactly match self.assertEqual(sorted(expect.columns.keys()), sorted(actual.columns.keys())) # As should the types self.assertEqual(expect.c.id.name, actual.c.id.name) self.assertEqual(expect.c.id.type.__class__, actual.c.id.type.__class__) self.assertEqual(expect.c.text.name, actual.c.text.name) self.assertEqual(expect.c.text.type.__class__, actual.c.text.type.__class__) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(SQLAlchemyTestCase, 'test')) return suite if __name__ == '__main__': unittest.main() jsonpickle-1.2/tests/stdlib_test.py0000664000372000037200000000322513471677136020434 0ustar travistravis00000000000000# -*- coding: utf-8 -*- """Test miscellaneous objects from the standard library""" import uuid import unittest import jsonpickle class UUIDTestCase(unittest.TestCase): def test_random_uuid(self): u = uuid.uuid4() encoded = jsonpickle.encode(u) decoded = jsonpickle.decode(encoded) expect = u.hex actual = decoded.hex self.assertEqual(expect, actual) def test_known_uuid(self): expect = '28b56adbd18f44e2a5556bba2f23e6f6' exemplar = uuid.UUID(expect) encoded = jsonpickle.encode(exemplar) decoded = jsonpickle.decode(encoded) actual = decoded.hex self.assertEqual(expect, actual) class BytesTestCase(unittest.TestCase): def test_bytestream(self): expect = (b'\x89HDF\r\n\x1a\n\x00\x00\x00\x00\x00\x08\x08\x00' b'\x04\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00' b'\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xffh' b'\x848\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff' b'\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00`\x00\x00' b'\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00' b'\x00\x88\x00\x00\x00\x00\x00\x00\x00\xa8\x02\x00' b'\x00\x00\x00\x00\x00\x01\x00\x01\x00') encoded = jsonpickle.encode(expect) actual = jsonpickle.decode(encoded) self.assertEqual(expect, actual) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(UUIDTestCase)) suite.addTest(unittest.makeSuite(BytesTestCase)) return suite if __name__ == '__main__': unittest.main(defaultTest='suite') jsonpickle-1.2/tests/util_test.py0000664000372000037200000001660513471677136020136 0ustar travistravis00000000000000# Copyright (C) 2008 John Paulett (john -at- paulett.org) # Copyright (C) 2009-2018 David Aguilar (davvid -at- gmail.com) # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. from __future__ import absolute_import, division, unicode_literals import unittest import doctest import time from jsonpickle import compat from jsonpickle import util class Thing(object): def __init__(self, name): self.name = name self.child = None class DictSubclass(dict): pass class ListSubclass(list): pass class MethodTestClass(object): @staticmethod def static_method(): pass @classmethod def class_method(cls): pass def bound_method(self): pass variable = None class MethodTestSubclass(MethodTestClass): pass class MethodTestOldStyle: def bound_method(self): pass class UtilTestCase(unittest.TestCase): @unittest.skipIf(not compat.PY2, 'Python 2-specific Base85 test') def test_b85encode_crashes_on_python2(self): with self.assertRaises(NotImplementedError): util.b85encode(b'') @unittest.skipIf(not compat.PY2, 'Python 2-specific Base85 test') def test_b85decode_crashes_on_python2(self): with self.assertRaises(NotImplementedError): util.b85decode(u'RC2?pb0AN3baKO~') def test_is_primitive_int(self): self.assertTrue(util.is_primitive(0)) self.assertTrue(util.is_primitive(3)) self.assertTrue(util.is_primitive(-3)) def test_is_primitive_float(self): self.assertTrue(util.is_primitive(0)) self.assertTrue(util.is_primitive(3.5)) self.assertTrue(util.is_primitive(-3.5)) self.assertTrue(util.is_primitive(float(3))) def test_is_primitive_long(self): self.assertTrue(util.is_primitive(2 ** 64)) def test_is_primitive_bool(self): self.assertTrue(util.is_primitive(True)) self.assertTrue(util.is_primitive(False)) def test_is_primitive_None(self): self.assertTrue(util.is_primitive(None)) def test_is_primitive_bytes(self): self.assertFalse(util.is_primitive(b'hello')) self.assertFalse(util.is_primitive('foo'.encode('utf-8'))) self.assertTrue(util.is_primitive('foo')) def test_is_primitive_unicode(self): self.assertTrue(util.is_primitive(compat.ustr(''))) self.assertTrue(util.is_primitive('hello')) def test_is_primitive_list(self): self.assertFalse(util.is_primitive([])) self.assertFalse(util.is_primitive([4, 4])) def test_is_primitive_dict(self): self.assertFalse(util.is_primitive({'key': 'value'})) self.assertFalse(util.is_primitive({})) def test_is_primitive_tuple(self): self.assertFalse(util.is_primitive((1, 3))) self.assertFalse(util.is_primitive((1,))) def test_is_primitive_set(self): self.assertFalse(util.is_primitive({1, 3})) def test_is_primitive_object(self): self.assertFalse(util.is_primitive(Thing('test'))) def test_is_list_list(self): self.assertTrue(util.is_list([1, 2])) def test_is_list_set(self): self.assertTrue(util.is_set({1, 2})) def test_is_list_tuple(self): self.assertTrue(util.is_tuple((1, 2))) def test_is_list_dict(self): self.assertFalse(util.is_list({'key': 'value'})) self.assertFalse(util.is_set({'key': 'value'})) self.assertFalse(util.is_tuple({'key': 'value'})) def test_is_list_other(self): self.assertFalse(util.is_list(1)) self.assertFalse(util.is_set(1)) self.assertFalse(util.is_tuple(1)) def test_is_sequence_various(self): self.assertTrue(util.is_sequence([])) self.assertTrue(util.is_sequence(tuple())) self.assertTrue(util.is_sequence(set())) def test_is_dictionary_dict(self): self.assertTrue(util.is_dictionary({})) def test_is_dicitonary_sequences(self): self.assertFalse(util.is_dictionary([])) self.assertFalse(util.is_dictionary(set())) def test_is_dictionary_tuple(self): self.assertFalse(util.is_dictionary(tuple())) def test_is_dictionary_primitive(self): self.assertFalse(util.is_dictionary(int())) self.assertFalse(util.is_dictionary(None)) self.assertFalse(util.is_dictionary(str())) def test_is_dictionary_subclass_dict(self): self.assertFalse(util.is_dictionary_subclass({})) def test_is_dictionary_subclass_subclass(self): self.assertTrue(util.is_dictionary_subclass(DictSubclass())) def test_is_sequence_subclass_subclass(self): self.assertTrue(util.is_sequence_subclass(ListSubclass())) def test_is_sequence_subclass_list(self): self.assertFalse(util.is_sequence_subclass([])) def test_is_noncomplex_time_struct(self): t = time.struct_time('123456789') self.assertTrue(util.is_noncomplex(t)) def test_is_noncomplex_other(self): self.assertFalse(util.is_noncomplex('a')) def test_is_function_builtins(self): self.assertTrue(util.is_function(globals)) def test_is_function_lambda(self): self.assertTrue(util.is_function(lambda: False)) def test_is_function_instance_method(self): class Foo(object): def method(self): pass @staticmethod def staticmethod(): pass @classmethod def classmethod(cls): pass f = Foo() self.assertTrue(util.is_function(f.method)) self.assertTrue(util.is_function(f.staticmethod)) self.assertTrue(util.is_function(f.classmethod)) def test_itemgetter(self): expect = '0' actual = util.itemgetter((0, 'zero')) self.assertEqual(expect, actual) def test_has_method(self): instance = MethodTestClass() x = 1 has_method = util.has_method # no attribute self.assertFalse(has_method(instance, 'foo')) # builtin method type self.assertFalse(has_method(int, '__getnewargs__')) self.assertTrue(has_method(x, '__getnewargs__')) # staticmethod self.assertTrue(has_method(instance, 'static_method')) self.assertTrue(has_method(MethodTestClass, 'static_method')) # not a method self.assertFalse(has_method(instance, 'variable')) self.assertFalse(has_method(MethodTestClass, 'variable')) # classmethod self.assertTrue(has_method(instance, 'class_method')) self.assertTrue(has_method(MethodTestClass, 'class_method')) # bound method self.assertTrue(has_method(instance, 'bound_method')) self.assertFalse(has_method(MethodTestClass, 'bound_method')) # subclass bound method sub_instance = MethodTestSubclass() self.assertTrue(has_method(sub_instance, 'bound_method')) self.assertFalse(has_method(MethodTestSubclass, 'bound_method')) # old style object old_instance = MethodTestOldStyle() self.assertTrue(has_method(old_instance, 'bound_method')) self.assertFalse(has_method(MethodTestOldStyle, 'bound_method')) def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(UtilTestCase)) suite.addTest(doctest.DocTestSuite(util)) return suite if __name__ == '__main__': unittest.main(defaultTest='suite')