pax_global_header00006660000000000000000000000064131255433070014515gustar00rootroot0000000000000052 comment=103a20192aa1c900863db327bd522c04418de94a deepdiff-3.3.0/000077500000000000000000000000001312554330700132665ustar00rootroot00000000000000deepdiff-3.3.0/.coveragerc000066400000000000000000000001211312554330700154010ustar00rootroot00000000000000[report] omit = */python?.?/* */site-packages/nose/* *__init__* deepdiff-3.3.0/.gitignore000066400000000000000000000013701312554330700152570ustar00rootroot00000000000000# Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] # C extensions *.so # Distribution / packaging .Python env/ build/ develop-eggs/ dist/ downloads/ eggs/ lib/ lib64/ parts/ sdist/ var/ no_upload/ *.egg-info/ .installed.cfg *.egg # PyInstaller # Usually these files are written by a python script from a template # before PyInstaller builds the exe, so as to inject date/other infos into it. *.manifest *.spec # Installer logs pip-log.txt pip-delete-this-directory.txt # Unit test / coverage reports htmlcov/ .tox/ .coverage .cache nosetests.xml coverage.xml # Translations *.mo *.pot # Django stuff: *.log # Sphinx documentation docs/_build/ # PyBuilder target/ # OS-specific spam .DS_Store # Editor / IDE files *.swp .idea/ .~lock* deepdiff-3.3.0/.travis.yml000066400000000000000000000005031312554330700153750ustar00rootroot00000000000000language: python python: - "2.7" - "3.3" - "3.4" - "3.5" - "3.6" - "pypy-5.4" # pypy on python 2.7 # - "pypy3" # Removing pypy3 from travis since travis's pypy3 seems buggy sudo: false install: - pip install coveralls script: coverage run --source deepdiff setup.py test after_success: - coveralls deepdiff-3.3.0/AUTHORS000066400000000000000000000011241312554330700143340ustar00rootroot00000000000000Authors: - Seperman - Victor Hahn Castell @ Flexoptix Also thanks to: - nfvs for Travis-CI setup script. - brbsix for initial Py3 porting. - WangFenjin for unicode support. - timoilya for comparing list of sets when ignoring order. - Bernhard10 for significant digits comparison. - b-jazz for PEP257 cleanup, Standardize on full names, fixing line endings. - finnhughes for fixing __slots__ - moloney for Unicode vs. Bytes default - serv-inc for adding help(deepdiff) - movermeyer for updating docs - maxrothman for search in inherited class attributes - maxrothman for search for types/objects deepdiff-3.3.0/LICENSE000066400000000000000000000021471312554330700142770ustar00rootroot00000000000000The MIT License (MIT) Copyright (c) 2014 - 2016 Sep Ehr (Seperman) and contributors www.zepworks.com Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. deepdiff-3.3.0/README.md000066400000000000000000000664261312554330700145630ustar00rootroot00000000000000# deepdiff v 3.3.0 [![Join the chat at https://gitter.im/deepdiff/Lobby](https://badges.gitter.im/deepdiff/Lobby.svg)](https://gitter.im/deepdiff/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) ![Python Versions](https://img.shields.io/pypi/pyversions/deepdiff.svg?style=flat) ![Doc](https://readthedocs.org/projects/deepdiff/badge/?version=latest) ![License](https://img.shields.io/pypi/l/deepdiff.svg?version=latest) [![Build Status](https://travis-ci.org/seperman/deepdiff.svg?branch=master)](https://travis-ci.org/seperman/deepdiff) [![Coverage Status](https://coveralls.io/repos/github/seperman/deepdiff/badge.svg?branch=master)](https://coveralls.io/github/seperman/deepdiff?branch=master) Deep Difference of dictionaries, iterables, strings and other objects. It will recursively look for all the changes. Tested on Python 2.7, 3.3, 3.4, 3.5, 3.6, Pypy, Pypy3 ## Table of Contents - [Installation](#Installation) - [Parameters](#parameters) - [Ignore Order](#ignore-order) - [Report repetitions](#report-repetitions) - [Exclude types or paths](#exclude-type-or-paths) - [Significant Digits](#significant-digits) - [Verbose Level](#verbose-level) - [Deep Search](#deep-search) - [Using DeepDiff in unit tests](#using-deepdiff-in-unit-tests) - [Difference with Json Patch](#difference-with-json-patch) - [Views](#views) - [Text View](#text-view) - [Tree View](#tree-view) - [Serialization](#serialization) - [Documentation](http://deepdiff.readthedocs.io/en/latest/) ## Installation ### Install from PyPi: pip install deepdiff ### Importing ```python >>> from deepdiff import DeepDiff # For Deep Difference of 2 objects >>> from deepdiff import DeepSearch # For finding if item exists in an object ``` ## Parameters In addition to the 2 objects being compared: - [ignore_order](#ignore-order) - [report_repetition](#report-repetitions) - [verbose_level](#verbose-level) ## Supported data types int, string, dictionary, list, tuple, set, frozenset, OrderedDict, NamedTuple and custom objects! ## Ignore Order Sometimes you don't care about the order of objects when comparing them. In those cases, you can set `ignore_order=True`. However this flag won't report the repetitions to you. You need to additionally enable `report_report_repetition=True` for getting a report of repetitions. ### List difference ignoring order or duplicates ```python >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 3, 2, 3]}} >>> ddiff = DeepDiff(t1, t2, ignore_order=True) >>> print (ddiff) {} ``` ## Report repetitions This flag ONLY works when ignoring order is enabled. Note that this feature is experimental. ```python t1 = [1, 3, 1, 4] t2 = [4, 4, 1] ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True) print(ddiff) ``` which will print you: ```python {'iterable_item_removed': {'root[1]': 3}, 'repetition_change': {'root[0]': {'old_repeat': 2, 'old_indexes': [0, 2], 'new_indexes': [2], 'value': 1, 'new_repeat': 1}, 'root[3]': {'old_repeat': 1, 'old_indexes': [3], 'new_indexes': [0, 1], 'value': 4, 'new_repeat': 2}}} ``` ## Exclude types or paths ### Exclude certain types from comparison: ```python >>> l1 = logging.getLogger("test") >>> l2 = logging.getLogger("test2") >>> t1 = {"log": l1, 2: 1337} >>> t2 = {"log": l2, 2: 1337} >>> print(DeepDiff(t1, t2, exclude_types={logging.Logger})) {} ``` ### Exclude part of your object tree from comparison: ```python >>> t1 = {"for life": "vegan", "ingredients": ["no meat", "no eggs", "no dairy"]} >>> t2 = {"for life": "vegan", "ingredients": ["veggies", "tofu", "soy sauce"]} >>> print (DeepDiff(t1, t2, exclude_paths={"root['ingredients']"})) {} ``` ## Significant Digits Digits **after** the decimal point. Internally it uses "{:.Xf}".format(Your Number) to compare numbers where X=significant_digits ```python >>> t1 = Decimal('1.52') >>> t2 = Decimal('1.57') >>> DeepDiff(t1, t2, significant_digits=0) {} >>> DeepDiff(t1, t2, significant_digits=1) {'values_changed': {'root': {'old_value': Decimal('1.52'), 'new_value': Decimal('1.57')}}} ``` Approximate float comparison: ```python >>> t1 = [ 1.1129, 1.3359 ] >>> t2 = [ 1.113, 1.3362 ] >>> pprint(DeepDiff(t1, t2, significant_digits=3)) {} >>> pprint(DeepDiff(t1, t2)) {'values_changed': {'root[0]': {'new_value': 1.113, 'old_value': 1.1129}, 'root[1]': {'new_value': 1.3362, 'old_value': 1.3359}}} >>> pprint(DeepDiff(1.23*10**20, 1.24*10**20, significant_digits=1)) {'values_changed': {'root': {'new_value': 1.24e+20, 'old_value': 1.23e+20}}} ``` ## Verbose Level Verbose level by default is 1. The possible values are 0, 1 and 2. - Verbose level 0: won't report values when type changed. [Example](##type-of-an-item-has-changed) - Verbose level 1: default - Verbose level 2: will report values when custom objects or dictionaries have items added or removed. [Example](#items-added-or-removed-verbose) ## Deep Search (New in v2-1-0) Tip: Take a look at [grep](#grep) which gives you a new interface for DeepSearch! DeepDiff comes with a utility to find the path to the item you are looking for. It is called DeepSearch and it has a similar interface to DeepDiff. Let's say you have a huge nested object and want to see if any item with the word `somewhere` exists in it. ```py from deepdiff import DeepSearch obj = {"long": "somewhere", "string": 2, 0: 0, "somewhere": "around"} ds = DeepSearch(obj, "somewhere", verbose_level=2) print(ds) ``` Which will print: ```py {'matched_paths': {"root['somewhere']": "around"}, 'matched_values': {"root['long']": "somewhere"}} ``` Tip: An interesting use case is to search inside `locals()` when doing pdb. ## Grep (New in v3-2-0) Grep is another interface for DeepSearch. Just grep through your objects as you would in shell! ```py from deepdiff import grep obj = {"long": "somewhere", "string": 2, 0: 0, "somewhere": "around"} ds = obj | grep("somewhere") print(ds) ``` Which will print: ```py {'matched_paths': {"root['somewhere']"}, 'matched_values': {"root['long']"}} ``` And you can pass all the same kwargs as DeepSearch to grep too: ```py >>> obj | grep(item, verbose_level=2) {'matched_paths': {"root['somewhere']": 'around'}, 'matched_values': {"root['long']": 'somewhere'}} ``` ## Using DeepDiff in unit tests `result` is the output of the function that is being tests. `expected` is the expected output of the function. ```python assertEqual(DeepDiff(result, expected), {}) ``` ## Difference with Json Patch Unlike [Json Patch](https://tools.ietf.org/html/rfc6902) which is designed only for Json objects, DeepDiff is designed specifically for almost all Python types. In addition to that, DeepDiff checks for type changes and attribute value changes that Json Patch does not cover since there are no such things in Json. Last but not least, DeepDiff gives you the exact path of the item(s) that were changed in Python syntax. Example in Json Patch for replacing: `{ "op": "replace", "path": "/a/b/c", "value": 42 }` Example in DeepDiff for the same operation: ```python >>> item1 = {'a':{'b':{'c':'foo'}}} >>> item2 = {'a':{'b':{'c':42}}} >>> DeepDiff(item1, item2) {'type_changes': {"root['a']['b']['c']": {'old_type': , 'new_value': 42, 'old_value': 'foo', 'new_type': }}} ``` # Views Starting with DeepDiff v 3, there are two different views into your diffed data: text view (original) and tree view (new). ## Text View Text view is the original and currently the default view of DeepDiff. It is called text view because the results contain texts that represent the path to the data: Example of using the text view. ```python >>> from deepdiff import DeepDiff >>> t1 = {1:1, 3:3, 4:4} >>> t2 = {1:1, 3:3, 5:5, 6:6} >>> ddiff = DeepDiff(t1, t2) >>> print(ddiff) {'dictionary_item_added': {'root[5]', 'root[6]'}, 'dictionary_item_removed': {'root[4]'}} ``` So for example `ddiff['dictionary_item_removed']` is a set if strings thus this is called the text view. The following examples are using the *default text view.* The Tree View is introduced in DeepDiff v3 and provides traversing capabilities through your diffed data and more! Read more about the Tree View at the bottom of this page. ### Importing ```python >>> from deepdiff import DeepDiff >>> from pprint import pprint >>> from __future__ import print_function # In case running on Python 2 ``` ### Same object returns empty ```python >>> t1 = {1:1, 2:2, 3:3} >>> t2 = t1 >>> print(DeepDiff(t1, t2)) {} ``` ### Type of an item has changed ```python >>> t1 = {1:1, 2:2, 3:3} >>> t2 = {1:1, 2:"2", 3:3} >>> pprint(DeepDiff(t1, t2), indent=2) { 'type_changes': { 'root[2]': { 'new_type': , 'new_value': '2', 'old_type': , 'old_value': 2}}} ``` And if you don't care about the value of items that have changed type, please set verbose level to 0: ```python >>> t1 = {1:1, 2:2, 3:3} >>> t2 = {1:1, 2:"2", 3:3} >>> pprint(DeepDiff(t1, t2, verbose_level=0), indent=2) { 'type_changes': { 'root[2]': { 'new_type': , 'old_type': ,}}} ``` ### Value of an item has changed ```python >>> t1 = {1:1, 2:2, 3:3} >>> t2 = {1:1, 2:4, 3:3} >>> pprint(DeepDiff(t1, t2), indent=2) {'values_changed': {'root[2]': {'new_value': 4, 'old_value': 2}}} ``` ### Item added or removed ```python >>> t1 = {1:1, 3:3, 4:4} >>> t2 = {1:1, 3:3, 5:5, 6:6} >>> ddiff = DeepDiff(t1, t2) >>> pprint(ddiff) {'dictionary_item_added': {'root[5]', 'root[6]'}, 'dictionary_item_removed': {'root[4]'}} ``` #### Items added or removed verbose And if you would like to know the values of items added or removed, please set the verbose_level to 2: ```python >>> t1 = {1:1, 3:3, 4:4} >>> t2 = {1:1, 3:3, 5:5, 6:6} >>> ddiff = DeepDiff(t1, t2, verbose_level=2) >>> pprint(ddiff, indent=2) { 'dictionary_item_added': {'root[5]': 5, 'root[6]': 6}, 'dictionary_item_removed': {'root[4]': 4}} ``` ### String difference ```python >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":"world"}} >>> t2 = {1:1, 2:4, 3:3, 4:{"a":"hello", "b":"world!"}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'values_changed': { 'root[2]': {'new_value': 4, 'old_value': 2}, "root[4]['b']": { 'new_value': 'world!', 'old_value': 'world'}}} ``` ### String difference 2 ```python >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":"world!\nGoodbye!\n1\n2\nEnd"}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":"world\n1\n2\nEnd"}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'values_changed': { "root[4]['b']": { 'diff': '--- \n' '+++ \n' '@@ -1,5 +1,4 @@\n' '-world!\n' '-Goodbye!\n' '+world\n' ' 1\n' ' 2\n' ' End', 'new_value': 'world\n1\n2\nEnd', 'old_value': 'world!\n' 'Goodbye!\n' '1\n' '2\n' 'End'}}} >>> >>> print (ddiff['values_changed']["root[4]['b']"]["diff"]) --- +++ @@ -1,5 +1,4 @@ -world! -Goodbye! +world 1 2 End ``` ### List difference ```python >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3, 4]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2]}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) {'iterable_item_removed': {"root[4]['b'][2]": 3, "root[4]['b'][3]": 4}} ``` ### List difference Example 2 ```python >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 3, 2, 3]}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'iterable_item_added': {"root[4]['b'][3]": 3}, 'values_changed': { "root[4]['b'][1]": {'new_value': 3, 'old_value': 2}, "root[4]['b'][2]": {'new_value': 2, 'old_value': 3}}} ``` ### List that contains dictionary: ```python >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, {1:1, 2:2}]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, {1:3}]}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'dictionary_item_removed': ["root[4]['b'][2][2]"], 'values_changed': {"root[4]['b'][2][1]": {'new_value': 3, 'old_value': 1}}} ``` ### Sets: ```python >>> t1 = {1, 2, 8} >>> t2 = {1, 2, 3, 5} >>> ddiff = DeepDiff(t1, t2) >>> pprint (DeepDiff(t1, t2)) {'set_item_added': ['root[3]', 'root[5]'], 'set_item_removed': ['root[8]']} ``` ### Named Tuples: ```python >>> from collections import namedtuple >>> Point = namedtuple('Point', ['x', 'y']) >>> t1 = Point(x=11, y=22) >>> t2 = Point(x=11, y=23) >>> pprint (DeepDiff(t1, t2)) {'values_changed': {'root.y': {'new_value': 23, 'old_value': 22}}} ``` ### Custom objects: ```python >>> class ClassA(object): ... a = 1 ... def __init__(self, b): ... self.b = b ... >>> t1 = ClassA(1) >>> t2 = ClassA(2) >>> >>> pprint(DeepDiff(t1, t2)) {'values_changed': {'root.b': {'new_value': 2, 'old_value': 1}}} ``` ### Object attribute added: ```python >>> t2.c = "new attribute" >>> pprint(DeepDiff(t1, t2)) {'attribute_added': ['root.c'], 'values_changed': {'root.b': {'new_value': 2, 'old_value': 1}}} ``` ### Exclude certain types from comparison: ```python >>> l1 = logging.getLogger("test") >>> l2 = logging.getLogger("test2") >>> t1 = {"log": l1, 2: 1337} >>> t2 = {"log": l2, 2: 1337} >>> print(DeepDiff(t1, t2, exclude_types={logging.Logger})) {} ``` ### Exclude part of your object tree from comparison: ```python >>> t1 = {"for life": "vegan", "ingredients": ["no meat", "no eggs", "no dairy"]} >>> t2 = {"for life": "vegan", "ingredients": ["veggies", "tofu", "soy sauce"]} >>> print (DeepDiff(t1, t2, exclude_paths={"root['ingredients']"})) {} ``` All the examples for the text view work for the tree view too. You just need to set view='tree' to get it in tree form. ## Tree View Starting the version v3 You can choose the view into the deepdiff results. The tree view provides you with tree objects that you can traverse through to find the parents of the objects that are diffed and the actual objects that are being diffed. This view is very useful when dealing with nested objects. Note that tree view always returns results in the form of Python sets. You can traverse through the tree elements! The Tree view is just a different representation of the diffed data. Behind the scene, DeepDiff creates the tree view first and then converts it to textual representation for the text view. ``` +---------------------------------------------------------------+ | | | parent(t1) parent node parent(t2) | | + ^ + | +------|--------------------------|---------------------|-------+ | | | up | | Child | | | ChildRelationship | Relationship | | | | down | | | +------|----------------------|-------------------------|-------+ | v v v | | child(t1) child node child(t2) | | | +---------------------------------------------------------------+ ``` - up - Move up to the parent node - down - Move down to the child node - path() - Get the path to the current node - t1 - The first item in the current node that is being diffed - t2 - The second item in the current node that is being diffed - additional - Additional information about the node i.e. repetition - repetition - Shortcut to get the repetition report The tree view allows you to have more than mere textual representaion of the diffed objects. It gives you the actual objects (t1, t2) throughout the tree of parents and children. ## Examples - Tree View The Tree View is introduced in DeepDiff v3 Set view='tree' in order to use this view. ### Value of an item has changed (Tree View) ```python >>> from deepdiff import DeepDiff >>> from pprint import pprint >>> t1 = {1:1, 2:2, 3:3} >>> t2 = {1:1, 2:4, 3:3} >>> ddiff_verbose0 = DeepDiff(t1, t2, verbose_level=0, view='tree') >>> ddiff_verbose0 {'values_changed': {}} >>> >>> ddiff_verbose1 = DeepDiff(t1, t2, verbose_level=1, view='tree') >>> ddiff_verbose1 {'values_changed': {}} >>> set_of_values_changed = ddiff_verbose1['values_changed'] >>> # since set_of_values_changed includes only one item in a set >>> # in order to get that one item we can: >>> (changed,) = set_of_values_changed >>> changed # Another way to get this is to do: changed=list(set_of_values_changed)[0] >>> changed.t1 2 >>> changed.t2 4 >>> # You can traverse through the tree, get to the parents! >>> changed.up ``` ### List difference (Tree View) ```python >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3, 4]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2]}} >>> ddiff = DeepDiff(t1, t2, view='tree') >>> ddiff {'iterable_item_removed': {, }} >>> # Note that the iterable_item_removed is a set. In this case it has 2 items in it. >>> # One way to get one item from the set is to convert it to a list >>> # And then get the first item of the list: >>> removed = list(ddiff['iterable_item_removed'])[0] >>> removed >>> >>> parent = removed.up >>> parent >>> parent.path() "root[4]['b']" >>> parent.t1 [1, 2, 3, 4] >>> parent.t2 [1, 2] >>> parent.up >>> parent.up.up >>> parent.up.up.t1 {1: 1, 2: 2, 3: 3, 4: {'a': 'hello', 'b': [1, 2, 3, 4]}} >>> parent.up.up.t1 == t1 # It is holding the original t1 that we passed to DeepDiff True ``` ### List difference 2 (Tree View) ```python >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 3, 2, 3]}} >>> ddiff = DeepDiff(t1, t2, view='tree') >>> pprint(ddiff, indent = 2) { 'iterable_item_added': {}, 'values_changed': { , }} >>> >>> # Note that iterable_item_added is a set with one item. >>> # So in order to get that one item from it, we can do: >>> >>> (added,) = ddiff['iterable_item_added'] >>> added >>> added.up.up >>> added.up.up.path() 'root[4]' >>> added.up.up.down >>> >>> # going up twice and then down twice gives you the same node in the tree: >>> added.up.up.down.down == added True ``` ### List difference ignoring order but reporting repetitions (Tree View) ```python >>> t1 = [1, 3, 1, 4] >>> t2 = [4, 4, 1] >>> ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True, view='tree') >>> pprint(ddiff, indent=2) { 'iterable_item_removed': {}, 'repetition_change': { , }} >>> >>> # repetition_change is a set with 2 items. >>> # in order to get those 2 items, we can do the following. >>> # or we can convert the set to list and get the list items. >>> # or we can iterate through the set items >>> >>> (repeat1, repeat2) = ddiff['repetition_change'] >>> repeat1 # the default verbosity is set to 1. >>> # The actual data regarding the repetitions can be found in the repetition attribute: >>> repeat1.repetition {'old_repeat': 1, 'new_repeat': 2, 'old_indexes': [3], 'new_indexes': [0, 1]} >>> >>> # If you change the verbosity, you will see less: >>> ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True, view='tree', verbose_level=0) >>> ddiff {'repetition_change': {, }, 'iterable_item_removed': {}} >>> (repeat1, repeat2) = ddiff['repetition_change'] >>> repeat1 >>> >>> # But the verbosity level does not change the actual report object. >>> # It only changes the textual representaion of the object. We get the actual object here: >>> repeat1.repetition {'old_repeat': 1, 'new_repeat': 2, 'old_indexes': [3], 'new_indexes': [0, 1]} >>> repeat1.t1 4 >>> repeat1.t2 4 >>> repeat1.up ``` ### List that contains dictionary (Tree View) ```python >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, {1:1, 2:2}]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, {1:3}]}} >>> ddiff = DeepDiff(t1, t2, view='tree') >>> pprint (ddiff, indent = 2) { 'dictionary_item_removed': {}, 'values_changed': {}} Sets (Tree View): >>> t1 = {1, 2, 8} >>> t2 = {1, 2, 3, 5} >>> ddiff = DeepDiff(t1, t2, view='tree') >>> print(ddiff) {'set_item_removed': {}, 'set_item_added': {, }} >>> # grabbing one item from set_item_removed set which has one item only >>> (item,) = ddiff['set_item_removed'] >>> item.up >>> item.up.t1 == t1 True ``` ### Named Tuples (Tree View): ```python >>> from collections import namedtuple >>> Point = namedtuple('Point', ['x', 'y']) >>> t1 = Point(x=11, y=22) >>> t2 = Point(x=11, y=23) >>> print(DeepDiff(t1, t2, view='tree')) {'values_changed': {}} ``` ### Custom objects (Tree View): ```python >>> class ClassA(object): ... a = 1 ... def __init__(self, b): ... self.b = b ... >>> t1 = ClassA(1) >>> t2 = ClassA(2) >>> >>> print(DeepDiff(t1, t2, view='tree')) {'values_changed': {}} ``` ### Object attribute added (Tree View): ```python >>> t2.c = "new attribute" >>> pprint(DeepDiff(t1, t2, view='tree')) {'attribute_added': {}, 'values_changed': {}} ``` ### Approximate decimals comparison (Significant digits after the point) (Tree View): ```python >>> t1 = Decimal('1.52') >>> t2 = Decimal('1.57') >>> DeepDiff(t1, t2, significant_digits=0, view='tree') {} >>> ddiff = DeepDiff(t1, t2, significant_digits=1, view='tree') >>> ddiff {'values_changed': {}} >>> (change1,) = ddiff['values_changed'] >>> change1 >>> change1.t1 Decimal('1.52') >>> change1.t2 Decimal('1.57') >>> change1.path() 'root' ``` ### Approximate float comparison (Significant digits after the point) (Tree View): ```python >>> t1 = [ 1.1129, 1.3359 ] >>> t2 = [ 1.113, 1.3362 ] >>> ddiff = DeepDiff(t1, t2, significant_digits=3, view='tree') >>> ddiff {} >>> ddiff = DeepDiff(t1, t2, view='tree') >>> pprint(ddiff, indent=2) { 'values_changed': { , }} >>> ddiff = DeepDiff(1.23*10**20, 1.24*10**20, significant_digits=1, view='tree') >>> ddiff {'values_changed': {}} ``` All the examples for the text view work for the tree view too. You just need to set view='tree' to get it in tree form. ## Serialization DeepDiff uses jsonpickle in order to serialize and deserialize its results into json. This works for both tree view and text view. ### Serialize and then deserialize back to deepdiff ```python >>> t1 = {1: 1, 2: 2, 3: 3} >>> t2 = {1: 1, 2: "2", 3: 3} >>> ddiff = DeepDiff(t1, t2) >>> jsoned = ddiff.json >>> jsoned '{"type_changes": {"root[2]": {"py/object": "deepdiff.helper.RemapDict", "new_type": {"py/type": "__builtin__.str"}, "new_value": "2", "old_type": {"py/type": "__builtin__.int"}, "old_value": 2}}}' >>> ddiff_new = DeepDiff.from_json(jsoned) >>> ddiff == ddiff_new True ``` ## Pycon 2016 I was honored to give a talk about how DeepDiff does what it does at Pycon 2016. Please check out the video and let me know what you think: [Diff It To Dig It Video](https://www.youtube.com/watch?v=J5r99eJIxF4) And here is more info: ## Documentation ## Change log - v3-3-0: Searching for objects and class attributes - v3-2-2: Adding help(deepdiff) - v3-2-1: Fixing hash of None - v3-2-0: Adding grep for search: object | grep(item) - v3-1-3: Unicode vs. Bytes default fix - v3-1-2: NotPresent Fix when item is added or removed. - v3-1-1: Bug fix when item value is None (#58) - v3-1-0: Serialization to/from json - v3-0-0: Introducing Tree View - v2-5-3: Bug fix on logging for content hash. - v2-5-2: Bug fixes on content hash. - v2-5-0: Adding ContentHash module to fix ignore_order once and for all. - v2-1-0: Adding Deep Search. Now you can search for item in an object. - v2-0-0: Exclusion patterns better coverage. Updating docs. - v1-8-0: Exclusion patterns. - v1-7-0: Deep Set comparison. - v1-6-0: Unifying key names. i.e newvalue is new_value now. For backward compatibility, newvalue still works. - v1-5-0: Fixing ignore order containers with unordered items. Adding significant digits when comparing decimals. Changes property is deprecated. - v1-1-0: Changing Set, Dictionary and Object Attribute Add/Removal to be reported as Set instead of List. Adding Pypy compatibility. - v1-0-2: Checking for ImmutableMapping type instead of dict - v1-0-1: Better ignore order support - v1-0-0: Restructuring output to make it more useful. This is NOT backward compatible. - v0-6-1: Fixiing iterables with unhashable when order is ignored - v0-6-0: Adding unicode support - v0-5-9: Adding decimal support - v0-5-8: Adding ignore order of unhashables support - v0-5-7: Adding ignore order support - v0-5-6: Adding slots support - v0-5-5: Adding loop detection ## Authors Seperman (Sep Dehpour) - [Github](https://github.com/seperman) - [Linkedin](http://www.linkedin.com/in/sepehr) - [ZepWorks](http://www.zepworks.com) Victor Hahn Castell - [hahncastell.de](http://hahncastell.de) - [flexoptix.net](http://www.flexoptix.net) Also thanks to: - nfvs for Travis-CI setup script. - brbsix for initial Py3 porting. - WangFenjin for unicode support. - timoilya for comparing list of sets when ignoring order. - Bernhard10 for significant digits comparison. - b-jazz for PEP257 cleanup, Standardize on full names, fixing line endings. - finnhughes for fixing __slots__ - moloney for Unicode vs. Bytes default - serv-inc for adding help(deepdiff) - movermeyer for updating docs - maxrothman for search in inherited class attributes - maxrothman for search for types/objects deepdiff-3.3.0/README.txt000066400000000000000000000262411312554330700147710ustar00rootroot00000000000000**DeepDiff v 3.3.0** Deep Difference of dictionaries, iterables, strings and other objects. It will recursively look for all the changes. Tested on Python 2.7, 3.3, 3.4, 3.5, 3.6, Pypy, Pypy3 Note: Checkout the github repo's readme for complete coverage of features: https://github.com/seperman/deepdiff **Parameters** In addition to the 2 objects being compared: - ignore_order - report_repetition - verbose_level **Returns** A DeepDiff object that has already calculated the difference of the 2 items. **Supported data types** int, string, unicode, dictionary, list, tuple, set, frozenset, OrderedDict, NamedTuple and custom objects! **Examples** Importing >>> from deepdiff import DeepDiff >>> from pprint import pprint >>> from __future__ import print_function # In case running on Python 2 Same object returns empty >>> t1 = {1:1, 2:2, 3:3} >>> t2 = t1 >>> print(DeepDiff(t1, t2)) {} Type of an item has changed >>> t1 = {1:1, 2:2, 3:3} >>> t2 = {1:1, 2:"2", 3:3} >>> pprint(DeepDiff(t1, t2), indent=2) { 'type_changes': { 'root[2]': { 'new_type': , 'new_value': '2', 'old_type': , 'old_value': 2}}} Value of an item has changed >>> t1 = {1:1, 2:2, 3:3} >>> t2 = {1:1, 2:4, 3:3} >>> pprint(DeepDiff(t1, t2), indent=2) {'values_changed': {'root[2]': {'new_value': 4, 'old_value': 2}}} Item added and/or removed >>> t1 = {1:1, 2:2, 3:3, 4:4} >>> t2 = {1:1, 2:4, 3:3, 5:5, 6:6} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff) {'dictionary_item_added': ['root[5]', 'root[6]'], 'dictionary_item_removed': ['root[4]'], 'values_changed': {'root[2]': {'new_value': 4, 'old_value': 2}}} String difference >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":"world"}} >>> t2 = {1:1, 2:4, 3:3, 4:{"a":"hello", "b":"world!"}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'values_changed': { 'root[2]': {'new_value': 4, 'old_value': 2}, "root[4]['b']": { 'new_value': 'world!', 'old_value': 'world'}}} String difference 2 >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":"world!\nGoodbye!\n1\n2\nEnd"}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":"world\n1\n2\nEnd"}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'values_changed': { "root[4]['b']": { 'diff': '--- \n' '+++ \n' '@@ -1,5 +1,4 @@\n' '-world!\n' '-Goodbye!\n' '+world\n' ' 1\n' ' 2\n' ' End', 'new_value': 'world\n1\n2\nEnd', 'old_value': 'world!\n' 'Goodbye!\n' '1\n' '2\n' 'End'}}} >>> >>> print (ddiff['values_changed']["root[4]['b']"]["diff"]) --- +++ @@ -1,5 +1,4 @@ -world! -Goodbye! +world 1 2 End Type change >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":"world\n\n\nEnd"}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'type_changes': { "root[4]['b']": { 'new_type': , 'new_value': 'world\n\n\nEnd', 'old_type': , 'old_value': [1, 2, 3]}}} List difference >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3, 4]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2]}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) {'iterable_item_removed': {"root[4]['b'][2]": 3, "root[4]['b'][3]": 4}} List difference 2: >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 3, 2, 3]}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'iterable_item_added': {"root[4]['b'][3]": 3}, 'values_changed': { "root[4]['b'][1]": {'new_value': 3, 'old_value': 2}, "root[4]['b'][2]": {'new_value': 2, 'old_value': 3}}} List difference ignoring order or duplicates: (with the same dictionaries as above) >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 3, 2, 3]}} >>> ddiff = DeepDiff(t1, t2, ignore_order=True) >>> print (ddiff) {} List that contains dictionary: >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, {1:1, 2:2}]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, {1:3}]}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'dictionary_item_removed': ["root[4]['b'][2][2]"], 'values_changed': {"root[4]['b'][2][1]": {'new_value': 3, 'old_value': 1}}} Sets: >>> t1 = {1, 2, 8} >>> t2 = {1, 2, 3, 5} >>> ddiff = DeepDiff(t1, t2) >>> pprint (DeepDiff(t1, t2)) {'set_item_added': ['root[3]', 'root[5]'], 'set_item_removed': ['root[8]']} Named Tuples: >>> from collections import namedtuple >>> Point = namedtuple('Point', ['x', 'y']) >>> t1 = Point(x=11, y=22) >>> t2 = Point(x=11, y=23) >>> pprint (DeepDiff(t1, t2)) {'values_changed': {'root.y': {'new_value': 23, 'old_value': 22}}} Custom objects: >>> class ClassA(object): ... a = 1 ... def __init__(self, b): ... self.b = b ... >>> t1 = ClassA(1) >>> t2 = ClassA(2) >>> >>> pprint(DeepDiff(t1, t2)) {'values_changed': {'root.b': {'new_value': 2, 'old_value': 1}}} Object attribute added: >>> t2.c = "new attribute" >>> pprint(DeepDiff(t1, t2)) {'attribute_added': ['root.c'], 'values_changed': {'root.b': {'new_value': 2, 'old_value': 1}}} Exclude certain types from comparison: >>> l1 = logging.getLogger("test") >>> l2 = logging.getLogger("test2") >>> t1 = {"log": l1, 2: 1337} >>> t2 = {"log": l2, 2: 1337} >>> print(DeepDiff(t1, t2, exclude_types={logging.Logger})) {} Exclude part of your object tree from comparison: >>> t1 = {"for life": "vegan", "ingredients": ["no meat", "no eggs", "no dairy"]} >>> t2 = {"for life": "vegan", "ingredients": ["veggies", "tofu", "soy sauce"]} >>> print (DeepDiff(t1, t2, exclude_paths={"root['ingredients']"})) {} Using DeepDiff in unit tests result is the output of the function that is being tests. expected is the expected output of the function. >>> assertEqual(DeepDiff(result, expected), {}) **Difference with Json Patch** Unlike Json Patch https://tools.ietf.org/html/rfc6902 which is designed only for Json objects, DeepDiff is designed specifically for almost all Python types. In addition to that, DeepDiff checks for type changes and attribute value changes that Json Patch does not cover since there are no such things in Json. Last but not least, DeepDiff gives you the exact path of the item(s) that were changed in Python syntax. Example in Json Patch for replacing: { "op": "replace", "path": "/a/b/c", "value": 42 } Example in DeepDiff for the same operation: >>> item1 = {'a':{'b':{'c':'foo'}}} >>> item2 = {'a':{'b':{'c':42}}} >>> DeepDiff(item1, item2) {'type_changes': {"root['a']['b']['c']": {'old_type': , 'new_value': 42, 'old_value': 'foo', 'new_type': >> t1 = {1: 1, 2: 2, 3: 3} >>> t2 = {1: 1, 2: "2", 3: 3} >>> ddiff = DeepDiff(t1, t2) >>> jsoned = ddiff.json >>> jsoned '{"type_changes": {"root[2]": {"py/object": "deepdiff.helper.RemapDict", "new_type": {"py/type": "__builtin__.str"}, "new_value": "2", "old_type": {"py/type": "__builtin__.int"}, "old_value": 2}}}' >>> ddiff_new = DeepDiff.from_json(jsoned) >>> ddiff == ddiff_new True **Pycon 2016** I was honored to give a talk about how DeepDiff does what it does at Pycon 2016. Please check out the video and let me know what you think: Diff It To Dig It Video https://www.youtube.com/watch?v=J5r99eJIxF4 And here is more info: http://zepworks.com/blog/diff-it-to-digg-it/ **Changelog** - v3-3-0: Searching for objects and class attributes - v3-2-2: Adding help(deepdiff) - v3-2-1: Fixing hash of None - v3-2-0: Adding grep for search: object | grep(item) - v3-1-3: Unicode vs. Bytes default fix - v3-1-2: NotPresent Fix when item is added or removed. - v3-1-1: Bug fix when item value is None (#58) - v3-1-0: Serialization to/from json - v3-0-0: Introducing Tree View - v2-5-3: Bug fix on logging for content hash. - v2-5-2: Bug fixes on content hash. - v2-5-0: Adding ContentHash module to fix ignore_order once and for all. - v2-1-0: Adding Deep Search. Now you can search for item in an object. - v2-0-0: Exclusion patterns better coverage. Updating docs. - v1-8-0: Exclusion patterns. - v1-7-0: Deep Set comparison. - v1-6-0: Unifying key names. i.e newvalue is new_value now. For backward compatibility, newvalue still works. - v1-5-0: Fixing ignore order containers with unordered items. Adding significant digits when comparing decimals. Changes property is deprecated. - v1-1-0: Changing Set, Dictionary and Object Attribute Add/Removal to be reported as Set instead of List. Adding Pypy compatibility. - v1-0-2: Checking for ImmutableMapping type instead of dict - v1-0-1: Better ignore order support - v1-0-0: Restructuring output to make it more useful. This is NOT backward compatible. - v0-6-1: Fixiing iterables with unhashable when order is ignored - v0-6-0: Adding unicode support - v0-5-9: Adding decimal support - v0-5-8: Adding ignore order of unhashables support - v0-5-7: Adding ignore order support - v0-5-6: Adding slots support - v0-5-5: Adding loop detection **Authors** Sep Dehpour Github: https://github.com/seperman Linkedin: http://www.linkedin.com/in/sepehr ZepWorks: http://www.zepworks.com Article about Deepdiff: http://zepworks.com/blog/diff-it-to-digg-it/ Victor Hahn Castell - [hahncastell.de](http://hahncastell.de) - [flexoptix.net](http://www.flexoptix.net) Also thanks to: - nfvs for Travis-CI setup script. - brbsix for initial Py3 porting. - WangFenjin for unicode support. - timoilya for comparing list of sets when ignoring order. - Bernhard10 for significant digits comparison. - b-jazz for PEP257 cleanup, Standardize on full names, fixing line endings. - finnhughes for fixing __slots__ - moloney for Unicode vs. Bytes default - serv-inc for adding help(deepdiff) - movermeyer for updating docs - maxrothman for search in inherited class attributes - maxrothman for search for types/objects deepdiff-3.3.0/deepdiff/000077500000000000000000000000001312554330700150345ustar00rootroot00000000000000deepdiff-3.3.0/deepdiff/__init__.py000066400000000000000000000005071312554330700171470ustar00rootroot00000000000000"""This module offers the DeepDiff, DeepSearch, grep and DeepHash classes.""" import logging if __name__ == '__main__': logging.basicConfig(format='%(asctime)s %(levelname)8s %(message)s') from .diff import DeepDiff from .search import DeepSearch, grep from .contenthash import DeepHash from .helper import py3 deepdiff-3.3.0/deepdiff/contenthash.py000066400000000000000000000202561312554330700177310ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- from __future__ import absolute_import from __future__ import print_function import sys from collections import Iterable from collections import MutableMapping from collections import defaultdict from decimal import Decimal from hashlib import sha1 import logging from deepdiff.helper import py3, int, strings, numbers, items logger = logging.getLogger(__name__) class Skipped(object): def __repr__(self): return "Skipped" # pragma: no cover def __str__(self): return "Skipped" # pragma: no cover class Unprocessed(object): def __repr__(self): return "Error: Unprocessed" # pragma: no cover def __str__(self): return "Error: Unprocessed" # pragma: no cover class NotHashed(object): def __repr__(self): return "Error: NotHashed" # pragma: no cover def __str__(self): return "Error: NotHashed" # pragma: no cover class DeepHash(dict): r""" **DeepHash** """ def __init__(self, obj, hashes=None, exclude_types=set(), hasher=hash, ignore_repetition=True, significant_digits=None, **kwargs): if kwargs: raise ValueError( ("The following parameter(s) are not valid: %s\n" "The valid parameters are obj, hashes, exclude_types." "hasher and ignore_repetition.") % ', '.join(kwargs.keys())) self.obj = obj self.exclude_types = set(exclude_types) self.exclude_types_tuple = tuple( exclude_types) # we need tuple for checking isinstance self.ignore_repetition = ignore_repetition self.hasher = hasher hashes = hashes if hashes else {} self.update(hashes) self['unprocessed'] = [] self.unprocessed = Unprocessed() self.skipped = Skipped() self.not_hashed = NotHashed() self.significant_digits = significant_digits self.__hash(obj, parents_ids=frozenset({id(obj)})) if self['unprocessed']: logger.warning("Can not hash the following items: {}.".format(self['unprocessed'])) else: del self['unprocessed'] @staticmethod def sha1hex(obj): """Use Sha1 for more accuracy.""" if py3: # pragma: no cover if isinstance(obj, str): obj = "{}:{}".format(type(obj).__name__, obj) obj = obj.encode('utf-8') elif isinstance(obj, bytes): obj = type(obj).__name__.encode('utf-8') + b":" + obj else: # pragma: no cover if isinstance(obj, unicode): obj = u"{}:{}".format(type(obj).__name__, obj) obj = obj.encode('utf-8') elif isinstance(obj, str): obj = type(obj).__name__ + ":" + obj return sha1(obj).hexdigest() @staticmethod def __add_to_frozen_set(parents_ids, item_id): parents_ids = set(parents_ids) parents_ids.add(item_id) return frozenset(parents_ids) def __get_and_set_str_hash(self, obj): obj_id = id(obj) result = self.hasher(obj) result = "str:{}".format(result) self[obj_id] = result return result def __hash_obj(self, obj, parents_ids=frozenset({}), is_namedtuple=False): """Difference of 2 objects""" try: if is_namedtuple: obj = obj._asdict() else: obj = obj.__dict__ except AttributeError: try: obj = {i: getattr(obj, i) for i in obj.__slots__} except AttributeError: self['unprocessed'].append(obj) return self.unprocessed result = self.__hash_dict(obj, parents_ids) result = "nt{}".format(result) if is_namedtuple else "obj{}".format( result) return result def __skip_this(self, obj): skip = False if isinstance(obj, self.exclude_types_tuple): skip = True return skip def __hash_dict(self, obj, parents_ids=frozenset({})): result = [] obj_keys = set(obj.keys()) for key in obj_keys: key_hash = self.__hash(key) item = obj[key] item_id = id(item) if parents_ids and item_id in parents_ids: continue parents_ids_added = self.__add_to_frozen_set(parents_ids, item_id) hashed = self.__hash(item, parents_ids_added) hashed = "{}:{}".format(key_hash, hashed) result.append(hashed) result.sort() result = ';'.join(result) result = "dict:{%s}" % result return result def __hash_set(self, obj): return "set:{}".format(self.__hash_iterable(obj)) def __hash_iterable(self, obj, parents_ids=frozenset({})): result = defaultdict(int) for i, x in enumerate(obj): if self.__skip_this(x): continue item_id = id(x) if parents_ids and item_id in parents_ids: continue parents_ids_added = self.__add_to_frozen_set(parents_ids, item_id) hashed = self.__hash(x, parents_ids_added) result[hashed] += 1 if self.ignore_repetition: result = list(result.keys()) else: result = [ '{}|{}'.format(i[0], i[1]) for i in getattr(result, items)() ] result.sort() result = ','.join(result) result = "{}:{}".format(type(obj).__name__, result) return result def __hash_str(self, obj): return self.__get_and_set_str_hash(obj) def __hash_number(self, obj): # Based on diff.DeepDiff.__diff_numbers if self.significant_digits is not None and isinstance(obj, ( float, complex, Decimal)): obj_s = ("{:.%sf}" % self.significant_digits).format(obj) # Special case for 0: "-0.00" should compare equal to "0.00" if set(obj_s) <= set("-0."): obj_s = "0.00" result = "number:{}".format(obj_s) obj_id = id(obj) self[obj_id] = result else: result = "{}:{}".format(type(obj).__name__, obj) return result def __hash_tuple(self, obj, parents_ids): # Checking to see if it has _fields. Which probably means it is a named # tuple. try: obj._asdict # It must be a normal tuple except AttributeError: result = self.__hash_iterable(obj, parents_ids) # We assume it is a namedtuple then else: result = self.__hash_obj(obj, parents_ids, is_namedtuple=True) return result def __hash(self, obj, parent="root", parents_ids=frozenset({})): """The main diff method""" obj_id = id(obj) if obj_id in self: return self[obj_id] result = self.not_hashed if self.__skip_this(obj): result = self.skipped elif obj is None: result = 'NONE' elif isinstance(obj, strings): result = self.__hash_str(obj) elif isinstance(obj, numbers): result = self.__hash_number(obj) elif isinstance(obj, MutableMapping): result = self.__hash_dict(obj, parents_ids) elif isinstance(obj, tuple): result = self.__hash_tuple(obj, parents_ids) elif isinstance(obj, (set, frozenset)): result = self.__hash_set(obj) elif isinstance(obj, Iterable): result = self.__hash_iterable(obj, parents_ids) else: result = self.__hash_obj(obj, parents_ids) if result != self.not_hashed and obj_id not in self and not isinstance( obj, numbers): self[obj_id] = result if result is self.not_hashed: # pragma: no cover self[obj_id] = self.not_hashed self['unprocessed'].append(obj) return result if __name__ == "__main__": # pragma: no cover if not py3: sys.exit("Please run with Python 3 to verify the doc strings.") import doctest doctest.testmod() deepdiff-3.3.0/deepdiff/diff.py000066400000000000000000001321771312554330700163310ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # In order to run the docstrings: # python3 -m deepdiff.diff # You might need to run it many times since dictionaries come in different orders # every time you run the docstrings. # However the docstring expects it in a specific order in order to pass! from __future__ import absolute_import from __future__ import print_function import difflib import logging import jsonpickle from decimal import Decimal from collections import Mapping from collections import Iterable from deepdiff.helper import py3, strings, bytes_type, numbers, ListItemRemovedOrAdded, notpresent, IndexedHash, Verbose from deepdiff.model import RemapDict, ResultDict, TextResult, TreeResult, DiffLevel from deepdiff.model import DictRelationship, AttributeRelationship # , REPORT_KEYS from deepdiff.model import SubscriptableIterableRelationship, NonSubscriptableIterableRelationship, SetRelationship from deepdiff.contenthash import DeepHash if py3: # pragma: no cover from itertools import zip_longest else: # pragma: no cover from itertools import izip_longest as zip_longest logger = logging.getLogger(__name__) class DeepDiff(ResultDict): r""" **DeepDiff** Deep Difference of dictionaries, iterables, strings and almost any other object. It will recursively look for all the changes. DeepDiff 3.0 added the concept of views. There is a default "text" view and a "tree" view. **Parameters** t1 : A dictionary, list, string or any python object that has __dict__ or __slots__ This is the first item to be compared to the second item t2 : dictionary, list, string or almost any python object that has __dict__ or __slots__ The second item is to be compared to the first one ignore_order : Boolean, defalt=False ignores orders for iterables. Note that if you have iterables contatining any unhashable, ignoring order can be expensive. Normally ignore_order does not report duplicates and repetition changes. In order to report repetitions, set report_repetition=True in addition to ignore_order=True report_repetition : Boolean, default=False reports repetitions when set True ONLY when ignore_order is set True too. This works for iterables. This feature currently is experimental and is not production ready. significant_digits : int >= 0, default=None. If it is a non negative integer, it compares only that many digits AFTER the decimal point. This only affects floats, decimal.Decimal and complex. Internally it uses "{:.Xf}".format(Your Number) to compare numbers where X=significant_digits Note that "{:.3f}".format(1.1135) = 1.113, but "{:.3f}".format(1.11351) = 1.114 For Decimals, Python's format rounds 2.5 to 2 and 3.5 to 4 (to the closest even number) verbose_level : int >= 0, default = 1. Higher verbose level shows you more details. For example verbose level 1 shows what dictionary item are added or removed. And verbose level 2 shows the value of the items that are added or removed too. exclude_paths: list, default = None. List of paths to exclude from the report. exclude_types: list, default = None. List of object types to exclude from the report. view: string, default = text Starting the version 3 you can choosethe view into the deepdiff results. The default is the text view which has been the only view up until now. The new view is called the tree view which allows you to traverse through the tree of changed items. **Returns** A DeepDiff object that has already calculated the difference of the 2 items. **Supported data types** int, string, unicode, dictionary, list, tuple, set, frozenset, OrderedDict, NamedTuple and custom objects! **Text View** Text view is the original and currently the default view of DeepDiff. It is called text view because the results contain texts that represent the path to the data: Example of using the text view. >>> from deepdiff import DeepDiff >>> t1 = {1:1, 3:3, 4:4} >>> t2 = {1:1, 3:3, 5:5, 6:6} >>> ddiff = DeepDiff(t1, t2) >>> print(ddiff) {'dictionary_item_added': {'root[5]', 'root[6]'}, 'dictionary_item_removed': {'root[4]'}} So for example ddiff['dictionary_item_removed'] is a set if strings thus this is called the text view. .. seealso:: The following examples are using the *default text view.* The Tree View is introduced in DeepDiff v3 and provides traversing capabilitie through your diffed data and more! Read more about the Tree View at the bottom of this page. Importing >>> from deepdiff import DeepDiff >>> from pprint import pprint Same object returns empty >>> t1 = {1:1, 2:2, 3:3} >>> t2 = t1 >>> print(DeepDiff(t1, t2)) {} Type of an item has changed >>> t1 = {1:1, 2:2, 3:3} >>> t2 = {1:1, 2:"2", 3:3} >>> pprint(DeepDiff(t1, t2), indent=2) { 'type_changes': { 'root[2]': { 'new_type': , 'new_value': '2', 'old_type': , 'old_value': 2}}} Value of an item has changed >>> t1 = {1:1, 2:2, 3:3} >>> t2 = {1:1, 2:4, 3:3} >>> pprint(DeepDiff(t1, t2, verbose_level=0), indent=2) {'values_changed': {'root[2]': {'new_value': 4, 'old_value': 2}}} Item added and/or removed >>> t1 = {1:1, 3:3, 4:4} >>> t2 = {1:1, 3:3, 5:5, 6:6} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff) {'dictionary_item_added': {'root[5]', 'root[6]'}, 'dictionary_item_removed': {'root[4]'}} Set verbose level to 2 in order to see the added or removed items with their values >>> t1 = {1:1, 3:3, 4:4} >>> t2 = {1:1, 3:3, 5:5, 6:6} >>> ddiff = DeepDiff(t1, t2, verbose_level=2) >>> pprint(ddiff, indent=2) { 'dictionary_item_added': {'root[5]': 5, 'root[6]': 6}, 'dictionary_item_removed': {'root[4]': 4}} String difference >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":"world"}} >>> t2 = {1:1, 2:4, 3:3, 4:{"a":"hello", "b":"world!"}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'values_changed': { 'root[2]': {'new_value': 4, 'old_value': 2}, "root[4]['b']": { 'new_value': 'world!', 'old_value': 'world'}}} String difference 2 >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":"world!\nGoodbye!\n1\n2\nEnd"}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":"world\n1\n2\nEnd"}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'values_changed': { "root[4]['b']": { 'diff': '--- \n' '+++ \n' '@@ -1,5 +1,4 @@\n' '-world!\n' '-Goodbye!\n' '+world\n' ' 1\n' ' 2\n' ' End', 'new_value': 'world\n1\n2\nEnd', 'old_value': 'world!\n' 'Goodbye!\n' '1\n' '2\n' 'End'}}} >>> >>> print (ddiff['values_changed']["root[4]['b']"]["diff"]) --- +++ @@ -1,5 +1,4 @@ -world! -Goodbye! +world 1 2 End Type change >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":"world\n\n\nEnd"}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'type_changes': { "root[4]['b']": { 'new_type': , 'new_value': 'world\n\n\nEnd', 'old_type': , 'old_value': [1, 2, 3]}}} And if you don't care about the value of items that have changed type, please set verbose level to 0 >>> t1 = {1:1, 2:2, 3:3} >>> t2 = {1:1, 2:"2", 3:3} >>> pprint(DeepDiff(t1, t2, verbose_level=0), indent=2) { 'type_changes': { 'root[2]': { 'new_type': , 'old_type': }}} List difference >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3, 4]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2]}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) {'iterable_item_removed': {"root[4]['b'][2]": 3, "root[4]['b'][3]": 4}} List difference 2: >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 3, 2, 3]}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'iterable_item_added': {"root[4]['b'][3]": 3}, 'values_changed': { "root[4]['b'][1]": {'new_value': 3, 'old_value': 2}, "root[4]['b'][2]": {'new_value': 2, 'old_value': 3}}} List difference ignoring order or duplicates: (with the same dictionaries as above) >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 3, 2, 3]}} >>> ddiff = DeepDiff(t1, t2, ignore_order=True) >>> print (ddiff) {} List difference ignoring order but reporting repetitions: >>> from deepdiff import DeepDiff >>> from pprint import pprint >>> t1 = [1, 3, 1, 4] >>> t2 = [4, 4, 1] >>> ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True) >>> pprint(ddiff, indent=2) { 'iterable_item_removed': {'root[1]': 3}, 'repetition_change': { 'root[0]': { 'new_indexes': [2], 'new_repeat': 1, 'old_indexes': [0, 2], 'old_repeat': 2, 'value': 1}, 'root[3]': { 'new_indexes': [0, 1], 'new_repeat': 2, 'old_indexes': [3], 'old_repeat': 1, 'value': 4}}} List that contains dictionary: >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, {1:1, 2:2}]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, {1:3}]}} >>> ddiff = DeepDiff(t1, t2) >>> pprint (ddiff, indent = 2) { 'dictionary_item_removed': {"root[4]['b'][2][2]"}, 'values_changed': {"root[4]['b'][2][1]": {'new_value': 3, 'old_value': 1}}} Sets: >>> t1 = {1, 2, 8} >>> t2 = {1, 2, 3, 5} >>> ddiff = DeepDiff(t1, t2) >>> pprint(ddiff) {'set_item_added': {'root[5]', 'root[3]'}, 'set_item_removed': {'root[8]'}} Named Tuples: >>> from collections import namedtuple >>> Point = namedtuple('Point', ['x', 'y']) >>> t1 = Point(x=11, y=22) >>> t2 = Point(x=11, y=23) >>> pprint (DeepDiff(t1, t2)) {'values_changed': {'root.y': {'new_value': 23, 'old_value': 22}}} Custom objects: >>> class ClassA(object): ... a = 1 ... def __init__(self, b): ... self.b = b ... >>> t1 = ClassA(1) >>> t2 = ClassA(2) >>> >>> pprint(DeepDiff(t1, t2)) {'values_changed': {'root.b': {'new_value': 2, 'old_value': 1}}} Object attribute added: >>> t2.c = "new attribute" >>> pprint(DeepDiff(t1, t2)) {'attribute_added': {'root.c'}, 'values_changed': {'root.b': {'new_value': 2, 'old_value': 1}}} Approximate decimals comparison (Significant digits after the point): >>> t1 = Decimal('1.52') >>> t2 = Decimal('1.57') >>> DeepDiff(t1, t2, significant_digits=0) {} >>> DeepDiff(t1, t2, significant_digits=1) {'values_changed': {'root': {'old_value': Decimal('1.52'), 'new_value': Decimal('1.57')}}} Approximate float comparison (Significant digits after the point): >>> t1 = [ 1.1129, 1.3359 ] >>> t2 = [ 1.113, 1.3362 ] >>> pprint(DeepDiff(t1, t2, significant_digits=3)) {} >>> pprint(DeepDiff(t1, t2)) {'values_changed': {'root[0]': {'new_value': 1.113, 'old_value': 1.1129}, 'root[1]': {'new_value': 1.3362, 'old_value': 1.3359}}} >>> pprint(DeepDiff(1.23*10**20, 1.24*10**20, significant_digits=1)) {'values_changed': {'root': {'new_value': 1.24e+20, 'old_value': 1.23e+20}}} .. note:: All the examples for the text view work for the tree view too. You just need to set view='tree' to get it in tree form. **Tree View** Starting the version 3 You can chooe the view into the deepdiff results. The tree view provides you with tree objects that you can traverse through to find the parents of the objects that are diffed and the actual objects that are being diffed. This view is very useful when dealing with nested objects. Note that tree view always returns results in the form of Python sets. You can traverse through the tree elements! .. note:: The Tree view is just a different representation of the diffed data. Behind the scene, DeepDiff creates the tree view first and then converts it to textual representation for the text view. .. code:: text +---------------------------------------------------------------+ | | | parent(t1) parent node parent(t2) | | + ^ + | +------|--------------------------|---------------------|-------+ | | | up | | Child | | | ChildRelationship | Relationship | | | | down | | | +------|----------------------|-------------------------|-------+ | v v v | | child(t1) child node child(t2) | | | +---------------------------------------------------------------+ :up: Move up to the parent node :down: Move down to the child node :path(): Get the path to the current node :t1: The first item in the current node that is being diffed :t2: The second item in the current node that is being diffed :additional: Additional information about the node i.e. repetition :repetition: Shortcut to get the repetition report The tree view allows you to have more than mere textual representaion of the diffed objects. It gives you the actual objects (t1, t2) throughout the tree of parents and children. **Examples Tree View** .. note:: The Tree View is introduced in DeepDiff 3. Set view='tree' in order to use this view. Value of an item has changed (Tree View) >>> from deepdiff import DeepDiff >>> from pprint import pprint >>> t1 = {1:1, 2:2, 3:3} >>> t2 = {1:1, 2:4, 3:3} >>> ddiff_verbose0 = DeepDiff(t1, t2, verbose_level=0, view='tree') >>> ddiff_verbose0 {'values_changed': {}} >>> >>> ddiff_verbose1 = DeepDiff(t1, t2, verbose_level=1, view='tree') >>> ddiff_verbose1 {'values_changed': {}} >>> set_of_values_changed = ddiff_verbose1['values_changed'] >>> # since set_of_values_changed includes only one item in a set >>> # in order to get that one item we can: >>> (changed,) = set_of_values_changed >>> changed # Another way to get this is to do: changed=list(set_of_values_changed)[0] >>> changed.t1 2 >>> changed.t2 4 >>> # You can traverse through the tree, get to the parents! >>> changed.up List difference (Tree View) >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3, 4]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2]}} >>> ddiff = DeepDiff(t1, t2, view='tree') >>> ddiff {'iterable_item_removed': {, }} >>> # Note that the iterable_item_removed is a set. In this case it has 2 items in it. >>> # One way to get one item from the set is to convert it to a list >>> # And then get the first item of the list: >>> removed = list(ddiff['iterable_item_removed'])[0] >>> removed >>> >>> parent = removed.up >>> parent >>> parent.path() "root[4]['b']" >>> parent.t1 [1, 2, 3, 4] >>> parent.t2 [1, 2] >>> parent.up >>> parent.up.up >>> parent.up.up.t1 {1: 1, 2: 2, 3: 3, 4: {'a': 'hello', 'b': [1, 2, 3, 4]}} >>> parent.up.up.t1 == t1 # It is holding the original t1 that we passed to DeepDiff True List difference 2 (Tree View) >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 3, 2, 3]}} >>> ddiff = DeepDiff(t1, t2, view='tree') >>> pprint(ddiff, indent = 2) { 'iterable_item_added': {}, 'values_changed': { , }} >>> >>> # Note that iterable_item_added is a set with one item. >>> # So in order to get that one item from it, we can do: >>> >>> (added,) = ddiff['iterable_item_added'] >>> added >>> added.up.up >>> added.up.up.path() 'root[4]' >>> added.up.up.down >>> >>> # going up twice and then down twice gives you the same node in the tree: >>> added.up.up.down.down == added True List difference ignoring order but reporting repetitions (Tree View) >>> t1 = [1, 3, 1, 4] >>> t2 = [4, 4, 1] >>> ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True, view='tree') >>> pprint(ddiff, indent=2) { 'iterable_item_removed': {}, 'repetition_change': { , }} >>> >>> # repetition_change is a set with 2 items. >>> # in order to get those 2 items, we can do the following. >>> # or we can convert the set to list and get the list items. >>> # or we can iterate through the set items >>> >>> (repeat1, repeat2) = ddiff['repetition_change'] >>> repeat1 # the default verbosity is set to 1. >>> # The actual data regarding the repetitions can be found in the repetition attribute: >>> repeat1.repetition {'old_repeat': 1, 'new_repeat': 2, 'old_indexes': [3], 'new_indexes': [0, 1]} >>> >>> # If you change the verbosity, you will see less: >>> ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True, view='tree', verbose_level=0) >>> ddiff {'repetition_change': {, }, 'iterable_item_removed': {}} >>> (repeat1, repeat2) = ddiff['repetition_change'] >>> repeat1 >>> >>> # But the verbosity level does not change the actual report object. >>> # It only changes the textual representaion of the object. We get the actual object here: >>> repeat1.repetition {'old_repeat': 1, 'new_repeat': 2, 'old_indexes': [3], 'new_indexes': [0, 1]} >>> repeat1.t1 4 >>> repeat1.t2 4 >>> repeat1.up List that contains dictionary (Tree View) >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, {1:1, 2:2}]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, {1:3}]}} >>> ddiff = DeepDiff(t1, t2, view='tree') >>> pprint (ddiff, indent = 2) { 'dictionary_item_removed': {}, 'values_changed': {}} Sets (Tree View): >>> t1 = {1, 2, 8} >>> t2 = {1, 2, 3, 5} >>> ddiff = DeepDiff(t1, t2, view='tree') >>> print(ddiff) {'set_item_removed': {}, 'set_item_added': {, }} >>> # grabbing one item from set_item_removed set which has one item only >>> (item,) = ddiff['set_item_removed'] >>> item.up >>> item.up.t1 == t1 True Named Tuples (Tree View): >>> from collections import namedtuple >>> Point = namedtuple('Point', ['x', 'y']) >>> t1 = Point(x=11, y=22) >>> t2 = Point(x=11, y=23) >>> print(DeepDiff(t1, t2, view='tree')) {'values_changed': {}} Custom objects (Tree View): >>> class ClassA(object): ... a = 1 ... def __init__(self, b): ... self.b = b ... >>> t1 = ClassA(1) >>> t2 = ClassA(2) >>> >>> print(DeepDiff(t1, t2, view='tree')) {'values_changed': {}} Object attribute added (Tree View): >>> t2.c = "new attribute" >>> pprint(DeepDiff(t1, t2, view='tree')) {'attribute_added': {}, 'values_changed': {}} Approximate decimals comparison (Significant digits after the point) (Tree View): >>> t1 = Decimal('1.52') >>> t2 = Decimal('1.57') >>> DeepDiff(t1, t2, significant_digits=0, view='tree') {} >>> ddiff = DeepDiff(t1, t2, significant_digits=1, view='tree') >>> ddiff {'values_changed': {}} >>> (change1,) = ddiff['values_changed'] >>> change1 >>> change1.t1 Decimal('1.52') >>> change1.t2 Decimal('1.57') >>> change1.path() 'root' Approximate float comparison (Significant digits after the point) (Tree View): >>> t1 = [ 1.1129, 1.3359 ] >>> t2 = [ 1.113, 1.3362 ] >>> ddiff = DeepDiff(t1, t2, significant_digits=3, view='tree') >>> ddiff {} >>> ddiff = DeepDiff(t1, t2, view='tree') >>> pprint(ddiff, indent=2) { 'values_changed': { , }} >>> ddiff = DeepDiff(1.23*10**20, 1.24*10**20, significant_digits=1, view='tree') >>> ddiff {'values_changed': {}} .. note:: All the examples for the text view work for the tree view too. You just need to set view='tree' to get it in tree form. **Serialization** DeepDiff uses jsonpickle in order to serialize and deserialize its results into json. Serialize and then deserialize back to deepdiff >>> t1 = {1: 1, 2: 2, 3: 3} >>> t2 = {1: 1, 2: "2", 3: 3} >>> ddiff = DeepDiff(t1, t2) >>> jsoned = ddiff.json >>> jsoned '{"type_changes": {"root[2]": {"py/object": "deepdiff.helper.RemapDict", "new_type": {"py/type": "__builtin__.str"}, "new_value": "2", "old_type": {"py/type": "__builtin__.int"}, "old_value": 2}}}' >>> ddiff_new = DeepDiff.from_json(jsoned) >>> ddiff == ddiff_new True **Pycon 2016 Talk** I gave a talk about how DeepDiff does what it does at Pycon 2016. `Diff it to Dig it Pycon 2016 video `_ And here is more info: http://zepworks.com/blog/diff-it-to-digg-it/ """ def __init__(self, t1, t2, ignore_order=False, report_repetition=False, significant_digits=None, exclude_paths=set(), exclude_types=set(), verbose_level=1, view='text', **kwargs): if kwargs: raise ValueError(( "The following parameter(s) are not valid: %s\n" "The valid parameters are ignore_order, report_repetition, significant_digits," "exclude_paths, exclude_types, verbose_level and view.") % ', '.join(kwargs.keys())) self.ignore_order = ignore_order self.report_repetition = report_repetition self.exclude_paths = set(exclude_paths) self.exclude_types = set(exclude_types) self.exclude_types_tuple = tuple( exclude_types) # we need tuple for checking isinstance self.hashes = {} if significant_digits is not None and significant_digits < 0: raise ValueError( "significant_digits must be None or a non-negative integer") self.significant_digits = significant_digits self.tree = TreeResult() Verbose.level = verbose_level root = DiffLevel(t1, t2) self.__diff(root, parents_ids=frozenset({id(t1)})) self.tree.cleanup() if view == 'tree': self.update(self.tree) del self.tree else: result_text = TextResult(tree_results=self.tree) result_text.cleanup() # clean up text-style result dictionary self.update( result_text ) # be compatible to DeepDiff 2.x if user didn't specify otherwise # TODO: adding adding functionality # def __add__(self, other): # if isinstance(other, DeepDiff): # result = deepcopy(self) # result.update(other) # else: # result = deepcopy(other) # for key in REPORT_KEYS: # if key in self: # getattr(self, "_do_{}".format(key))(result) # return result # __radd__ = __add__ # def _do_iterable_item_added(self, result): # for item in self['iterable_item_added']: # pass def __report_result(self, report_type, level): """ Add a detected change to the reference-style result dictionary. report_type will be added to level. (We'll create the text-style report from there later.) :param report_type: A well defined string key describing the type of change. Examples: "set_item_added", "values_changed" :param parent: A DiffLevel object describing the objects in question in their before-change and after-change object structure. :rtype: None """ if not self.__skip_this(level): level.report_type = report_type self.tree[report_type].add(level) @staticmethod def __add_to_frozen_set(parents_ids, item_id): parents_ids = set(parents_ids) parents_ids.add(item_id) return frozenset(parents_ids) @staticmethod def __dict_from_slots(object): def unmangle(attribute): if attribute.startswith('__'): return '_{type}{attribute}'.format( type=type(object).__name__, attribute=attribute ) return attribute slots = object.__slots__ if isinstance(slots, strings): return {slots: getattr(object, unmangle(slots))} return {i: getattr(object, unmangle(i)) for i in slots} def __diff_obj(self, level, parents_ids=frozenset({}), is_namedtuple=False): """Difference of 2 objects""" try: if is_namedtuple: t1 = level.t1._asdict() t2 = level.t2._asdict() else: t1 = level.t1.__dict__ t2 = level.t2.__dict__ except AttributeError: try: t1 = self.__dict_from_slots(level.t1) t2 = self.__dict_from_slots(level.t2) except AttributeError: self.__report_result('unprocessed', level) return self.__diff_dict( level, parents_ids, print_as_attribute=True, override=True, override_t1=t1, override_t2=t2) def __skip_this(self, level): """ Check whether this comparison should be skipped because one of the objects to compare meets exclusion criteria. :rtype: bool """ skip = False if self.exclude_paths and level.path() in self.exclude_paths: skip = True else: if isinstance(level.t1, self.exclude_types_tuple) or isinstance( level.t2, self.exclude_types_tuple): skip = True return skip def __diff_dict(self, level, parents_ids=frozenset({}), print_as_attribute=False, override=False, override_t1=None, override_t2=None): """Difference of 2 dictionaries""" if override: # for special stuff like custom objects and named tuples we receive preprocessed t1 and t2 # but must not spoil the chain (=level) with it t1 = override_t1 t2 = override_t2 else: t1 = level.t1 t2 = level.t2 if print_as_attribute: item_added_key = "attribute_added" item_removed_key = "attribute_removed" rel_class = AttributeRelationship else: item_added_key = "dictionary_item_added" item_removed_key = "dictionary_item_removed" rel_class = DictRelationship t1_keys = set(t1.keys()) t2_keys = set(t2.keys()) t_keys_intersect = t2_keys.intersection(t1_keys) t_keys_added = t2_keys - t_keys_intersect t_keys_removed = t1_keys - t_keys_intersect for key in t_keys_added: change_level = level.branch_deeper( notpresent, t2[key], child_relationship_class=rel_class, child_relationship_param=key) self.__report_result(item_added_key, change_level) for key in t_keys_removed: change_level = level.branch_deeper( t1[key], notpresent, child_relationship_class=rel_class, child_relationship_param=key) self.__report_result(item_removed_key, change_level) for key in t_keys_intersect: # key present in both dicts - need to compare values item_id = id(t1[key]) if parents_ids and item_id in parents_ids: continue parents_ids_added = self.__add_to_frozen_set(parents_ids, item_id) # Go one level deeper next_level = level.branch_deeper( t1[key], t2[key], child_relationship_class=rel_class, child_relationship_param=key) self.__diff(next_level, parents_ids_added) def __diff_set(self, level): """Difference of sets""" t1_hashtable = self.__create_hashtable(level.t1, level) t2_hashtable = self.__create_hashtable(level.t2, level) t1_hashes = set(t1_hashtable.keys()) t2_hashes = set(t2_hashtable.keys()) hashes_added = t2_hashes - t1_hashes hashes_removed = t1_hashes - t2_hashes items_added = [t2_hashtable[i].item for i in hashes_added] items_removed = [t1_hashtable[i].item for i in hashes_removed] for item in items_added: change_level = level.branch_deeper( notpresent, item, child_relationship_class=SetRelationship) self.__report_result('set_item_added', change_level) for item in items_removed: change_level = level.branch_deeper( item, notpresent, child_relationship_class=SetRelationship) self.__report_result('set_item_removed', change_level) @staticmethod def __iterables_subscriptable(t1, t2): try: if getattr(t1, '__getitem__') and getattr(t2, '__getitem__'): return True else: # pragma: no cover return False # should never happen except AttributeError: return False def __diff_iterable(self, level, parents_ids=frozenset({})): """Difference of iterables""" # We're handling both subscriptable and non-subscriptable iterables. Which one is it? subscriptable = self.__iterables_subscriptable(level.t1, level.t2) if subscriptable: child_relationship_class = SubscriptableIterableRelationship else: child_relationship_class = NonSubscriptableIterableRelationship for i, (x, y) in enumerate( zip_longest( level.t1, level.t2, fillvalue=ListItemRemovedOrAdded)): if y is ListItemRemovedOrAdded: # item removed completely change_level = level.branch_deeper( x, notpresent, child_relationship_class=child_relationship_class, child_relationship_param=i) self.__report_result('iterable_item_removed', change_level) elif x is ListItemRemovedOrAdded: # new item added change_level = level.branch_deeper( notpresent, y, child_relationship_class=child_relationship_class, child_relationship_param=i) self.__report_result('iterable_item_added', change_level) else: # check if item value has changed item_id = id(x) if parents_ids and item_id in parents_ids: continue parents_ids_added = self.__add_to_frozen_set(parents_ids, item_id) # Go one level deeper next_level = level.branch_deeper( x, y, child_relationship_class=child_relationship_class, child_relationship_param=i) self.__diff(next_level, parents_ids_added) def __diff_str(self, level): """Compare strings""" if level.t1 == level.t2: return # do we add a diff for convenience? do_diff = True if isinstance(level.t1, bytes_type): try: t1_str = level.t1.decode('ascii') t2_str = level.t2.decode('ascii') except UnicodeDecodeError: do_diff = False else: t1_str = level.t1 t2_str = level.t2 if do_diff: if u'\n' in t1_str or u'\n' in t2_str: diff = difflib.unified_diff( t1_str.splitlines(), t2_str.splitlines(), lineterm='') diff = list(diff) if diff: level.additional['diff'] = u'\n'.join(diff) self.__report_result('values_changed', level) def __diff_tuple(self, level, parents_ids): # Checking to see if it has _fields. Which probably means it is a named # tuple. try: level.t1._asdict # It must be a normal tuple except AttributeError: self.__diff_iterable(level, parents_ids) # We assume it is a namedtuple then else: self.__diff_obj(level, parents_ids, is_namedtuple=True) def __create_hashtable(self, t, level): """Create hashtable of {item_hash: item}""" def add_hash(hashes, item_hash, item, i): if item_hash in hashes: hashes[item_hash].indexes.append(i) else: hashes[item_hash] = IndexedHash([i], item) hashes = {} for (i, item) in enumerate(t): try: hashes_all = DeepHash(item, hashes=self.hashes, significant_digits=self.significant_digits) item_hash = hashes_all.get(id(item), item) except Exception as e: # pragma: no cover logger.warning("Can not produce a hash for %s." "Not counting this object.\n %s" % (level.path(), e)) else: if item_hash is hashes_all.unprocessed: # pragma: no cover logger.warning("Item %s was not processed while hashing " "thus not counting this object." % level.path()) else: add_hash(hashes, item_hash, item, i) return hashes def __diff_iterable_with_contenthash(self, level): """Diff of unhashable iterables. Only used when ignoring the order.""" t1_hashtable = self.__create_hashtable(level.t1, level) t2_hashtable = self.__create_hashtable(level.t2, level) t1_hashes = set(t1_hashtable.keys()) t2_hashes = set(t2_hashtable.keys()) hashes_added = t2_hashes - t1_hashes hashes_removed = t1_hashes - t2_hashes if self.report_repetition: for hash_value in hashes_added: for i in t2_hashtable[hash_value].indexes: change_level = level.branch_deeper( notpresent, t2_hashtable[hash_value].item, child_relationship_class=SubscriptableIterableRelationship, # TODO: that might be a lie! child_relationship_param=i ) # TODO: what is this value exactly? self.__report_result('iterable_item_added', change_level) for hash_value in hashes_removed: for i in t1_hashtable[hash_value].indexes: change_level = level.branch_deeper( t1_hashtable[hash_value].item, notpresent, child_relationship_class=SubscriptableIterableRelationship, # TODO: that might be a lie! child_relationship_param=i) self.__report_result('iterable_item_removed', change_level) items_intersect = t2_hashes.intersection(t1_hashes) for hash_value in items_intersect: t1_indexes = t1_hashtable[hash_value].indexes t2_indexes = t2_hashtable[hash_value].indexes t1_indexes_len = len(t1_indexes) t2_indexes_len = len(t2_indexes) if t1_indexes_len != t2_indexes_len: # this is a repetition change! # create "change" entry, keep current level untouched to handle further changes repetition_change_level = level.branch_deeper( t1_hashtable[hash_value].item, t2_hashtable[hash_value].item, # nb: those are equal! child_relationship_class=SubscriptableIterableRelationship, # TODO: that might be a lie! child_relationship_param=t1_hashtable[hash_value] .indexes[0]) repetition_change_level.additional['repetition'] = RemapDict( old_repeat=t1_indexes_len, new_repeat=t2_indexes_len, old_indexes=t1_indexes, new_indexes=t2_indexes) self.__report_result('repetition_change', repetition_change_level) else: for hash_value in hashes_added: change_level = level.branch_deeper( notpresent, t2_hashtable[hash_value].item, child_relationship_class=SubscriptableIterableRelationship, # TODO: that might be a lie! child_relationship_param=t2_hashtable[hash_value].indexes[ 0]) # TODO: what is this value exactly? self.__report_result('iterable_item_added', change_level) for hash_value in hashes_removed: change_level = level.branch_deeper( t1_hashtable[hash_value].item, notpresent, child_relationship_class=SubscriptableIterableRelationship, # TODO: that might be a lie! child_relationship_param=t1_hashtable[hash_value].indexes[ 0]) self.__report_result('iterable_item_removed', change_level) def __diff_numbers(self, level): """Diff Numbers""" if self.significant_digits is not None and isinstance(level.t1, ( float, complex, Decimal)): # Bernhard10: I use string formatting for comparison, to be consistent with usecases where # data is read from files that were previousely written from python and # to be consistent with on-screen representation of numbers. # Other options would be abs(t1-t2)<10**-self.significant_digits # or math.is_close (python3.5+) # Note that abs(3.25-3.251) = 0.0009999999999998899 < 0.001 # Note also that "{:.3f}".format(1.1135) = 1.113, but "{:.3f}".format(1.11351) = 1.114 # For Decimals, format seems to round 2.5 to 2 and 3.5 to 4 (to closest even number) t1_s = ("{:.%sf}" % self.significant_digits).format(level.t1) t2_s = ("{:.%sf}" % self.significant_digits).format(level.t2) # Special case for 0: "-0.00" should compare equal to "0.00" if set(t1_s) <= set("-0.") and set(t2_s) <= set("-0."): return elif t1_s != t2_s: self.__report_result('values_changed', level) else: if level.t1 != level.t2: self.__report_result('values_changed', level) def __diff_types(self, level): """Diff types""" level.report_type = 'type_changes' self.__report_result('type_changes', level) def __diff(self, level, parents_ids=frozenset({})): """The main diff method""" if level.t1 is level.t2: return if self.__skip_this(level): return if type(level.t1) != type(level.t2): self.__diff_types(level) elif isinstance(level.t1, strings): self.__diff_str(level) elif isinstance(level.t1, numbers): self.__diff_numbers(level) elif isinstance(level.t1, Mapping): self.__diff_dict(level, parents_ids) elif isinstance(level.t1, tuple): self.__diff_tuple(level, parents_ids) elif isinstance(level.t1, (set, frozenset)): self.__diff_set(level) elif isinstance(level.t1, Iterable): if self.ignore_order: self.__diff_iterable_with_contenthash(level) else: self.__diff_iterable(level, parents_ids) else: self.__diff_obj(level, parents_ids) return @property def json(self): if not hasattr(self, '_json'): # copy of self removes all the extra attributes since it assumes # we have only a simple dictionary. copied = self.copy() self._json = jsonpickle.encode(copied) return self._json @json.deleter def json(self): del self._json @classmethod def from_json(self, value): return jsonpickle.decode(value) if __name__ == "__main__": # pragma: no cover if not py3: import sys sys.exit( "Please run with Python 3 to verify the doc strings: python3 -m deepdiff.diff" ) import doctest doctest.testmod() deepdiff-3.3.0/deepdiff/helper.py000066400000000000000000000057501312554330700166740ustar00rootroot00000000000000# -*- coding: utf-8 -*- import sys import datetime from decimal import Decimal from collections import namedtuple import logging logger = logging.getLogger(__name__) py_major_version = sys.version[0] py_minor_version = sys.version[2] py3 = py_major_version == '3' if (py_major_version, py_minor_version) == (2.6): # pragma: no cover sys.exit('Python 2.6 is not supported.') pypy3 = py3 and hasattr(sys, "pypy_translation_info") if py3: # pragma: no cover from builtins import int strings = (str, bytes) # which are both basestring unicode_type = str bytes_type = bytes numbers = (int, float, complex, datetime.datetime, datetime.date, Decimal) items = 'items' else: # pragma: no cover int = int strings = (str, unicode) unicode_type = unicode bytes_type = str numbers = (int, float, long, complex, datetime.datetime, datetime.date, Decimal) items = 'iteritems' IndexedHash = namedtuple('IndexedHash', 'indexes item') EXPANDED_KEY_MAP = { # pragma: no cover 'dic_item_added': 'dictionary_item_added', 'dic_item_removed': 'dictionary_item_removed', 'newindexes': 'new_indexes', 'newrepeat': 'new_repeat', 'newtype': 'new_type', 'newvalue': 'new_value', 'oldindexes': 'old_indexes', 'oldrepeat': 'old_repeat', 'oldtype': 'old_type', 'oldvalue': 'old_value'} def short_repr(item, max_length=15): """Short representation of item if it is too long""" item = repr(item) if len(item) > max_length: item = '{}...{}'.format(item[:max_length - 3], item[-1]) return item class ListItemRemovedOrAdded(object): # pragma: no cover """Class of conditions to be checked""" pass class NotPresent(object): # pragma: no cover """ In a change tree, this indicated that a previously existing object has been removed -- or will only be added in the future. We previously used None for this but this caused problem when users actually added and removed None. Srsly guys? :D """ def __repr__(self): return "Not Present" def __str__(self): return self.__repr__() notpresent = NotPresent() WARNING_NUM = 0 def warn(*args, **kwargs): global WARNING_NUM if WARNING_NUM < 10: WARNING_NUM += 1 logger.warning(*args, **kwargs) class RemapDict(dict): """ Remap Dictionary. For keys that have a new, longer name, remap the old key to the new key. Other keys that don't have a new name are handled as before. """ def __getitem__(self, old_key): new_key = EXPANDED_KEY_MAP.get(old_key, old_key) if new_key != old_key: warn( "DeepDiff Deprecation: %s is renamed to %s. Please start using " "the new unified naming convention.", old_key, new_key) if new_key in self: return self.get(new_key) else: # pragma: no cover raise KeyError(new_key) class Verbose(object): """ Global verbose level """ level = 1 deepdiff-3.3.0/deepdiff/model.py000066400000000000000000000623421312554330700165150ustar00rootroot00000000000000# -*- coding: utf-8 -*- from deepdiff.helper import items, RemapDict, strings, short_repr, Verbose, notpresent from ast import literal_eval from copy import copy FORCE_DEFAULT = 'fake' UP_DOWN = {'up': 'down', 'down': 'up'} REPORT_KEYS = { "type_changes", "dictionary_item_added", "dictionary_item_removed", "values_changed", "unprocessed", "iterable_item_added", "iterable_item_removed", "attribute_added", "attribute_removed", "set_item_removed", "set_item_added", "repetition_change", } class DoesNotExist(Exception): pass class ResultDict(RemapDict): def cleanup(self): """ Remove empty keys from this object. Should always be called after the result is final. :return: """ empty_keys = [k for k, v in getattr(self, items)() if not v] for k in empty_keys: del self[k] class TreeResult(ResultDict): def __init__(self): for key in REPORT_KEYS: self[key] = set() class TextResult(ResultDict): def __init__(self, tree_results=None): # TODO: centralize keys self.update({ "type_changes": {}, "dictionary_item_added": self.__set_or_dict(), "dictionary_item_removed": self.__set_or_dict(), "values_changed": {}, "unprocessed": [], "iterable_item_added": {}, "iterable_item_removed": {}, "attribute_added": self.__set_or_dict(), "attribute_removed": self.__set_or_dict(), "set_item_removed": set(), "set_item_added": set(), "repetition_change": {} }) if tree_results: self._from_tree_results(tree_results) def __set_or_dict(self): return {} if Verbose.level >= 2 else set() def _from_tree_results(self, tree): """ Populate this object by parsing an existing reference-style result dictionary. :param tree: A TreeResult :return: """ self._from_tree_type_changes(tree) self._from_tree_default(tree, 'dictionary_item_added') self._from_tree_default(tree, 'dictionary_item_removed') self._from_tree_value_changed(tree) self._from_tree_unprocessed(tree) self._from_tree_default(tree, 'iterable_item_added') self._from_tree_default(tree, 'iterable_item_removed') self._from_tree_default(tree, 'attribute_added') self._from_tree_default(tree, 'attribute_removed') self._from_tree_set_item_removed(tree) self._from_tree_set_item_added(tree) self._from_tree_repetition_change(tree) def _from_tree_default(self, tree, report_type): if report_type in tree: for change in tree[report_type]: # report each change # determine change direction (added or removed) # Report t2 (the new one) whenever possible. # In cases where t2 doesn't exist (i.e. stuff removed), report t1. if change.t2 is not notpresent: item = change.t2 else: item = change.t1 # do the reporting report = self[report_type] if isinstance(report, set): report.add(change.path(force=FORCE_DEFAULT)) elif isinstance(report, dict): report[change.path(force=FORCE_DEFAULT)] = item elif isinstance(report, list): # pragma: no cover # we don't actually have any of those right now, but just in case report.append(change.path(force=FORCE_DEFAULT)) else: # pragma: no cover # should never happen raise TypeError("Cannot handle {} report container type.". format(report)) def _from_tree_type_changes(self, tree): if 'type_changes' in tree: for change in tree['type_changes']: remap_dict = RemapDict({ 'old_type': type(change.t1), 'new_type': type(change.t2) }) self['type_changes'][change.path( force=FORCE_DEFAULT)] = remap_dict if Verbose.level: remap_dict.update(old_value=change.t1, new_value=change.t2) def _from_tree_value_changed(self, tree): if 'values_changed' in tree: for change in tree['values_changed']: the_changed = {'new_value': change.t2, 'old_value': change.t1} self['values_changed'][change.path( force=FORCE_DEFAULT)] = the_changed if 'diff' in change.additional: the_changed.update({'diff': change.additional['diff']}) def _from_tree_unprocessed(self, tree): if 'unprocessed' in tree: for change in tree['unprocessed']: self['unprocessed'].append("%s: %s and %s" % (change.path( force=FORCE_DEFAULT), change.t1, change.t2)) def _from_tree_set_item_removed(self, tree): if 'set_item_removed' in tree: for change in tree['set_item_removed']: path = change.up.path( ) # we want't the set's path, the removed item is not directly accessible item = change.t1 if isinstance(item, strings): item = "'%s'" % item self['set_item_removed'].add("%s[%s]" % (path, str(item))) # this syntax is rather peculiar, but it's DeepDiff 2.x compatible def _from_tree_set_item_added(self, tree): if 'set_item_added' in tree: for change in tree['set_item_added']: path = change.up.path( ) # we want't the set's path, the added item is not directly accessible item = change.t2 if isinstance(item, strings): item = "'%s'" % item self['set_item_added'].add("%s[%s]" % (path, str(item))) # this syntax is rather peculiar, but it's DeepDiff 2.x compatible) def _from_tree_repetition_change(self, tree): if 'repetition_change' in tree: for change in tree['repetition_change']: path = change.path(force=FORCE_DEFAULT) self['repetition_change'][path] = RemapDict(change.additional[ 'repetition']) self['repetition_change'][path]['value'] = change.t1 class DiffLevel(object): """ An object of this class represents a single object-tree-level in a reported change. A double-linked list of these object describes a single change on all of its levels. Looking at the tree of all changes, a list of those objects represents a single path through the tree (which is just fancy for "a change"). This is the result object class for object reference style reports. Example: >>> t1 = {2: 2, 4: 44} >>> t2 = {2: "b", 5: 55} >>> ddiff = DeepDiff(t1, t2, view='tree') >>> ddiff {'dictionary_item_added': {}, 'dictionary_item_removed': {}, 'type_changes': {}} Graph: ↑up ↑up | | | ChildRelationship | ChildRelationship | | ↓down ↓down .path() = 'root[5]' .path() = 'root[4]' Note that the 2 top level DiffLevel objects are 2 different objects even though they are essentially talking about the same diff operation. A ChildRelationship object describing the relationship between t1 and it's child object, where t1's child object equals down.t1. Think about it like a graph: +---------------------------------------------------------------+ | | | parent difflevel parent | | + ^ + | +------|--------------------------|---------------------|-------+ | | | up | | Child | | | ChildRelationship | Relationship | | | | down | | | +------|----------------------|-------------------------|-------+ | v v v | | child difflevel child | | | +---------------------------------------------------------------+ The child_rel example: # dictionary_item_removed is a set so in order to get an item from it: >>> (difflevel,) = ddiff['dictionary_item_removed']) >>> difflevel.up.t1_child_rel >>> (difflevel,) = ddiff['dictionary_item_added']) >>> difflevel >>> difflevel.up >>> >>> difflevel.up # t1 didn't exist >>> difflevel.up.t1_child_rel # t2 is added >>> difflevel.up.t2_child_rel """ def __init__(self, t1, t2, down=None, up=None, report_type=None, child_rel1=None, child_rel2=None, additional=None, verbose_level=1): """ :param child_rel1: Either: - An existing ChildRelationship object describing the "down" relationship for t1; or - A ChildRelationship subclass. In this case, we will create the ChildRelationship objects for both t1 and t2. Alternatives for child_rel1 and child_rel2 must be used consistently. :param child_rel2: Either: - An existing ChildRelationship object describing the "down" relationship for t2; or - The param argument for a ChildRelationship class we shall create. Alternatives for child_rel1 and child_rel2 must be used consistently. """ # The current-level object in the left hand tree self.t1 = t1 # The current-level object in the right hand tree self.t2 = t2 # Another DiffLevel object describing this change one level deeper down the object tree self.down = down # Another DiffLevel object describing this change one level further up the object tree self.up = up self.report_type = report_type # If this object is this change's deepest level, this contains a string describing the type of change. # Examples: "set_item_added", "values_changed" # Note: don't use {} as additional's default value - this would turn out to be always the same dict object self.additional = {} if additional is None else additional # For some types of changes we store some additional information. # This is a dict containing this information. # Currently, this is used for: # - values_changed: In case the changes data is a multi-line string, # we include a textual diff as additional['diff']. # - repetition_change: additional['repetition']: # e.g. {'old_repeat': 2, 'new_repeat': 1, 'old_indexes': [0, 2], 'new_indexes': [2]} # the user supplied ChildRelationship objects for t1 and t2 # A ChildRelationship object describing the relationship between t1 and it's child object, # where t1's child object equals down.t1. # If this relationship is representable as a string, str(self.t1_child_rel) returns a formatted param parsable python string, # e.g. "[2]", ".my_attribute" self.t1_child_rel = child_rel1 # Another ChildRelationship object describing the relationship between t2 and it's child object. self.t2_child_rel = child_rel2 # Will cache result of .path() per 'force' as key for performance self._path = {} def __repr__(self): if Verbose.level: if self.additional: additional_repr = short_repr(self.additional, max_length=35) result = "<{} {}>".format(self.path(), additional_repr) else: t1_repr = short_repr(self.t1) t2_repr = short_repr(self.t2) result = "<{} t1:{}, t2:{}>".format(self.path(), t1_repr, t2_repr) else: result = "<{}>".format(self.path()) return result def __setattr__(self, key, value): # Setting up or down, will set the opposite link in this linked list. if key in UP_DOWN and value is not None: self.__dict__[key] = value opposite_key = UP_DOWN[key] value.__dict__[opposite_key] = self else: self.__dict__[key] = value @property def repetition(self): return self.additional['repetition'] def auto_generate_child_rel(self, klass, param): """ Auto-populate self.child_rel1 and self.child_rel2. This requires self.down to be another valid DiffLevel object. :param klass: A ChildRelationship subclass describing the kind of parent-child relationship, e.g. DictRelationship. :param param: A ChildRelationship subclass-dependent parameter describing how to get from parent to child, e.g. the key in a dict """ if self.down.t1 is not notpresent: self.t1_child_rel = ChildRelationship.create( klass=klass, parent=self.t1, child=self.down.t1, param=param) if self.down.t2 is not notpresent: self.t2_child_rel = ChildRelationship.create( klass=klass, parent=self.t2, child=self.down.t2, param=param) @property def all_up(self): """ Get the root object of this comparison. (This is a convenient wrapper for following the up attribute as often as you can.) :rtype: DiffLevel """ level = self while level.up: level = level.up return level @property def all_down(self): """ Get the leaf object of this comparison. (This is a convenient wrapper for following the down attribute as often as you can.) :rtype: DiffLevel """ level = self while level.down: level = level.down return level def path(self, root="root", force=None): """ A python syntax string describing how to descend to this level, assuming the top level object is called root. Returns None if the path is not representable as a string. This might be the case for example if there are sets involved (because then there's not path at all) or because custom objects used as dictionary keys (then there is a path but it's not representable). Example: root['ingredients'][0] Note: We will follow the left side of the comparison branch, i.e. using the t1's to build the path. Using t1 or t2 should make no difference at all, except for the last step of a child-added/removed relationship. If it does in any other case, your comparison path is corrupt. :param root: The result string shall start with this var name :param force: Bends the meaning of "no string representation". If None: Will strictly return Python-parsable expressions. The result those yield will compare equal to the objects in question. If 'yes': Will return a path including '(unrepresentable)' in place of non string-representable parts. If 'fake': Will try to produce an output optimized for readability. This will pretend all iterables are subscriptable, for example. """ # TODO: We could optimize this by building on top of self.up's path if it is cached there if force in self._path: result = self._path[force] return None if result is None else "{}{}".format(root, result) result = "" level = self.all_up # start at the root # traverse all levels of this relationship while level and level is not self: # get this level's relationship object next_rel = level.t1_child_rel or level.t2_child_rel # next relationship object to get a formatted param from # t1 and t2 both are empty if next_rel is None: break # Build path for this level item = next_rel.get_param_repr(force) if item: result += item else: # it seems this path is not representable as a string result = None break # Prepare processing next level level = level.down self._path[force] = result result = None if result is None else "{}{}".format(root, result) return result def create_deeper(self, new_t1, new_t2, child_relationship_class, child_relationship_param=None, report_type=None): """ Start a new comparison level and correctly link it to this one. :rtype: DiffLevel :return: New level """ level = self.all_down result = DiffLevel( new_t1, new_t2, down=None, up=level, report_type=report_type) level.down = result level.auto_generate_child_rel( klass=child_relationship_class, param=child_relationship_param) return result def branch_deeper(self, new_t1, new_t2, child_relationship_class, child_relationship_param=None, report_type=None): """ Branch this comparison: Do not touch this comparison line, but create a new one with exactly the same content, just one level deeper. :rtype: DiffLevel :return: New level in new comparison line """ branch = self.copy() return branch.create_deeper(new_t1, new_t2, child_relationship_class, child_relationship_param, report_type) def copy(self): """ Get a deep copy of this comparision line. :return: The leaf ("downmost") object of the copy. """ orig = self.all_up result = copy(orig) # copy top level while orig is not None: result.additional = copy(orig.additional) if orig.down is not None: # copy and create references to the following level # copy following level result.down = copy(orig.down) if orig.t1_child_rel is not None: result.t1_child_rel = ChildRelationship.create( klass=orig.t1_child_rel.__class__, parent=result.t1, child=result.down.t1, param=orig.t1_child_rel.param) if orig.t2_child_rel is not None: result.t2_child_rel = ChildRelationship.create( klass=orig.t2_child_rel.__class__, parent=result.t2, child=result.down.t2, param=orig.t2_child_rel.param) # descend to next level orig = orig.down if result.down is not None: result = result.down return result class ChildRelationship(object): """ Describes the relationship between a container object (the "parent") and the contained "child" object. """ # Format to a be used for representing param. # E.g. for a dict, this turns a formatted param param "42" into "[42]". param_repr_format = None # This is a hook allowing subclasses to manipulate param strings. # :param string: Input string # :return: Manipulated string, as appropriate in this context. quote_str = None @staticmethod def create(klass, parent, child, param=None): if not issubclass(klass, ChildRelationship): raise TypeError return klass(parent, child, param) def __init__(self, parent, child, param=None): # The parent object of this relationship, e.g. a dict self.parent = parent # The child object of this relationship, e.g. a value in a dict self.child = child # A subclass-dependent parameter describing how to get from parent to child, e.g. the key in a dict self.param = param def __repr__(self): name = "<{} parent:{}, child:{}, param:{}>" parent = short_repr(self.parent) child = short_repr(self.child) param = short_repr(self.param) return name.format(self.__class__.__name__, parent, child, param) def get_param_repr(self, force=None): """ Returns a formatted param python parsable string describing this relationship, or None if the relationship is not representable as a string. This string can be appended to the parent Name. Subclasses representing a relationship that cannot be expressed as a string override this method to return None. Examples: "[2]", ".attribute", "['mykey']" :param force: Bends the meaning of "no string representation". If None: Will strictly return partials of Python-parsable expressions. The result those yield will compare equal to the objects in question. If 'yes': Will return a formatted param including '(unrepresentable)' instead of the non string-representable part. """ return self.stringify_param(force) def get_param_from_obj(self, obj): # pragma: no cover """ Get item from external object. This is used to get the item with the same path from another object. This way you can apply the path tree to any object. """ pass def stringify_param(self, force=None): """ Convert param to a string. Return None if there is no string representation. This is called by get_param_repr() :param force: Bends the meaning of "no string representation". If None: Will strictly return Python-parsable expressions. The result those yield will compare equal to the objects in question. If 'yes': Will return '(unrepresentable)' instead of None if there is no string representation """ param = self.param if isinstance(param, strings): result = param if self.quote_str is None else self.quote_str.format(param) else: candidate = str(param) try: resurrected = literal_eval(candidate) # Note: This will miss string-representable custom objects. # However, the only alternative I can currently think of is using eval() which is inherently dangerous. except (SyntaxError, ValueError): result = None else: result = candidate if resurrected == param else None if result: result = ':' if self.param_repr_format is None else self.param_repr_format.format(result) return result class DictRelationship(ChildRelationship): param_repr_format = "[{}]" quote_str = "'{}'" def get_param_from_obj(self, obj): return obj.get(self.param) class SubscriptableIterableRelationship(DictRelationship): # for our purposes, we can see lists etc. as special cases of dicts def get_param_from_obj(self, obj): return obj[self.param] class InaccessibleRelationship(ChildRelationship): pass # there is no random access to set elements class SetRelationship(InaccessibleRelationship): pass class NonSubscriptableIterableRelationship(InaccessibleRelationship): param_repr_format = "[{}]" def get_param_repr(self, force=None): if force == 'yes': result = "(unrepresentable)" elif force == 'fake' and self.param: result = self.stringify_param() else: result = None return result class AttributeRelationship(ChildRelationship): param_repr_format = ".{}" def get_param_from_obj(self, obj): return getattr(obj, self.param) deepdiff-3.3.0/deepdiff/search.py000066400000000000000000000260751312554330700166650ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # In order to run the docstrings: # python3 -m deepdiff.search from __future__ import absolute_import from __future__ import print_function import sys from collections import Iterable from collections import MutableMapping import logging from deepdiff.helper import py3, strings, numbers, items logger = logging.getLogger(__name__) class DeepSearch(dict): r""" **DeepSearch** Deep Search inside objects to find the item matching your criteria. **Parameters** obj : The object to search within item : The item to search for verbose_level : int >= 0, default = 1. Verbose level one shows the paths of found items. Verbose level 2 shows the path and value of the found items. exclude_paths: list, default = None. List of paths to exclude from the report. exclude_types: list, default = None. List of object types to exclude from the report. **Returns** A DeepSearch object that has the matched paths and matched values. **Supported data types** int, string, unicode, dictionary, list, tuple, set, frozenset, OrderedDict, NamedTuple and custom objects! **Examples** Importing >>> from deepdiff import DeepSearch >>> from pprint import pprint Search in list for string >>> obj = ["long somewhere", "string", 0, "somewhere great!"] >>> item = "somewhere" >>> ds = DeepSearch(obj, item, verbose_level=2) >>> print(ds) {'matched_values': {'root[3]': 'somewhere great!', 'root[0]': 'long somewhere'}} Search in nested data for string >>> obj = ["something somewhere", {"long": "somewhere", "string": 2, 0: 0, "somewhere": "around"}] >>> item = "somewhere" >>> ds = DeepSearch(obj, item, verbose_level=2) >>> pprint(ds, indent=2) { 'matched_paths': {"root[1]['somewhere']": 'around'}, 'matched_values': { 'root[0]': 'something somewhere', "root[1]['long']": 'somewhere'}} """ warning_num = 0 def __init__(self, obj, item, exclude_paths=set(), exclude_types=set(), verbose_level=1, case_sensitive=False, **kwargs): if kwargs: raise ValueError(( "The following parameter(s) are not valid: %s\n" "The valid parameters are obj, item, exclude_paths, exclude_types and verbose_level." ) % ', '.join(kwargs.keys())) self.obj = obj self.case_sensitive = case_sensitive if isinstance(item, strings) else True item = item if self.case_sensitive else item.lower() self.exclude_paths = set(exclude_paths) self.exclude_types = set(exclude_types) self.exclude_types_tuple = tuple( exclude_types) # we need tuple for checking isinstance self.verbose_level = verbose_level self.update( matched_paths=self.__set_or_dict(), matched_values=self.__set_or_dict(), unprocessed=[]) self.__search(obj, item, parents_ids=frozenset({id(obj)})) empty_keys = [k for k, v in getattr(self, items)() if not v] for k in empty_keys: del self[k] def __set_or_dict(self): return {} if self.verbose_level >= 2 else set() def __report(self, report_key, key, value): if self.verbose_level >= 2: self[report_key][key] = value else: self[report_key].add(key) @staticmethod def __add_to_frozen_set(parents_ids, item_id): parents_ids = set(parents_ids) parents_ids.add(item_id) return frozenset(parents_ids) def __search_obj(self, obj, item, parent, parents_ids=frozenset({}), is_namedtuple=False): """Search objects""" found = False if obj == item: found = True # We report the match but also continue inside the match to see if there are # furthur matches inside the `looped` object. self.__report(report_key='matched_values', key=parent, value=obj) try: if is_namedtuple: obj = obj._asdict() else: # Skip magic methods. Slightly hacky, but unless people are defining # new magic methods they want to search, it should work fine. obj = {i: getattr(obj, i) for i in dir(obj) if not (i.startswith('__') and i.endswith('__'))} except AttributeError: try: obj = {i: getattr(obj, i) for i in obj.__slots__} except AttributeError: if not found: self['unprocessed'].append("%s" % parent) return self.__search_dict( obj, item, parent, parents_ids, print_as_attribute=True) def __skip_this(self, item, parent): skip = False if parent in self.exclude_paths: skip = True else: if isinstance(item, self.exclude_types_tuple): skip = True return skip def __search_dict(self, obj, item, parent, parents_ids=frozenset({}), print_as_attribute=False): """Search dictionaries""" if print_as_attribute: parent_text = "%s.%s" else: parent_text = "%s[%s]" obj_keys = set(obj.keys()) for item_key in obj_keys: if not print_as_attribute and isinstance(item_key, strings): item_key_str = "'%s'" % item_key else: item_key_str = item_key obj_child = obj[item_key] item_id = id(obj_child) if parents_ids and item_id in parents_ids: continue parents_ids_added = self.__add_to_frozen_set(parents_ids, item_id) new_parent = parent_text % (parent, item_key_str) new_parent_cased = new_parent if self.case_sensitive else new_parent.lower() if str(item) in new_parent_cased: self.__report( report_key='matched_paths', key=new_parent, value=obj_child) self.__search( obj_child, item, parent=new_parent, parents_ids=parents_ids_added) def __search_iterable(self, obj, item, parent="root", parents_ids=frozenset({})): """Search iterables except dictionaries, sets and strings.""" for i, thing in enumerate(obj): new_parent = "%s[%s]" % (parent, i) if self.__skip_this(thing, parent=new_parent): continue if self.case_sensitive or not isinstance(thing, strings): thing_cased = thing else: thing_cased = thing.lower() if thing_cased == item: self.__report( report_key='matched_values', key=new_parent, value=thing) else: item_id = id(thing) if parents_ids and item_id in parents_ids: continue parents_ids_added = self.__add_to_frozen_set(parents_ids, item_id) self.__search(thing, item, "%s[%s]" % (parent, i), parents_ids_added) def __search_str(self, obj, item, parent): """Compare strings""" obj_text = obj if self.case_sensitive else obj.lower() if item in obj_text: self.__report(report_key='matched_values', key=parent, value=obj) def __search_numbers(self, obj, item, parent): if item == obj: self.__report(report_key='matched_values', key=parent, value=obj) def __search_tuple(self, obj, item, parent, parents_ids): # Checking to see if it has _fields. Which probably means it is a named # tuple. try: obj._asdict # It must be a normal tuple except AttributeError: self.__search_iterable(obj, item, parent, parents_ids) # We assume it is a namedtuple then else: self.__search_obj( obj, item, parent, parents_ids, is_namedtuple=True) def __search(self, obj, item, parent="root", parents_ids=frozenset({})): """The main search method""" if self.__skip_this(item, parent): return elif isinstance(obj, strings) and isinstance(item, strings): self.__search_str(obj, item, parent) elif isinstance(obj, strings) and isinstance(item, numbers): return elif isinstance(obj, numbers): self.__search_numbers(obj, item, parent) elif isinstance(obj, MutableMapping): self.__search_dict(obj, item, parent, parents_ids) elif isinstance(obj, tuple): self.__search_tuple(obj, item, parent, parents_ids) elif isinstance(obj, (set, frozenset)): if self.warning_num < 10: logger.warning( "Set item detected in the path." "'set' objects do NOT support indexing. But DeepSearch will still report a path." ) self.warning_num += 1 self.__search_iterable(obj, item, parent, parents_ids) elif isinstance(obj, Iterable): self.__search_iterable(obj, item, parent, parents_ids) else: self.__search_obj(obj, item, parent, parents_ids) class grep(object): """ **Grep!** grep is a new interface for Deep Search. It takes exactly the same arguments. And it works just like grep in shell! **Examples** Importing >>> from deepdiff import grep >>> from pprint import pprint Search in list for string >>> obj = ["long somewhere", "string", 0, "somewhere great!"] >>> item = "somewhere" >>> ds = obj | grep(item) >>> print(ds) {'matched_values': {'root[3]', 'root[0]'} Search in nested data for string >>> obj = ["something somewhere", {"long": "somewhere", "string": 2, 0: 0, "somewhere": "around"}] >>> item = "somewhere" >>> ds = obj | grep(item, verbose_level=2) >>> pprint(ds, indent=2) { 'matched_paths': {"root[1]['somewhere']": 'around'}, 'matched_values': { 'root[0]': 'something somewhere', "root[1]['long']": 'somewhere'}} """ def __init__(self, item, **kwargs): self.item = item self.kwargs = kwargs def __ror__(self, other): return DeepSearch(obj=other, item=self.item, **self.kwargs) if __name__ == "__main__": # pragma: no cover if not py3: sys.exit("Please run with Python 3 to verify the doc strings.") import doctest doctest.testmod() deepdiff-3.3.0/docs/000077500000000000000000000000001312554330700142165ustar00rootroot00000000000000deepdiff-3.3.0/docs/Makefile000066400000000000000000000163711312554330700156660ustar00rootroot00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # User-friendly check for sphinx-build ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) endif # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest coverage gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " applehelp to make an Apple Help Book" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " xml to make Docutils-native XML files" @echo " pseudoxml to make pseudoxml-XML files for display purposes" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" @echo " coverage to run coverage check of the documentation (if enabled)" clean: rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/DeepDiff.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/DeepDiff.qhc" applehelp: $(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp @echo @echo "Build finished. The help book is in $(BUILDDIR)/applehelp." @echo "N.B. You won't be able to view it unless you put it in" \ "~/Library/Documentation/Help or install it in your application" \ "bundle." devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/DeepDiff" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/DeepDiff" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." latexpdfja: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through platex and dvipdfmx..." $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." coverage: $(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage @echo "Testing of coverage in the sources finished, look at the " \ "results in $(BUILDDIR)/coverage/python.txt." xml: $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml @echo @echo "Build finished. The XML files are in $(BUILDDIR)/xml." pseudoxml: $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml @echo @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." deepdiff-3.3.0/docs/conf.py000066400000000000000000000220741312554330700155220ustar00rootroot00000000000000#!/usr/bin/env python3 # -*- coding: utf-8 -*- # # DeepDiff documentation build configuration file, created by # sphinx-quickstart on Mon Jul 20 06:06:44 2015. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys import os import shlex # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # import pdb; pdb.set_trace() sys.path.insert(0, os.path.abspath('..')) # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.doctest', ] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: # source_suffix = ['.rst', '.md'] source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = 'DeepDiff' copyright = '2015-2017, Sep Dehpour' author = 'Sep Dehpour' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = '3.3.0' # The full version, including alpha/beta/rc tags. release = '3.3.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build'] # The reST default role (used for this markup: `text`) to use for all # documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built documents. #keep_warnings = False # If true, `todo` and `todoList` produce output, else they produce nothing. todo_include_todos = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'alabaster' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied # directly to the root of the documentation. #html_extra_path = [] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Language to be used for generating the HTML full-text search index. # Sphinx supports the following languages: # 'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja' # 'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr' #html_search_language = 'en' # A dictionary with options for the search language support, empty by default. # Now only 'ja' uses this config value #html_search_options = {'type': 'default'} # The name of a javascript file (relative to the configuration directory) that # implements a search results scorer. If empty, the default will be used. #html_search_scorer = 'scorer.js' # Output file base name for HTML help builder. htmlhelp_basename = 'DeepDiffdoc' # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', # Latex figure (float) alignment #'figure_align': 'htbp', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'DeepDiff.tex', 'DeepDiff Documentation', 'Sep Dehpour', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'deepdiff', 'DeepDiff Documentation', [author], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ (master_doc, 'DeepDiff', 'DeepDiff Documentation', author, 'DeepDiff', 'Deep difference of Python objects.', 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' # If true, do not generate a @detailmenu in the "Top" node's menu. #texinfo_no_detailmenu = False deepdiff-3.3.0/docs/diff.rst000066400000000000000000000002601312554330700156560ustar00rootroot00000000000000:doc:`/index` DeepDiff Reference ================== .. toctree:: :maxdepth: 3 .. automodule:: deepdiff.diff .. autoclass:: DeepDiff :members: Back to :doc:`/index` deepdiff-3.3.0/docs/dsearch.rst000066400000000000000000000002701312554330700163600ustar00rootroot00000000000000:doc:`/index` DeepSearch Reference ==================== .. toctree:: :maxdepth: 3 .. automodule:: deepdiff.search .. autoclass:: DeepSearch :members: Back to :doc:`/index` deepdiff-3.3.0/docs/index.rst000066400000000000000000000310421312554330700160570ustar00rootroot00000000000000.. DeepDiff documentation master file, created by sphinx-quickstart on Mon Jul 20 06:06:44 2015. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. DeepDiff 3.3.0 documentation! ============================= **DeepDiff: Deep Difference of dictionaries, iterables and almost any other object recursively.** DeepDiff works with Python 2.7, 3.3, 3.4, 3.5, 3.6, Pypy, Pypy3 ************ Installation ************ Install from PyPi:: pip install deepdiff Importing ~~~~~~~~~ .. code:: python >>> from deepdiff import DeepDiff # For Deep Difference of 2 objects >>> from deepdiff import DeepSearch # For finding if item exists in an object ******** Features ******** Parameters ~~~~~~~~~~ - t1 (the first object) - t2 (the second object) - `ignore\_order`_ - `report\_repetition`_ - `exclude\_types\_or\_paths`_ - `significant\_digits`_ - `views`_ Supported data types ~~~~~~~~~~~~~~~~~~~~ int, string, dictionary, list, tuple, set, frozenset, OrderedDict, NamedTuple and custom objects! Ignore Order ~~~~~~~~~~~~ Sometimes you don’t care about the order of objects when comparing them. In those cases, you can set ``ignore_order=True``. However this flag won’t report the repetitions to you. You need to additionally enable ``report_report_repetition=True`` for getting a report of repetitions. List difference ignoring order or duplicates -------------------------------------------- .. code:: python >>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} >>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 3, 2, 3]}} >>> ddiff = DeepDiff(t1, t2, ignore_order=True) >>> print (ddiff) {} Report repetitions ~~~~~~~~~~~~~~~~~~ This flag ONLY works when ignoring order is enabled. .. code:: python t1 = [1, 3, 1, 4] t2 = [4, 4, 1] ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True) print(ddiff) which will print you: .. code:: python {'iterable_item_removed': {'root[1]': 3}, 'repetition_change': {'root[0]': {'old_repeat': 2, 'old_indexes': [0, 2], 'new_indexes': [2], 'value': 1, 'new_repeat': 1}, 'root[3]': {'old_repeat': 1, 'old_indexes': [3], 'new_indexes': [0, 1], 'value': 4, 'new_repeat': 2}}} Exclude types or paths ~~~~~~~~~~~~~~~~~~~~~~ Exclude certain types from comparison ------------------------------------- .. code:: python >>> l1 = logging.getLogger("test") >>> l2 = logging.getLogger("test2") >>> t1 = {"log": l1, 2: 1337} >>> t2 = {"log": l2, 2: 1337} >>> print(DeepDiff(t1, t2, exclude_types={logging.Logger})) {} Exclude part of your object tree from comparison ------------------------------------------------ .. code:: python >>> t1 = {"for life": "vegan", "ingredients": ["no meat", "no eggs", "no dairy"]} >>> t2 = {"for life": "vegan", "ingredients": ["veggies", "tofu", "soy sauce"]} >>> print (DeepDiff(t1, t2, exclude_paths={"root['ingredients']"})) {} Significant Digits ~~~~~~~~~~~~~~~~~~ Digits **after** the decimal point. Internally it uses “{:.Xf}”.format(Your Number) to compare numbers where X=significant\_digits .. code:: python >>> t1 = Decimal('1.52') >>> t2 = Decimal('1.57') >>> DeepDiff(t1, t2, significant_digits=0) {} >>> DeepDiff(t1, t2, significant_digits=1) {'values_changed': {'root': {'old_value': Decimal('1.52'), 'new_value': Decimal('1.57')}}} Approximate float comparison: ----------------------------- .. code:: python >>> t1 = [ 1.1129, 1.3359 ] >>> t2 = [ 1.113, 1.3362 ] >>> pprint(DeepDiff(t1, t2, significant_digits=3)) {} >>> pprint(DeepDiff(t1, t2)) {'values_changed': {'root[0]': {'new_value': 1.113, 'old_value': 1.1129}, 'root[1]': {'new_value': 1.3362, 'old_value': 1.3359}}} >>> pprint(DeepDiff(1.23*10**20, 1.24*10**20, significant_digits=1)) {'values_changed': {'root': {'new_value': 1.24e+20, 'old_value': 1.23e+20}}} Views ~~~~~ Text View (default) ------------------- Text view is the original and currently the default view of DeepDiff. It is called text view because the results contain texts that represent the path to the data: Example of using the text view. >>> from deepdiff import DeepDiff >>> t1 = {1:1, 3:3, 4:4} >>> t2 = {1:1, 3:3, 5:5, 6:6} >>> ddiff = DeepDiff(t1, t2) >>> print(ddiff) {'dictionary_item_added': {'root[5]', 'root[6]'}, 'dictionary_item_removed': {'root[4]'}} So for example ddiff['dictionary_item_removed'] is a set if strings thus this is called the text view. .. seealso:: The following examples are using the *default text view.* The Tree View is introduced in DeepDiff v3 and provides traversing capabilities through your diffed data and more! Read more about the Tree View at :doc:`/diff` Tree View (new) --------------- Starting the version v3 You can choose the view into the deepdiff results. The tree view provides you with tree objects that you can traverse through to find the parents of the objects that are diffed and the actual objects that are being diffed. This view is very useful when dealing with nested objects. Note that tree view always returns results in the form of Python sets. You can traverse through the tree elements! .. note:: The Tree view is just a different representation of the diffed data. Behind the scene, DeepDiff creates the tree view first and then converts it to textual representation for the text view. .. code:: text +---------------------------------------------------------------+ | | | parent(t1) parent node parent(t2) | | + ^ + | +------|--------------------------|---------------------|-------+ | | | up | | Child | | | ChildRelationship | Relationship | | | | down | | | +------|----------------------|-------------------------|-------+ | v v v | | child(t1) child node child(t2) | | | +---------------------------------------------------------------+ The tree view allows you to have more than mere textual representaion of the diffed objects. It gives you the actual objects (t1, t2) throughout the tree of parents and children. :Example: .. code:: python >>> t1 = {1:1, 2:2, 3:3} >>> t2 = {1:1, 2:4, 3:3} >>> ddiff_verbose0 = DeepDiff(t1, t2, verbose_level=0, view='tree') >>> ddiff_verbose0 {'values_changed': {}} >>> >>> ddiff_verbose1 = DeepDiff(t1, t2, verbose_level=1, view='tree') >>> ddiff_verbose1 {'values_changed': {}} >>> set_of_values_changed = ddiff_verbose1['values_changed'] >>> # since set_of_values_changed includes only one item in a set >>> # in order to get that one item we can: >>> (changed,) = set_of_values_changed >>> changed # Another way to get this is to do: changed=list(set_of_values_changed)[0] >>> changed.t1 2 >>> changed.t2 4 >>> # You can traverse through the tree, get to the parents! >>> changed.up .. seealso:: Read more about the Tree View at :doc:`/diff` Verbose Level ~~~~~~~~~~~~~ Verbose level by default is 1. The possible values are 0, 1 and 2. - verbose_level 0: won’t report values when type changed. - verbose_level 1: default - verbose_level 2: will report values when custom objects or dictionaries have items added or removed. .. seealso:: Read more about the verbosity at :doc:`/diff` Serialization ~~~~~~~~~~~~~ DeepDiff uses jsonpickle in order to serialize and deserialize its results into json. This works for both tree view and text view. :Serialize and then deserialize back to deepdiff: .. code:: python >>> t1 = {1: 1, 2: 2, 3: 3} >>> t2 = {1: 1, 2: "2", 3: 3} >>> ddiff = DeepDiff(t1, t2) >>> jsoned = ddiff.json >>> jsoned '{"type_changes": {"root[2]": {"py/object": "deepdiff.helper.RemapDict", "new_type": {"py/type": "__builtin__.str"}, "new_value": "2", "old_type": {"py/type": "__builtin__.int"}, "old_value": 2}}}' >>> ddiff_new = DeepDiff.from_json(jsoned) >>> ddiff == ddiff_new True *********** Deep Search *********** Deep Search inside objects to find the item matching your criteria. Note that is searches for either the path to match your criteria or the word in an item. :Examples: Importing .. code:: python >>> from deepdiff import DeepSearch >>> from pprint import pprint Search in list for string .. code:: python >>> obj = ["long somewhere", "string", 0, "somewhere great!"] >>> item = "somewhere" >>> ds = DeepSearch(obj, item, verbose_level=2) >>> print(ds) {'matched_values': {'root[3]': 'somewhere great!', 'root[0]': 'long somewhere'}} Search in nested data for string .. code:: python >>> obj = ["something somewhere", {"long": "somewhere", "string": 2, 0: 0, "somewhere": "around"}] >>> item = "somewhere" >>> ds = DeepSearch(obj, item, verbose_level=2) >>> pprint(ds, indent=2) { 'matched_paths': {"root[1]['somewhere']": 'around'}, 'matched_values': { 'root[0]': 'something somewhere', "root[1]['long']": 'somewhere'}} .. _ignore\_order: #ignore-order .. _report\_repetition: #report-repetitions .. _verbose\_level: #verbose-level .. _exclude\_types\_or\_paths: #exclude-types-or-paths .. _significant\_digits: #significant-digits .. _views: #views DeepDiff Reference ================== :doc:`/diff` DeepSearch Reference ==================== :doc:`/dsearch` Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` Changelog ========= - v3-3-0: Searching for objects and class attributes - v3-2-2: Adding help(deepdiff) - v3-2-1: Fixing hash of None - v3-2-0: Adding grep for search: object | grep(item) - v3-1-3: Unicode vs. Bytes default fix - v3-1-2: NotPresent Fix when item is added or removed. - v3-1-1: Bug fix when item value is None (#58) - v3-1-0: Serialization to/from json - v3-0-0: Introducing Tree View - v2-5-3: Bug fix on logging for content hash. - v2-5-2: Bug fixes on content hash. - v2-5-0: Adding ContentHash module to fix ignore_order once and for all. - v2-1-0: Adding Deep Search. Now you can search for item in an object. - v2-0-0: Exclusion patterns better coverage. Updating docs. - v1-8-0: Exclusion patterns. - v1-7-0: Deep Set comparison. - v1-6-0: Unifying key names. i.e newvalue is new_value now. For backward compatibility, newvalue still works. - v1-5-0: Fixing ignore order containers with unordered items. Adding significant digits when comparing decimals. Changes property is deprecated. - v1-1-0: Changing Set, Dictionary and Object Attribute Add/Removal to be reported as Set instead of List. Adding Pypy compatibility. - v1-0-2: Checking for ImmutableMapping type instead of dict - v1-0-1: Better ignore order support - v1-0-0: Restructuring output to make it more useful. This is NOT backward compatible. - v0-6-1: Fixiing iterables with unhashable when order is ignored - v0-6-0: Adding unicode support - v0-5-9: Adding decimal support - v0-5-8: Adding ignore order for unhashables support - v0-5-7: Adding ignore order support - v0-5-6: Adding slots support - v0-5-5: Adding loop detection Authors ======= Sep Dehpour - `Github `_ - `ZepWorks `_ - `Linkedin `_ - `Article about Deepdiff `_ Victor Hahn Castell - `hahncastell.de `_ - `flexoptix.net `_ ALso thanks to: - nfvs for Travis-CI setup script - brbsix for initial Py3 porting - WangFenjin for unicode support - timoilya for comparing list of sets when ignoring order - Bernhard10 for significant digits comparison - b-jazz for PEP257 cleanup, Standardize on full names, fixing line endings. - Victor Hahn Castell @ Flexoptix for deep set comparison deepdiff-3.3.0/docs/make.bat000066400000000000000000000161201312554330700156230ustar00rootroot00000000000000@ECHO OFF REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set BUILDDIR=_build set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . set I18NSPHINXOPTS=%SPHINXOPTS% . if NOT "%PAPER%" == "" ( set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% ) if "%1" == "" goto help if "%1" == "help" ( :help echo.Please use `make ^` where ^ is one of echo. html to make standalone HTML files echo. dirhtml to make HTML files named index.html in directories echo. singlehtml to make a single large HTML file echo. pickle to make pickle files echo. json to make JSON files echo. htmlhelp to make HTML files and a HTML help project echo. qthelp to make HTML files and a qthelp project echo. devhelp to make HTML files and a Devhelp project echo. epub to make an epub echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter echo. text to make text files echo. man to make manual pages echo. texinfo to make Texinfo files echo. gettext to make PO message catalogs echo. changes to make an overview over all changed/added/deprecated items echo. xml to make Docutils-native XML files echo. pseudoxml to make pseudoxml-XML files for display purposes echo. linkcheck to check all external links for integrity echo. doctest to run all doctests embedded in the documentation if enabled echo. coverage to run coverage check of the documentation if enabled goto end ) if "%1" == "clean" ( for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i del /q /s %BUILDDIR%\* goto end ) REM Check if sphinx-build is available and fallback to Python version if any %SPHINXBUILD% 2> nul if errorlevel 9009 goto sphinx_python goto sphinx_ok :sphinx_python set SPHINXBUILD=python -m sphinx.__init__ %SPHINXBUILD% 2> nul if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx echo.installed, then set the SPHINXBUILD environment variable to point echo.to the full path of the 'sphinx-build' executable. Alternatively you echo.may add the Sphinx directory to PATH. echo. echo.If you don't have Sphinx installed, grab it from echo.http://sphinx-doc.org/ exit /b 1 ) :sphinx_ok if "%1" == "html" ( %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/html. goto end ) if "%1" == "dirhtml" ( %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. goto end ) if "%1" == "singlehtml" ( %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. goto end ) if "%1" == "pickle" ( %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the pickle files. goto end ) if "%1" == "json" ( %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the JSON files. goto end ) if "%1" == "htmlhelp" ( %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run HTML Help Workshop with the ^ .hhp project file in %BUILDDIR%/htmlhelp. goto end ) if "%1" == "qthelp" ( %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run "qcollectiongenerator" with the ^ .qhcp project file in %BUILDDIR%/qthelp, like this: echo.^> qcollectiongenerator %BUILDDIR%\qthelp\DeepDiff.qhcp echo.To view the help file: echo.^> assistant -collectionFile %BUILDDIR%\qthelp\DeepDiff.ghc goto end ) if "%1" == "devhelp" ( %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp if errorlevel 1 exit /b 1 echo. echo.Build finished. goto end ) if "%1" == "epub" ( %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub if errorlevel 1 exit /b 1 echo. echo.Build finished. The epub file is in %BUILDDIR%/epub. goto end ) if "%1" == "latex" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex if errorlevel 1 exit /b 1 echo. echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdf" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf cd %~dp0 echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdfja" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf-ja cd %~dp0 echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "text" ( %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text if errorlevel 1 exit /b 1 echo. echo.Build finished. The text files are in %BUILDDIR%/text. goto end ) if "%1" == "man" ( %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man if errorlevel 1 exit /b 1 echo. echo.Build finished. The manual pages are in %BUILDDIR%/man. goto end ) if "%1" == "texinfo" ( %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo if errorlevel 1 exit /b 1 echo. echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. goto end ) if "%1" == "gettext" ( %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale if errorlevel 1 exit /b 1 echo. echo.Build finished. The message catalogs are in %BUILDDIR%/locale. goto end ) if "%1" == "changes" ( %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes if errorlevel 1 exit /b 1 echo. echo.The overview file is in %BUILDDIR%/changes. goto end ) if "%1" == "linkcheck" ( %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck if errorlevel 1 exit /b 1 echo. echo.Link check complete; look for any errors in the above output ^ or in %BUILDDIR%/linkcheck/output.txt. goto end ) if "%1" == "doctest" ( %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest if errorlevel 1 exit /b 1 echo. echo.Testing of doctests in the sources finished, look at the ^ results in %BUILDDIR%/doctest/output.txt. goto end ) if "%1" == "coverage" ( %SPHINXBUILD% -b coverage %ALLSPHINXOPTS% %BUILDDIR%/coverage if errorlevel 1 exit /b 1 echo. echo.Testing of coverage in the sources finished, look at the ^ results in %BUILDDIR%/coverage/python.txt. goto end ) if "%1" == "xml" ( %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml if errorlevel 1 exit /b 1 echo. echo.Build finished. The XML files are in %BUILDDIR%/xml. goto end ) if "%1" == "pseudoxml" ( %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml if errorlevel 1 exit /b 1 echo. echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. goto end ) :end deepdiff-3.3.0/readthedocs-requirements.txt000066400000000000000000000000151312554330700210310ustar00rootroot00000000000000numpydoc==0.4deepdiff-3.3.0/run_tests.py000077500000000000000000000002771312554330700156770ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- import unittest loader = unittest.TestLoader() tests = loader.discover('.') testRunner = unittest.runner.TextTestRunner() testRunner.run(tests) deepdiff-3.3.0/setup.cfg000066400000000000000000000002111312554330700151010ustar00rootroot00000000000000[flake8] max-line-length = 120 builtins = json statistics = true ignore = E202 exclude = ./data,./src,.svn,CVS,.bzr,.hg,.git,__pycache__ deepdiff-3.3.0/setup.py000077500000000000000000000031231312554330700150020ustar00rootroot00000000000000import os from setuptools import setup # if you are not using vagrant, just delete os.link directly, # The hard link only saves a little disk space, so you should not care if os.environ.get('USER', '') == 'vagrant': del os.link try: with open('README.txt') as file: long_description = file.read() except: long_description = "Deep Difference and Search of any Python object/data." setup(name='deepdiff', version='3.3.0', description='Deep Difference and Search of any Python object/data.', url='https://github.com/seperman/deepdiff', download_url='https://github.com/seperman/deepdiff/tarball/master', author='Seperman', author_email='sep@zepworks.com', license='MIT', packages=['deepdiff'], zip_safe=False, test_suite="tests", tests_require=['mock'], # 'numpy==1.11.2' numpy is needed but comes already installed with travis long_description=long_description, install_requires=[ 'jsonpickle' ], classifiers=[ "Intended Audience :: Developers", "Operating System :: OS Independent", "Topic :: Software Development", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: Implementation :: PyPy", "Development Status :: 5 - Production/Stable", "License :: OSI Approved :: MIT License" ], ) deepdiff-3.3.0/tests/000077500000000000000000000000001312554330700144305ustar00rootroot00000000000000deepdiff-3.3.0/tests/__init__.py000066400000000000000000000006621312554330700165450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # To run all the tests: # python -m unittest discover class CustomClass(object): def __init__(self, a, b=None): self.a = a self.b = b def __str__(self): return "Custom({}, {})".format(self.a, self.b) def __repr__(self): return self.__str__() class CustomClassMisleadingRepr(CustomClass): def __str__(self): return "({}, {})".format(self.a, self.b) deepdiff-3.3.0/tests/test_diff_text.py000066400000000000000000001263141312554330700200240ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- """ To run only the search tests: python -m unittest tests.test_diff_text Or to run all the tests: python -m unittest discover Or to run all the tests with coverage: coverage run --source deepdiff setup.py test Or using Nose: nosetests --with-coverage --cover-package=deepdiff To run a specific test, run this from the root of repo: python -m unittest tests.test_diff_text.DeepDiffTextTestCase.test_same_objects or using nosetests: nosetests tests/test_diff_text.py:DeepDiffTestCase.test_diff_when_hash_fails """ import unittest import datetime from decimal import Decimal from deepdiff import DeepDiff from deepdiff.helper import py3 from tests import CustomClass if py3: from unittest import mock else: import mock import logging logging.disable(logging.CRITICAL) class DeepDiffTextTestCase(unittest.TestCase): """DeepDiff Tests.""" def test_same_objects(self): t1 = {1: 1, 2: 2, 3: 3} t2 = t1 self.assertEqual(DeepDiff(t1, t2), {}) def test_item_type_change(self): t1 = {1: 1, 2: 2, 3: 3} t2 = {1: 1, 2: "2", 3: 3} ddiff = DeepDiff(t1, t2) self.assertEqual(ddiff, { 'type_changes': { "root[2]": { "old_value": 2, "old_type": int, "new_value": "2", "new_type": str } } }) def test_item_type_change_less_verbose(self): t1 = {1: 1, 2: 2, 3: 3} t2 = {1: 1, 2: "2", 3: 3} self.assertEqual( DeepDiff( t1, t2, verbose_level=0), {'type_changes': { "root[2]": { "old_type": int, "new_type": str } }}) def test_value_change(self): t1 = {1: 1, 2: 2, 3: 3} t2 = {1: 1, 2: 4, 3: 3} result = { 'values_changed': { 'root[2]': { "old_value": 2, "new_value": 4 } } } self.assertEqual(DeepDiff(t1, t2), result) def test_item_added_and_removed(self): t1 = {1: 1, 2: 2, 3: 3, 4: 4} t2 = {1: 1, 2: 4, 3: 3, 5: 5, 6: 6} ddiff = DeepDiff(t1, t2) result = { 'dictionary_item_added': {'root[5]', 'root[6]'}, 'dictionary_item_removed': {'root[4]'}, 'values_changed': { 'root[2]': { "old_value": 2, "new_value": 4 } } } self.assertEqual(ddiff, result) def test_item_added_and_removed_verbose(self): t1 = {1: 1, 3: 3, 4: 4} t2 = {1: 1, 3: 3, 5: 5, 6: 6} ddiff = DeepDiff(t1, t2, verbose_level=2) result = { 'dictionary_item_removed': { 'root[4]': 4 }, 'dictionary_item_added': { 'root[6]': 6, 'root[5]': 5 } } self.assertEqual(ddiff, result) def test_diffs_dates(self): t1 = datetime.date(2016, 8, 8) t2 = datetime.date(2016, 8, 7) ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root': { 'new_value': t2, 'old_value': t1 } } } self.assertEqual(ddiff, result) def test_string_difference(self): t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": "world"}} t2 = {1: 1, 2: 4, 3: 3, 4: {"a": "hello", "b": "world!"}} ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root[2]': { 'old_value': 2, 'new_value': 4 }, "root[4]['b']": { 'old_value': 'world', 'new_value': 'world!' } } } self.assertEqual(ddiff, result) def test_diffs_equal_strings_when_not_identical(self): t1 = 'hello' t2 = 'hel' t2 += 'lo' assert t1 is not t2 ddiff = DeepDiff(t1, t2) self.assertEqual(ddiff, {}) def test_string_difference2(self): t1 = { 1: 1, 2: 2, 3: 3, 4: { "a": "hello", "b": "world!\nGoodbye!\n1\n2\nEnd" } } t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": "world\n1\n2\nEnd"}} ddiff = DeepDiff(t1, t2) result = { 'values_changed': { "root[4]['b']": { 'diff': '--- \n+++ \n@@ -1,5 +1,4 @@\n-world!\n-Goodbye!\n+world\n 1\n 2\n End', 'new_value': 'world\n1\n2\nEnd', 'old_value': 'world!\nGoodbye!\n1\n2\nEnd' } } } self.assertEqual(ddiff, result) def test_bytes(self): t1 = { 1: 1, 2: 2, 3: 3, 4: { "a": b"hello", "b": b"world!\nGoodbye!\n1\n2\nEnd", "c": b"\x80", } } t2 = {1: 1, 2: 2, 3: 3, 4: { "a": b"hello", "b": b"world\n1\n2\nEnd", "c": b'\x81', } } ddiff = DeepDiff(t1, t2) result = { 'values_changed': { "root[4]['b']": { 'diff': '--- \n+++ \n@@ -1,5 +1,4 @@\n-world!\n-Goodbye!\n+world\n 1\n 2\n End', 'new_value': b'world\n1\n2\nEnd', 'old_value': b'world!\nGoodbye!\n1\n2\nEnd' }, "root[4]['c']": { 'new_value': b'\x81', 'old_value': b'\x80' } } } self.assertEqual(ddiff, result) def test_unicode(self): t1 = { 1: 1, 2: 2, 3: 3, 4: { "a": u"hello", "b": u"world!\nGoodbye!\n1\n2\nEnd" } } t2 = {1: 1, 2: 2, 3: 3, 4: {"a": u"hello", "b": u"world\n1\n2\nEnd"}} ddiff = DeepDiff(t1, t2) result = { 'values_changed': { "root[4]['b']": { 'diff': '--- \n+++ \n@@ -1,5 +1,4 @@\n-world!\n-Goodbye!\n+world\n 1\n 2\n End', 'new_value': u'world\n1\n2\nEnd', 'old_value': u'world!\nGoodbye!\n1\n2\nEnd' } } } self.assertEqual(ddiff, result) def test_type_change(self): t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, 3]}} t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": "world\n\n\nEnd"}} ddiff = DeepDiff(t1, t2) result = { 'type_changes': { "root[4]['b']": { 'old_type': list, 'new_value': 'world\n\n\nEnd', 'old_value': [1, 2, 3], 'new_type': str } } } self.assertEqual(ddiff, result) def test_list_difference(self): t1 = { 1: 1, 2: 2, 3: 3, 4: { "a": "hello", "b": [1, 2, 'to_be_removed', 'to_be_removed2'] } } t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2]}} ddiff = DeepDiff(t1, t2) result = { 'iterable_item_removed': { "root[4]['b'][2]": "to_be_removed", "root[4]['b'][3]": 'to_be_removed2' } } self.assertEqual(ddiff, result) def test_list_difference_add(self): t1 = [1, 2] t2 = [1, 2, 3, 5] ddiff = DeepDiff(t1, t2) result = {'iterable_item_added': {'root[2]': 3, 'root[3]': 5}} self.assertEqual(ddiff, result) def test_list_difference2(self): t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, 3, 10, 12]}} t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 3, 2]}} result = { 'values_changed': { "root[4]['b'][2]": { 'new_value': 2, 'old_value': 3 }, "root[4]['b'][1]": { 'new_value': 3, 'old_value': 2 } }, 'iterable_item_removed': { "root[4]['b'][3]": 10, "root[4]['b'][4]": 12 } } ddiff = DeepDiff(t1, t2) self.assertEqual(ddiff, result) def test_list_difference3(self): t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, 5]}} t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 3, 2, 5]}} ddiff = DeepDiff(t1, t2) result = { 'values_changed': { "root[4]['b'][2]": { 'new_value': 2, 'old_value': 5 }, "root[4]['b'][1]": { 'new_value': 3, 'old_value': 2 } }, 'iterable_item_added': { "root[4]['b'][3]": 5 } } self.assertEqual(ddiff, result) def test_list_difference4(self): # TODO: Look into Levenshtein algorithm # So that the result is just insertion of "c" in this test. t1 = ["a", "b", "d", "e"] t2 = ["a", "b", "c", "d", "e"] ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root[2]': { 'new_value': 'c', 'old_value': 'd' }, 'root[3]': { 'new_value': 'd', 'old_value': 'e' } }, 'iterable_item_added': { 'root[4]': 'e' } } self.assertEqual(ddiff, result) def test_list_difference_ignore_order(self): t1 = {1: 1, 4: {"a": "hello", "b": [1, 2, 3]}} t2 = {1: 1, 4: {"a": "hello", "b": [1, 3, 2, 3]}} ddiff = DeepDiff(t1, t2, ignore_order=True) self.assertEqual(ddiff, {}) def test_dictionary_difference_ignore_order(self): t1 = {"a": [[{"b": 2, "c": 4}, {"b": 2, "c": 3}]]} t2 = {"a": [[{"b": 2, "c": 3}, {"b": 2, "c": 4}]]} ddiff = DeepDiff(t1, t2, ignore_order=True) self.assertEqual(ddiff, {}) def test_nested_list_ignore_order(self): t1 = [1, 2, [3, 4]] t2 = [[4, 3, 3], 2, 1] ddiff = DeepDiff(t1, t2, ignore_order=True) self.assertEqual(ddiff, {}) def test_nested_list_difference_ignore_order(self): t1 = [1, 2, [3, 4]] t2 = [[4, 3], 2, 1] ddiff = DeepDiff(t1, t2, ignore_order=True) self.assertEqual(ddiff, {}) def test_nested_list_with_dictionarry_difference_ignore_order(self): t1 = [1, 2, [3, 4, {1: 2}]] t2 = [[4, 3, {1: 2}], 2, 1] ddiff = DeepDiff(t1, t2, ignore_order=True) result = {} self.assertEqual(ddiff, result) def test_list_difference_ignore_order_report_repetition(self): t1 = [1, 3, 1, 4] t2 = [4, 4, 1] ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True) result = { 'iterable_item_removed': { 'root[1]': 3 }, 'repetition_change': { 'root[0]': { 'old_repeat': 2, 'old_indexes': [0, 2], 'new_indexes': [2], 'value': 1, 'new_repeat': 1 }, 'root[3]': { 'old_repeat': 1, 'old_indexes': [3], 'new_indexes': [0, 1], 'value': 4, 'new_repeat': 2 } } } self.assertEqual(ddiff, result) # TODO: fix repeition report def test_nested_list_ignore_order_report_repetition_wrong_currently(self): t1 = [1, 2, [3, 4]] t2 = [[4, 3, 3], 2, 1] ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True) result = { 'repetition_change': { 'root[2][0]': { 'old_repeat': 1, 'new_indexes': [1, 2], 'old_indexes': [1], 'value': 3, 'new_repeat': 2 } } } self.assertNotEqual(ddiff, result) def test_list_of_unhashable_difference_ignore_order(self): t1 = [{"a": 2}, {"b": [3, 4, {1: 1}]}] t2 = [{"b": [3, 4, {1: 1}]}, {"a": 2}] ddiff = DeepDiff(t1, t2, ignore_order=True) self.assertEqual(ddiff, {}) def test_list_of_unhashable_difference_ignore_order2(self): t1 = [1, {"a": 2}, {"b": [3, 4, {1: 1}]}, "B"] t2 = [{"b": [3, 4, {1: 1}]}, {"a": 2}, {1: 1}] ddiff = DeepDiff(t1, t2, ignore_order=True) result = { 'iterable_item_added': { 'root[2]': { 1: 1 } }, 'iterable_item_removed': { 'root[3]': 'B', 'root[0]': 1 } } self.assertEqual(ddiff, result) def test_list_of_unhashable_difference_ignore_order3(self): t1 = [1, {"a": 2}, {"a": 2}, {"b": [3, 4, {1: 1}]}, "B"] t2 = [{"b": [3, 4, {1: 1}]}, {1: 1}] ddiff = DeepDiff(t1, t2, ignore_order=True) result = { 'iterable_item_added': { 'root[1]': { 1: 1 } }, 'iterable_item_removed': { 'root[4]': 'B', 'root[0]': 1, 'root[1]': { 'a': 2 } } } self.assertEqual(ddiff, result) def test_list_of_unhashable_difference_ignore_order_report_repetition( self): t1 = [1, {"a": 2}, {"a": 2}, {"b": [3, 4, {1: 1}]}, "B"] t2 = [{"b": [3, 4, {1: 1}]}, {1: 1}] ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True) result = { 'iterable_item_added': { 'root[1]': { 1: 1 } }, 'iterable_item_removed': { 'root[4]': 'B', 'root[0]': 1, 'root[1]': { 'a': 2 }, 'root[2]': { 'a': 2 } } } self.assertEqual(ddiff, result) def test_list_of_unhashable_difference_ignore_order4(self): t1 = [{"a": 2}, {"a": 2}] t2 = [{"a": 2}] ddiff = DeepDiff(t1, t2, ignore_order=True) result = {} self.assertEqual(ddiff, result) def test_list_of_unhashable_difference_ignore_order_report_repetition2( self): t1 = [{"a": 2}, {"a": 2}] t2 = [{"a": 2}] ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True) result = { 'repetition_change': { 'root[0]': { 'old_repeat': 2, 'new_indexes': [0], 'old_indexes': [0, 1], 'value': { 'a': 2 }, 'new_repeat': 1 } } } self.assertEqual(ddiff, result) def test_list_of_sets_difference_ignore_order(self): t1 = [{1}, {2}, {3}] t2 = [{4}, {1}, {2}, {3}] ddiff = DeepDiff(t1, t2, ignore_order=True) result = {'iterable_item_added': {'root[0]': {4}}} self.assertEqual(ddiff, result) def test_list_of_sets_difference_ignore_order_when_there_is_duplicate( self): t1 = [{1}, {2}, {3}] t2 = [{4}, {1}, {2}, {3}, {3}] ddiff = DeepDiff(t1, t2, ignore_order=True) result = {'iterable_item_added': {'root[0]': {4}}} self.assertEqual(ddiff, result) def test_list_of_sets_difference_ignore_order_when_there_is_duplicate_and_mix_of_hashable_unhashable( self): t1 = [1, 1, {2}, {3}] t2 = [{4}, 1, {2}, {3}, {3}, 1, 1] ddiff = DeepDiff(t1, t2, ignore_order=True) result = {'iterable_item_added': {'root[0]': {4}}} self.assertEqual(ddiff, result) def test_set_of_none(self): """ https://github.com/seperman/deepdiff/issues/64 """ ddiff = DeepDiff(set(), set([None])) self.assertEqual(ddiff, {'set_item_added': {'root[None]'}}) def test_list_that_contains_dictionary(self): t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, {1: 1, 2: 2}]}} t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, {1: 3}]}} ddiff = DeepDiff(t1, t2) result = { 'dictionary_item_removed': {"root[4]['b'][2][2]"}, 'values_changed': { "root[4]['b'][2][1]": { 'old_value': 1, 'new_value': 3 } } } self.assertEqual(ddiff, result) def test_dictionary_of_list_of_dictionary_ignore_order(self): t1 = { 'item': [{ 'title': 1, 'http://purl.org/rss/1.0/modules/content/:encoded': '1' }, { 'title': 2, 'http://purl.org/rss/1.0/modules/content/:encoded': '2' }] } t2 = { 'item': [{ 'http://purl.org/rss/1.0/modules/content/:encoded': '1', 'title': 1 }, { 'http://purl.org/rss/1.0/modules/content/:encoded': '2', 'title': 2 }] } ddiff = DeepDiff(t1, t2, ignore_order=True) self.assertEqual(ddiff, {}) def test_comprehensive_ignore_order(self): t1 = { 'key1': 'val1', 'key2': [ { 'key3': 'val3', 'key4': 'val4', }, { 'key5': 'val5', 'key6': 'val6', }, ], } t2 = { 'key1': 'val1', 'key2': [ { 'key5': 'val5', 'key6': 'val6', }, { 'key3': 'val3', 'key4': 'val4', }, ], } ddiff = DeepDiff(t1, t2, ignore_order=True) self.assertEqual(ddiff, {}) def test_ignore_order_when_objects_similar(self): """ The current design can't recognize that { 'key5': 'val5, 'key6': 'val6', } at index 1 has become { 'key5': 'CHANGE', 'key6': 'val6', } at index 0. Further thought needs to go into designing an algorithm that can identify the modified objects when ignoring order. The current algorithm computes the hash of the objects and since the hashes are different, it assumes an object is removed and another one is added. """ t1 = { 'key1': 'val1', 'key2': [ { 'key3': 'val3', 'key4': 'val4', }, { 'key5': 'val5', 'key6': 'val6', }, ], } t2 = { 'key1': 'val1', 'key2': [ { 'key5': 'CHANGE', 'key6': 'val6', }, { 'key3': 'val3', 'key4': 'val4', }, ], } ddiff = DeepDiff(t1, t2, ignore_order=True) self.assertEqual(ddiff, { 'iterable_item_removed': { "root['key2'][1]": { 'key5': 'val5', 'key6': 'val6' } }, 'iterable_item_added': { "root['key2'][0]": { 'key5': 'CHANGE', 'key6': 'val6' } } }) def test_set_ignore_order_report_repetition(self): """ If this test fails, it means that DeepDiff is not checking for set types before general iterables. So it forces creating the hashtable because of report_repetition=True. """ t1 = {2, 1, 8} t2 = {1, 2, 3, 5} ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True) result = { 'set_item_added': {'root[3]', 'root[5]'}, 'set_item_removed': {'root[8]'} } self.assertEqual(ddiff, result) def test_set(self): t1 = {1, 2, 8} t2 = {1, 2, 3, 5} ddiff = DeepDiff(t1, t2) result = { 'set_item_added': {'root[3]', 'root[5]'}, 'set_item_removed': {'root[8]'} } self.assertEqual(ddiff, result) def test_set_strings(self): t1 = {"veggies", "tofu"} t2 = {"veggies", "tofu", "seitan"} ddiff = DeepDiff(t1, t2) result = {'set_item_added': {"root['seitan']"}} self.assertEqual(ddiff, result) def test_frozenset(self): t1 = frozenset([1, 2, 'B']) t2 = frozenset([1, 2, 3, 5]) ddiff = DeepDiff(t1, t2) result = { 'set_item_added': {'root[3]', 'root[5]'}, 'set_item_removed': {"root['B']"} } self.assertEqual(ddiff, result) def test_tuple(self): t1 = (1, 2, 8) t2 = (1, 2, 3, 5) ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root[2]': { 'old_value': 8, 'new_value': 3 } }, 'iterable_item_added': { 'root[3]': 5 } } self.assertEqual(ddiff, result) def test_named_tuples(self): from collections import namedtuple Point = namedtuple('Point', ['x', 'y']) t1 = Point(x=11, y=22) t2 = Point(x=11, y=23) ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root.y': { 'old_value': 22, 'new_value': 23 } } } self.assertEqual(ddiff, result) def test_custom_objects_change(self): t1 = CustomClass(1) t2 = CustomClass(2) ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root.a': { 'old_value': 1, 'new_value': 2 } } } self.assertEqual(ddiff, result) def test_custom_objects_slot_change(self): class ClassA(object): __slots__ = ['x', 'y'] def __init__(self, x, y): self.x = x self.y = y t1 = ClassA(1, 1) t2 = ClassA(1, 2) ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root.y': { 'old_value': 1, 'new_value': 2 } } } self.assertEqual(ddiff, result) def test_custom_objects_with_single_protected_slot(self): class ClassA(object): __slots__ = '__a' def __init__(self): self.__a = 1 def __str__(self): return str(self.__a) t1 = ClassA() t2 = ClassA() diff = DeepDiff(t1, t2) self.assertEqual(diff, {}) def get_custom_objects_add_and_remove(self): class ClassA(object): a = 1 def __init__(self, b): self.b = b self.d = 10 t1 = ClassA(1) t2 = ClassA(2) t2.c = "new attribute" del t2.d return t1, t2 def test_custom_objects_add_and_remove(self): t1, t2 = self.get_custom_objects_add_and_remove() ddiff = DeepDiff(t1, t2) result = { 'attribute_added': {'root.c'}, 'values_changed': { 'root.b': { 'new_value': 2, 'old_value': 1 } }, 'attribute_removed': {'root.d'} } self.assertEqual(ddiff, result) def test_custom_objects_add_and_remove_verbose(self): t1, t2 = self.get_custom_objects_add_and_remove() ddiff = DeepDiff(t1, t2, verbose_level=2) result = { 'attribute_added': { 'root.c': 'new attribute' }, 'attribute_removed': { 'root.d': 10 }, 'values_changed': { 'root.b': { 'new_value': 2, 'old_value': 1 } } } self.assertEqual(ddiff, result) def get_custom_object_with_added_removed_methods(self): class ClassA(object): def method_a(self): pass t1 = ClassA() t2 = ClassA() def method_b(self): pass def method_c(self): return "hello" t2.method_b = method_b t2.method_a = method_c # Note that we are comparing ClassA instances. method_a originally was in ClassA # But we also added another version of it to t2. So it comes up as # added attribute. return t1, t2 def test_custom_objects_add_and_remove_method(self): t1, t2 = self.get_custom_object_with_added_removed_methods() ddiff = DeepDiff(t1, t2) result = {'attribute_added': {'root.method_a', 'root.method_b'}} self.assertEqual(ddiff, result) def test_custom_objects_add_and_remove_method_verbose(self): t1, t2 = self.get_custom_object_with_added_removed_methods() ddiff = DeepDiff(t1, t2, verbose_level=2) self.assertTrue('root.method_a' in ddiff['attribute_added']) self.assertTrue('root.method_b' in ddiff['attribute_added']) def test_set_of_custom_objects(self): member1 = CustomClass(13, 37) member2 = CustomClass(13, 37) t1 = {member1} t2 = {member2} ddiff = DeepDiff(t1, t2) result = {} self.assertEqual(ddiff, result) def test_dictionary_of_custom_objects(self): member1 = CustomClass(13, 37) member2 = CustomClass(13, 37) t1 = {1: member1} t2 = {1: member2} ddiff = DeepDiff(t1, t2) result = {} self.assertEqual(ddiff, result) def test_loop(self): class LoopTest(object): def __init__(self, a): self.loop = self self.a = a t1 = LoopTest(1) t2 = LoopTest(2) ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root.a': { 'old_value': 1, 'new_value': 2 } } } self.assertEqual(ddiff, result) def test_loop2(self): class LoopTestA(object): def __init__(self, a): self.loop = LoopTestB self.a = a class LoopTestB(object): def __init__(self, a): self.loop = LoopTestA self.a = a t1 = LoopTestA(1) t2 = LoopTestA(2) ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root.a': { 'old_value': 1, 'new_value': 2 } } } self.assertEqual(ddiff, result) def test_loop3(self): class LoopTest(object): def __init__(self, a): self.loop = LoopTest self.a = a t1 = LoopTest(1) t2 = LoopTest(2) ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root.a': { 'old_value': 1, 'new_value': 2 } } } self.assertEqual(ddiff, result) def test_loop_in_lists(self): t1 = [1, 2, 3] t1.append(t1) t2 = [1, 2, 4] t2.append(t2) ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root[2]': { 'new_value': 4, 'old_value': 3 } } } self.assertEqual(ddiff, result) def test_loop_in_lists2(self): t1 = [1, 2, [3]] t1[2].append(t1) t2 = [1, 2, [4]] t2[2].append(t2) ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root[2][0]': { 'old_value': 3, 'new_value': 4 } } } self.assertEqual(ddiff, result) def test_decimal(self): t1 = {1: Decimal('10.1')} t2 = {1: Decimal('2.2')} ddiff = DeepDiff(t1, t2) result = { 'values_changed': { 'root[1]': { 'new_value': Decimal('2.2'), 'old_value': Decimal('10.1') } } } self.assertEqual(ddiff, result) def test_decimal_ignore_order(self): t1 = [{1: Decimal('10.1')}, {2: Decimal('10.2')}] t2 = [{2: Decimal('10.2')}, {1: Decimal('10.1')}] ddiff = DeepDiff(t1, t2, ignore_order=True) result = {} self.assertEqual(ddiff, result) def test_unicode_string_type_changes(self): unicode_string = {"hello": u"你好"} ascii_string = {"hello": "你好"} ddiff = DeepDiff(unicode_string, ascii_string) if py3: # In python3, all string is unicode, so diff is empty result = {} else: # In python2, these are 2 different type of strings result = { 'type_changes': { "root['hello']": { 'old_type': unicode, 'new_value': '\xe4\xbd\xa0\xe5\xa5\xbd', 'old_value': u'\u4f60\u597d', 'new_type': str } } } self.assertEqual(ddiff, result) def test_unicode_string_value_changes(self): unicode_string = {"hello": u"你好"} ascii_string = {"hello": u"你好hello"} ddiff = DeepDiff(unicode_string, ascii_string) if py3: result = { 'values_changed': { "root['hello']": { 'old_value': '你好', 'new_value': '你好hello' } } } else: result = { 'values_changed': { "root['hello']": { 'new_value': u'\u4f60\u597dhello', 'old_value': u'\u4f60\u597d' } } } self.assertEqual(ddiff, result) def test_unicode_string_value_and_type_changes(self): unicode_string = {"hello": u"你好"} ascii_string = {"hello": "你好hello"} ddiff = DeepDiff(unicode_string, ascii_string) if py3: # In python3, all string is unicode, so these 2 strings only diff # in values result = { 'values_changed': { "root['hello']": { 'new_value': '你好hello', 'old_value': '你好' } } } else: # In python2, these are 2 different type of strings result = { 'type_changes': { "root['hello']": { 'old_type': unicode, 'new_value': '\xe4\xbd\xa0\xe5\xa5\xbdhello', 'old_value': u'\u4f60\u597d', 'new_type': str } } } self.assertEqual(ddiff, result) def test_int_to_unicode_string(self): t1 = 1 ascii_string = "你好" ddiff = DeepDiff(t1, ascii_string) if py3: # In python3, all string is unicode, so these 2 strings only diff # in values result = { 'type_changes': { 'root': { 'old_type': int, 'new_type': str, 'old_value': 1, 'new_value': '你好' } } } else: # In python2, these are 2 different type of strings result = { 'type_changes': { 'root': { 'old_type': int, 'new_value': '\xe4\xbd\xa0\xe5\xa5\xbd', 'old_value': 1, 'new_type': str } } } self.assertEqual(ddiff, result) def test_int_to_unicode(self): t1 = 1 unicode_string = u"你好" ddiff = DeepDiff(t1, unicode_string) if py3: # In python3, all string is unicode, so these 2 strings only diff # in values result = { 'type_changes': { 'root': { 'old_type': int, 'new_type': str, 'old_value': 1, 'new_value': '你好' } } } else: # In python2, these are 2 different type of strings result = { 'type_changes': { 'root': { 'old_type': int, 'new_value': u'\u4f60\u597d', 'old_value': 1, 'new_type': unicode } } } self.assertEqual(ddiff, result) def test_significant_digits_for_decimals(self): t1 = Decimal('2.5') t2 = Decimal('1.5') ddiff = DeepDiff(t1, t2, significant_digits=0) self.assertEqual(ddiff, {}) def test_significant_digits_for_complex_imaginary_part(self): t1 = 1.23 + 1.222254j t2 = 1.23 + 1.222256j ddiff = DeepDiff(t1, t2, significant_digits=4) self.assertEqual(ddiff, {}) result = { 'values_changed': { 'root': { 'new_value': (1.23 + 1.222256j), 'old_value': (1.23 + 1.222254j) } } } ddiff = DeepDiff(t1, t2, significant_digits=5) self.assertEqual(ddiff, result) def test_significant_digits_for_complex_real_part(self): t1 = 1.23446879 + 1.22225j t2 = 1.23446764 + 1.22225j ddiff = DeepDiff(t1, t2, significant_digits=5) self.assertEqual(ddiff, {}) def test_significant_digits_for_list_of_floats(self): t1 = [1.2344, 5.67881, 6.778879] t2 = [1.2343, 5.67882, 6.778878] ddiff = DeepDiff(t1, t2, significant_digits=3) self.assertEqual(ddiff, {}) ddiff = DeepDiff(t1, t2, significant_digits=4) result = { 'values_changed': { 'root[0]': { 'new_value': 1.2343, 'old_value': 1.2344 } } } self.assertEqual(ddiff, result) ddiff = DeepDiff(t1, t2, significant_digits=5) result = { 'values_changed': { 'root[0]': { 'new_value': 1.2343, 'old_value': 1.2344 }, 'root[1]': { 'new_value': 5.67882, 'old_value': 5.67881 } } } self.assertEqual(ddiff, result) ddiff = DeepDiff(t1, t2) ddiff2 = DeepDiff(t1, t2, significant_digits=6) self.assertEqual(ddiff, ddiff2) def test_negative_significant_digits(self): with self.assertRaises(ValueError): DeepDiff(1, 1, significant_digits=-1) def test_base_level_dictionary_remapping(self): """ Since subclassed dictionaries that override __getitem__ treat newdict.get(key) differently than newdict['key'], we are unable to create a unittest with self.assertIn() and have to resort to fetching the values of two keys and making sure they are the same value. """ t1 = {1: 1, 2: 2} t2 = {2: 2, 3: 3} ddiff = DeepDiff(t1, t2) self.assertEqual(ddiff['dic_item_added'], ddiff['dictionary_item_added']) self.assertEqual(ddiff['dic_item_removed'], ddiff['dictionary_item_removed']) def test_index_and_repeat_dictionary_remapping(self): t1 = [1, 3, 1, 4] t2 = [4, 4, 1] ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True) self.assertEqual(ddiff['repetition_change']['root[0]']['newindexes'], ddiff['repetition_change']['root[0]']['new_indexes']) self.assertEqual(ddiff['repetition_change']['root[0]']['newrepeat'], ddiff['repetition_change']['root[0]']['new_repeat']) self.assertEqual(ddiff['repetition_change']['root[0]']['oldindexes'], ddiff['repetition_change']['root[0]']['old_indexes']) self.assertEqual(ddiff['repetition_change']['root[0]']['oldrepeat'], ddiff['repetition_change']['root[0]']['old_repeat']) def test_value_and_type_dictionary_remapping(self): t1 = {1: 1, 2: 2} t2 = {1: 1, 2: '2'} ddiff = DeepDiff(t1, t2) self.assertEqual(ddiff['type_changes']['root[2]']['newtype'], ddiff['type_changes']['root[2]']['new_type']) self.assertEqual(ddiff['type_changes']['root[2]']['newvalue'], ddiff['type_changes']['root[2]']['new_value']) self.assertEqual(ddiff['type_changes']['root[2]']['oldtype'], ddiff['type_changes']['root[2]']['old_type']) self.assertEqual(ddiff['type_changes']['root[2]']['oldvalue'], ddiff['type_changes']['root[2]']['old_value']) def test_skip_type(self): l1 = logging.getLogger("test") l2 = logging.getLogger("test2") t1 = {"log": l1, 2: 1337} t2 = {"log": l2, 2: 1337} ddiff = DeepDiff(t1, t2, exclude_types={logging.Logger}) self.assertEqual(ddiff, {}) t1 = {"log": "book", 2: 1337} t2 = {"log": l2, 2: 1337} ddiff = DeepDiff(t1, t2, exclude_types={logging.Logger}) self.assertEqual(ddiff, {}) def test_skip_path1(self): t1 = { "for life": "vegan", "ingredients": ["no meat", "no eggs", "no dairy"] } t2 = { "for life": "vegan", "ingredients": ["veggies", "tofu", "soy sauce"] } ddiff = DeepDiff(t1, t2, exclude_paths={"root['ingredients']"}) self.assertEqual(ddiff, {}) def test_skip_path2(self): t1 = { "for life": "vegan", "ingredients": ["no meat", "no eggs", "no dairy"] } t2 = {"for life": "vegan"} ddiff = DeepDiff(t1, t2, exclude_paths={"root['ingredients']"}) self.assertEqual(ddiff, {}) def test_skip_path2_reverse(self): t1 = { "for life": "vegan", "ingredients": ["no meat", "no eggs", "no dairy"] } t2 = {"for life": "vegan"} ddiff = DeepDiff(t2, t1, exclude_paths={"root['ingredients']"}) self.assertEqual(ddiff, {}) def test_skip_path4(self): t1 = { "for life": "vegan", "ingredients": ["no meat", "no eggs", "no dairy"] } t2 = {"for life": "vegan", "zutaten": ["veggies", "tofu", "soy sauce"]} ddiff = DeepDiff(t1, t2, exclude_paths={"root['ingredients']"}) self.assertTrue('dictionary_item_added' in ddiff, {}) self.assertTrue('dictionary_item_removed' not in ddiff, {}) def test_skip_custom_object_path(self): t1 = CustomClass(1) t2 = CustomClass(2) ddiff = DeepDiff(t1, t2, exclude_paths=['root.a']) result = {} self.assertEqual(ddiff, result) def test_skip_list_path(self): t1 = ['a', 'b'] t2 = ['a'] ddiff = DeepDiff(t1, t2, exclude_paths=['root[1]']) result = {} self.assertEqual(ddiff, result) def test_skip_dictionary_path(self): t1 = {1: {2: "a"}} t2 = {1: {}} ddiff = DeepDiff(t1, t2, exclude_paths=['root[1][2]']) result = {} self.assertEqual(ddiff, result) def test_skip_dictionary_path_with_custom_object(self): obj1 = CustomClass(1) obj2 = CustomClass(2) t1 = {1: {2: obj1}} t2 = {1: {2: obj2}} ddiff = DeepDiff(t1, t2, exclude_paths=['root[1][2].a']) result = {} self.assertEqual(ddiff, result) def test_skip_str_type_in_dictionary(self): t1 = {1: {2: "a"}} t2 = {1: {}} ddiff = DeepDiff(t1, t2, exclude_types=[str]) result = {} self.assertEqual(ddiff, result) def test_unknown_parameters(self): with self.assertRaises(ValueError): DeepDiff(1, 1, wrong_param=2) def test_bad_attribute(self): class Bad(object): __slots__ = ['x', 'y'] def __getattr__(self, key): raise AttributeError("Bad item") def __str__(self): return "Bad Object" t1 = Bad() t2 = Bad() ddiff = DeepDiff(t1, t2) result = {'unprocessed': ['root: Bad Object and Bad Object']} self.assertEqual(ddiff, result) def test_dict_none_item_removed(self): t1 = {1: None, 2: 2} t2 = {2: 2} ddiff = DeepDiff(t1, t2) result = { 'dictionary_item_removed': {'root[1]'} } self.assertEqual(ddiff, result) def test_list_none_item_removed(self): t1 = [1, 2, None] t2 = [1, 2] ddiff = DeepDiff(t1, t2) result = { 'iterable_item_removed': {'root[2]': None} } self.assertEqual(ddiff, result) def test_non_subscriptable_iterable(self): def gen1(): yield 42 yield 1337 yield 31337 def gen2(): yield 42 yield 1337 t1 = gen1() t2 = gen2() ddiff = DeepDiff(t1, t2) result = {'iterable_item_removed': {'root[2]': 31337}} # Note: In text-style results, we currently pretend this stuff is subscriptable for readability self.assertEqual(ddiff, result) @mock.patch('deepdiff.diff.logger') @mock.patch('deepdiff.diff.DeepHash') def test_diff_when_hash_fails(self, mock_DeepHash, mock_logger): mock_DeepHash.side_effect = Exception('Boom!') t1 = {"blah": {4}, 2: 1337} t2 = {"blah": {4}, 2: 1337} DeepDiff(t1, t2, ignore_order=True) assert mock_logger.warning.called deepdiff-3.3.0/tests/test_diff_tree.py000066400000000000000000000157251312554330700200020ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- """ To run only the search tests: python -m unittest tests.test_diff_tree Or to run all the tests: python -m unittest discover Or to run all the tests with coverage: coverage run --source deepdiff setup.py test Or using Nose: nosetests --with-coverage --cover-package=deepdiff To run a specific test, run this from the root of repo: python -m unittest tests.test_diff_tree.DeepDiffTreeTestCase.test_same_objects """ import unittest from deepdiff import DeepDiff from deepdiff.helper import pypy3, notpresent from deepdiff.model import DictRelationship, NonSubscriptableIterableRelationship import logging logging.disable(logging.CRITICAL) class DeepDiffTreeTestCase(unittest.TestCase): """DeepDiff Tests.""" def test_same_objects(self): t1 = {1: 1, 2: 2, 3: 3} t2 = t1 ddiff = DeepDiff(t1, t2) res = ddiff.tree self.assertEqual(res, {}) def test_significant_digits_signed_zero(self): t1 = 0.00001 t2 = -0.0001 ddiff = DeepDiff(t1, t2, significant_digits=2) res = ddiff.tree self.assertEqual(res, {}) t1 = 1 * 10**-12 t2 = -1 * 10**-12 ddiff = DeepDiff(t1, t2, significant_digits=10) res = ddiff.tree self.assertEqual(res, {}) def test_item_added_extensive(self): t1 = {'one': 1, 'two': 2, 'three': 3, 'four': 4} t2 = {'one': 1, 'two': 2, 'three': 3, 'four': 4, 'new': 1337} ddiff = DeepDiff(t1, t2) res = ddiff.tree (key, ) = res.keys() self.assertEqual(key, 'dictionary_item_added') self.assertEqual(len(res['dictionary_item_added']), 1) (added1, ) = res['dictionary_item_added'] # assert added1 DiffLevel chain is valid at all self.assertEqual(added1.up.down, added1) self.assertIsNone(added1.down) self.assertIsNone(added1.up.up) self.assertEqual(added1.all_up, added1.up) self.assertEqual(added1.up.all_down, added1) self.assertEqual(added1.report_type, 'dictionary_item_added') # assert DiffLevel chain points to the objects we entered self.assertEqual(added1.up.t1, t1) self.assertEqual(added1.up.t2, t2) self.assertEqual(added1.t1, notpresent) self.assertEqual(added1.t2, 1337) # assert DiffLevel child relationships are correct self.assertIsNone(added1.up.t1_child_rel) self.assertIsInstance(added1.up.t2_child_rel, DictRelationship) self.assertEqual(added1.up.t2_child_rel.parent, added1.up.t2) self.assertEqual(added1.up.t2_child_rel.child, added1.t2) self.assertEqual(added1.up.t2_child_rel.param, 'new') self.assertEqual(added1.up.path(), "root") self.assertEqual(added1.path(), "root['new']") def test_item_added_and_removed(self): t1 = {'one': 1, 'two': 2, 'three': 3, 'four': 4} t2 = {'one': 1, 'two': 4, 'three': 3, 'five': 5, 'six': 6} ddiff = DeepDiff(t1, t2, view='tree') self.assertEqual( set(ddiff.keys()), { 'dictionary_item_added', 'dictionary_item_removed', 'values_changed' }) self.assertEqual(len(ddiff['dictionary_item_added']), 2) self.assertEqual(len(ddiff['dictionary_item_removed']), 1) def test_item_added_and_removed2(self): t1 = {2: 2, 4: 4} t2 = {2: "b", 5: 5} ddiff = DeepDiff(t1, t2, view='tree') self.assertEqual( set(ddiff.keys()), { 'dictionary_item_added', 'dictionary_item_removed', 'type_changes' }) self.assertEqual(len(ddiff['dictionary_item_added']), 1) self.assertEqual(len(ddiff['dictionary_item_removed']), 1) def test_non_subscriptable_iterable(self): t1 = (i for i in [42, 1337, 31337]) t2 = (i for i in [ 42, 1337, ]) ddiff = DeepDiff(t1, t2, view='tree') (change, ) = ddiff['iterable_item_removed'] self.assertEqual(set(ddiff.keys()), {'iterable_item_removed'}) self.assertEqual(len(ddiff['iterable_item_removed']), 1) self.assertEqual(change.up.t1, t1) self.assertEqual(change.up.t2, t2) self.assertEqual(change.report_type, 'iterable_item_removed') self.assertEqual(change.t1, 31337) self.assertEqual(change.t2, notpresent) self.assertIsInstance(change.up.t1_child_rel, NonSubscriptableIterableRelationship) self.assertIsNone(change.up.t2_child_rel) def test_non_subscriptable_iterable_path(self): t1 = (i for i in [42, 1337, 31337]) t2 = (i for i in [42, 1337, ]) ddiff = DeepDiff(t1, t2, view='tree') (change, ) = ddiff['iterable_item_removed'] # testing path self.assertEqual(change.path(), None) self.assertEqual(change.path(force='yes'), 'root(unrepresentable)') self.assertEqual(change.path(force='fake'), 'root[2]') def test_significant_digits(self): ddiff = DeepDiff( [0.012, 0.98], [0.013, 0.99], significant_digits=1, view='tree') self.assertEqual(ddiff, {}) def test_significant_digits_with_sets(self): ddiff = DeepDiff( {0.012, 0.98}, {0.013, 0.99}, significant_digits=1, view='tree') self.assertEqual(ddiff, {}) def test_significant_digits_with_ignore_order(self): ddiff = DeepDiff( [0.012, 0.98], [0.013, 0.99], significant_digits=1, ignore_order=True, view='tree') self.assertEqual(ddiff, {}) def test_repr(self): t1 = {1, 2, 8} t2 = {1, 2, 3, 5} ddiff = DeepDiff(t1, t2, view='tree') try: str(ddiff) except Exception as e: self.fail("Converting ddiff to string raised: {}".format(e)) class DeepDiffTreeWithNumpyTestCase(unittest.TestCase): """DeepDiff Tests with Numpy.""" def setUp(self): if not pypy3: import numpy as np a1 = np.array([1.23, 1.66, 1.98]) a2 = np.array([1.23, 1.66, 1.98]) self.d1 = {'np': a1} self.d2 = {'np': a2} @unittest.skipIf(pypy3, "Numpy is not compatible with pypy3") def test_diff_with_numpy(self): ddiff = DeepDiff(self.d1, self.d2) res = ddiff.tree self.assertEqual(res, {}) @unittest.skipIf(pypy3, "Numpy is not compatible with pypy3") def test_diff_with_empty_seq(self): a1 = {"empty": []} a2 = {"empty": []} ddiff = DeepDiff(a1, a2) self.assertEqual(ddiff, {}) class DeepAdditionsTestCase(unittest.TestCase): """Tests for Additions and Subtractions.""" @unittest.expectedFailure def test_adding_list_diff(self): t1 = [1, 2] t2 = [1, 2, 3, 5] ddiff = DeepDiff(t1, t2, view='tree') addition = ddiff + t1 self.assertEqual(addition, t2) deepdiff-3.3.0/tests/test_hash.py000066400000000000000000000267561312554330700170040ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- """ To run only the search tests: python -m unittest tests.test_hash Or to run all the tests: python -m unittest discover Or to run all the tests with coverage: coverage run --source deepdiff setup.py test Or using Nose: nosetests --with-coverage --cover-package=deepdiff To run a specific test, run this from the root of repo: On linux: nosetests ./tests/test_hash.py:DeepHashTestCase.test_bytecode On windows: nosetests .\tests\test_hash.py:DeepHashTestCase.test_string_in_root """ import unittest from deepdiff import DeepHash from deepdiff.helper import py3, pypy3 from collections import namedtuple import logging logging.disable(logging.CRITICAL) class CustomClass: def __init__(self, a, b=None): self.a = a self.b = b def __str__(self): return "({}, {})".format(self.a, self.b) def __repr__(self): return self.__str__() def hash_and_format(obj): return "str:{}".format(hash(obj)) class DeepHashTestCase(unittest.TestCase): """DeepHash Tests.""" def test_hash_str(self): obj = "a" expected_result = {id(obj): hash_and_format(obj)} result = DeepHash(obj) self.assertEqual(result, expected_result) def test_hash_str_fail_if_mutable(self): """ This test fails if ContentHash is getting a mutable copy of hashes which means each init of the ContentHash will have hashes from the previous init. """ obj1 = "a" id_obj1 = id(obj1) expected_result = {id_obj1: hash_and_format(obj1)} result = DeepHash(obj1) self.assertEqual(result, expected_result) obj2 = "b" result = DeepHash(obj2) self.assertTrue(id_obj1 not in result) def test_list(self): string1 = "a" obj = [string1, 10, 20] expected_result = { id(string1): hash_and_format(string1), id(obj): 'list:int:10,int:20,str:%s' % hash(string1) } result = DeepHash(obj) self.assertEqual(result, expected_result) def test_tuple(self): string1 = "a" obj = (string1, 10, 20) expected_result = { id(string1): hash_and_format(string1), id(obj): 'tuple:int:10,int:20,str:%s' % hash(string1) } result = DeepHash(obj) self.assertEqual(result, expected_result) def test_named_tuples(self): # checking if pypy3 is running the test # in that case due to a pypy3 bug or something # the id of x inside the named tuple changes. x = "x" x_id = id(x) x_hash = hash(x) Point = namedtuple('Point', [x]) obj = Point(x=11) result = DeepHash(obj) if pypy3: self.assertEqual(result[id(obj)], 'ntdict:{str:%s:int:11}' % x_hash) else: expected_result = { x_id: 'str:{}'.format(x_hash), id(obj): 'ntdict:{str:%s:int:11}' % x_hash } self.assertEqual(result, expected_result) def test_dict(self): string1 = "a" hash_string1 = hash(string1) key1 = "key1" hash_key1 = hash(key1) obj = {key1: string1, 1: 10, 2: 20} expected_result = { id(key1): "str:{}".format(hash_key1), id(string1): "str:{}".format(hash_string1), id(obj): 'dict:{int:1:int:10;int:2:int:20;str:%s:str:%s}' % (hash_key1, hash_string1) } result = DeepHash(obj) self.assertEqual(result, expected_result) def test_dict_in_list(self): string1 = "a" hash_string1 = hash(string1) key1 = "key1" hash_key1 = hash(key1) dict1 = {key1: string1, 1: 10, 2: 20} obj = [0, dict1] expected_result = { id(key1): "str:{}".format(hash_key1), id(string1): "str:{}".format(hash_string1), id(dict1): 'dict:{int:1:int:10;int:2:int:20;str:%s:str:%s}' % (hash_key1, hash_string1), id(obj): 'list:dict:{int:1:int:10;int:2:int:20;str:%s:str:%s},int:0' % (hash_key1, hash_string1) } result = DeepHash(obj) self.assertEqual(result, expected_result) def test_nested_lists_same_hash(self): t1 = [1, 2, [3, 4]] t2 = [[4, 3], 2, 1] t1_hash = DeepHash(t1) t2_hash = DeepHash(t2) self.assertEqual(t1_hash[id(t1)], t2_hash[id(t2)]) def test_nested_lists_same_hash2(self): t1 = [1, 2, [3, [4, 5]]] t2 = [[[5, 4], 3], 2, 1] t1_hash = DeepHash(t1) t2_hash = DeepHash(t2) self.assertEqual(t1_hash[id(t1)], t2_hash[id(t2)]) def test_nested_lists_same_hash3(self): t1 = [{1: [2, 3], 4: [5, [6, 7]]}] t2 = [{4: [[7, 6], 5], 1: [3, 2]}] t1_hash = DeepHash(t1) t2_hash = DeepHash(t2) self.assertEqual(t1_hash[id(t1)], t2_hash[id(t2)]) def test_nested_lists_in_dictionary_same_hash(self): t1 = [{"c": 4}, {"c": 3}] t2 = [{"c": 3}, {"c": 4}] t1_hash = DeepHash(t1) t2_hash = DeepHash(t2) self.assertEqual(t1_hash[id(t1)], t2_hash[id(t2)]) def test_same_sets_same_hash(self): t1 = {1, 3, 2} t2 = {2, 3, 1} t1_hash = DeepHash(t1) t2_hash = DeepHash(t2) self.assertEqual(t1_hash[id(t1)], t2_hash[id(t2)]) def test_same_sets_in_lists_same_hash(self): t1 = ["a", {1, 3, 2}] t2 = [{2, 3, 1}, "a"] t1_hash = DeepHash(t1) t2_hash = DeepHash(t2) self.assertEqual(t1_hash[id(t1)], t2_hash[id(t2)]) def test_unknown_parameters(self): with self.assertRaises(ValueError): DeepHash(1, wrong_param=2) def test_bad_attribute(self): class Bad(object): __slots__ = ['x', 'y'] def __getattr__(self, key): raise AttributeError("Bad item") def __str__(self): return "Bad Object" t1 = Bad() result = DeepHash(t1) expected_result = {id(t1): result.unprocessed, 'unprocessed': [t1]} self.assertEqual(result, expected_result) def test_repetition_by_default_does_not_effect(self): list1 = [3, 4] list1_id = id(list1) a = [1, 2, list1] a_id = id(a) list2 = [4, 3, 3] list2_id = id(list2) b = [list2, 2, 1] b_id = id(b) hash_a = DeepHash(a) hash_b = DeepHash(b) self.assertEqual(hash_a[list1_id], hash_b[list2_id]) self.assertEqual(hash_a[a_id], hash_b[b_id]) def test_setting_repetition_off_unequal_hash(self): list1 = [3, 4] list1_id = id(list1) a = [1, 2, list1] a_id = id(a) list2 = [4, 3, 3] list2_id = id(list2) b = [list2, 2, 1] b_id = id(b) hash_a = DeepHash(a, ignore_repetition=False) hash_b = DeepHash(b, ignore_repetition=False) self.assertNotEqual(hash_a[list1_id], hash_b[list2_id]) self.assertNotEqual(hash_a[a_id], hash_b[b_id]) self.assertEqual(hash_a[list1_id].replace('3|1', '3|2'), hash_b[list2_id]) def test_already_calculated_hash_wont_be_recalculated(self): hashes = (i for i in range(10)) def hasher(obj): return next(hashes) obj = "a" expected_result = {id(obj): "str:0"} result = DeepHash(obj, hasher=hasher) self.assertEqual(result, expected_result) # we simply feed the last result to DeepHash # So it can re-use the results. result2 = DeepHash(obj, hasher=hasher, hashes=result) # if hashes are not cached and re-used, # then the next time hasher runs, it returns # number 1 instead of 0. self.assertEqual(result2, expected_result) result3 = DeepHash(obj, hasher=hasher) expected_result = {id(obj): "str:{}".format(1)} self.assertEqual(result3, expected_result) def test_skip_type(self): l1 = logging.getLogger("test") obj = {"log": l1, 2: 1337} result = DeepHash(obj, exclude_types={logging.Logger}) self.assertEqual(result[id(l1)], result.skipped) def test_hash_dic_with_loop(self): obj = {2: 1337} obj[1] = obj result = DeepHash(obj) expected_result = {id(obj): 'dict:{int:2:int:1337}'} self.assertEqual(result, expected_result) def test_hash_iterable_with_loop(self): obj = [1] obj.append(obj) result = DeepHash(obj) expected_result = {id(obj): 'list:int:1'} self.assertEqual(result, expected_result) def test_hash_iterable_with_excluded_type(self): l1 = logging.getLogger("test") obj = [1, l1] result = DeepHash(obj, exclude_types={logging.Logger}) self.assertTrue(id(l1) not in result) class DeepHashSHA1TestCase(unittest.TestCase): """DeepHash with SHA1 Tests.""" def test_hash_str(self): obj = "a" expected_result = { id(obj): 'str:48591f1d794734cabf55f96f5a5a72c084f13ac0' } result = DeepHash(obj, hasher=DeepHash.sha1hex) self.assertEqual(result, expected_result) def test_hash_str_fail_if_mutable(self): """ This test fails if ContentHash is getting a mutable copy of hashes which means each init of the ContentHash will have hashes from the previous init. """ obj1 = "a" id_obj1 = id(obj1) expected_result = { id_obj1: 'str:48591f1d794734cabf55f96f5a5a72c084f13ac0' } result = DeepHash(obj1, hasher=DeepHash.sha1hex) self.assertEqual(result, expected_result) obj2 = "b" result = DeepHash(obj2, hasher=DeepHash.sha1hex) self.assertTrue(id_obj1 not in result) def test_bytecode(self): obj = b"a" if py3: expected_result = { id(obj): 'str:066c7cf4158717c47244fa6cf1caafca605d550b' } else: expected_result = { id(obj): 'str:48591f1d794734cabf55f96f5a5a72c084f13ac0' } result = DeepHash(obj, hasher=DeepHash.sha1hex) self.assertEqual(result, expected_result) def test_list1(self): string1 = "a" obj = [string1, 10, 20] expected_result = { id(string1): 'str:48591f1d794734cabf55f96f5a5a72c084f13ac0', id(obj): 'list:int:10,int:20,str:48591f1d794734cabf55f96f5a5a72c084f13ac0' } result = DeepHash(obj, hasher=DeepHash.sha1hex) self.assertEqual(result, expected_result) def test_dict1(self): string1 = "a" key1 = "key1" obj = {key1: string1, 1: 10, 2: 20} expected_result = { id(key1): 'str:63216212fdf88fe0c838c36ab65278b9953000d6', id(string1): 'str:48591f1d794734cabf55f96f5a5a72c084f13ac0', id(obj): 'dict:{int:1:int:10;int:2:int:20;str:63216212fdf88fe0c838c36ab65278b9953000d6:str:48591f1d794734cabf55f96f5a5a72c084f13ac0}' } result = DeepHash(obj, hasher=DeepHash.sha1hex) self.assertEqual(result, expected_result) deepdiff-3.3.0/tests/test_helper.py000066400000000000000000000007421312554330700173230ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- import unittest from deepdiff.helper import short_repr class HelperTestCase(unittest.TestCase): """Helper Tests.""" def test_short_repr_when_short(self): item = {1: 2} output = short_repr(item) self.assertEqual(output, '{1: 2}') def test_short_repr_when_long(self): item = {'Eat more': 'burritos'} output = short_repr(item) self.assertEqual(output, "{'Eat more':...}") deepdiff-3.3.0/tests/test_model.py000066400000000000000000000206271312554330700171500ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- """ To run only the search tests: python -m unittest tests.test_diff_ref Or to run all the tests: python -m unittest discover Or to run all the tests with coverage: coverage run --source deepdiff setup.py test Or using Nose: nosetests --with-coverage --cover-package=deepdiff To run a specific test, run this from the root of repo: python -m unittest tests.test_model.DiffLevelTestCase.test_path_when_both_children_empty """ from unittest import TestCase import logging from tests import CustomClass, CustomClassMisleadingRepr from deepdiff.model import (DiffLevel, ChildRelationship, DictRelationship, SubscriptableIterableRelationship, AttributeRelationship) from deepdiff.helper import Verbose logging.disable(logging.CRITICAL) class WorkingChildRelationship(ChildRelationship): def get_param_from_obj(self, obj): return obj class DictRelationshipTestCase(TestCase): def setUp(self): self.customkey = CustomClass(a=13, b=37) self.customkey_misleading = CustomClassMisleadingRepr(a=11, b=20) self.d = { 42: 'answer', 'vegan': 'for life', self.customkey: 1337, self.customkey_misleading: 'banana' } def test_numkey(self): rel = DictRelationship(parent=self.d, child=self.d[42], param=42) self.assertEqual(rel.get_param_repr(), "[42]") def test_strkey(self): rel = ChildRelationship.create( klass=DictRelationship, parent=self.d, child=self.d['vegan'], param='vegan') result = rel.get_param_repr() self.assertEqual(result, "['vegan']") def test_objkey(self): rel = DictRelationship( parent=self.d, child=self.d[self.customkey], param=self.customkey) self.assertIsNone(rel.get_param_repr()) def test_objkey_misleading_repr(self): rel = DictRelationship( parent=self.d, child=self.d[self.customkey_misleading], param=self.customkey_misleading) self.assertIsNone(rel.get_param_repr()) def test_get_param_from_dict(self): param = 42 rel = DictRelationship(parent=self.d, child=self.d[param], param=param) obj = {10: 10, param: 123} self.assertEqual(rel.get_param_from_obj(obj), 123) class ListRelationshipTestCase(TestCase): def setUp(self): self.custom = CustomClass(13, 37) self.l = [1337, 'vegan', self.custom] def test_min(self): rel = SubscriptableIterableRelationship(self.l, self.l[0], 0) result = rel.get_param_repr() self.assertEqual(result, "[0]") def test_max(self): rel = ChildRelationship.create(SubscriptableIterableRelationship, self.l, self.custom, 2) self.assertEqual(rel.get_param_repr(), "[2]") def test_get_param_from_obj(self): param = 0 rel = SubscriptableIterableRelationship(parent=self.l, child=self.l[param], param=param) obj = ['a', 'b', 'c'] self.assertEqual(rel.get_param_from_obj(obj), 'a') class AttributeRelationshipTestCase(TestCase): def setUp(self): self.custom = CustomClass(13, 37) def test_a(self): rel = AttributeRelationship(self.custom, 13, "a") result = rel.get_param_repr() self.assertEqual(result, ".a") def test_get_param_from_obj(self): rel = AttributeRelationship(self.custom, 13, "a") self.assertEqual(rel.get_param_from_obj(self.custom), 13) class DiffLevelTestCase(TestCase): def setUp(self): # Test data self.custom1 = CustomClass(a='very long text here', b=37) self.custom2 = CustomClass(a=313, b=37) self.t1 = {42: 'answer', 'vegan': 'for life', 1337: self.custom1} self.t2 = { 42: 'answer', 'vegan': 'for the animals', 1337: self.custom2 } # Manually build diff, bottom up self.lowest = DiffLevel( self.custom1.a, self.custom2.a, report_type='values_changed') # Test manual child relationship rel_int_low_t1 = AttributeRelationship( parent=self.custom1, child=self.custom1.a, param="a") rel_int_low_t2 = AttributeRelationship( parent=self.custom2, child=self.custom2.a, param="a") self.intermediate = DiffLevel( self.custom1, self.custom2, down=self.lowest, child_rel1=rel_int_low_t1, child_rel2=rel_int_low_t2) self.lowest.up = self.intermediate # Test automatic child relationship t1_child_rel = ChildRelationship.create( klass=DictRelationship, parent=self.t1, child=self.intermediate.t1, param=1337) t2_child_rel = ChildRelationship.create( klass=DictRelationship, parent=self.t2, child=self.intermediate.t2, param=1337) self.highest = DiffLevel( self.t1, self.t2, down=self.intermediate, child_rel1=t1_child_rel, child_rel2=t2_child_rel) self.intermediate.up = self.highest def test_all_up(self): self.assertEqual(self.lowest.all_up, self.highest) def test_all_down(self): self.assertEqual(self.highest.all_down, self.lowest) def test_automatic_child_rel(self): self.assertIsInstance(self.highest.t1_child_rel, DictRelationship) self.assertIsInstance(self.highest.t2_child_rel, DictRelationship) self.assertEqual(self.highest.t1_child_rel.parent, self.highest.t1) self.assertEqual(self.highest.t2_child_rel.parent, self.highest.t2) self.assertEqual(self.highest.t1_child_rel.parent, self.highest.t1) self.assertEqual(self.highest.t2_child_rel.parent, self.highest.t2) # Provides textual relationship from t1 to t1[1337] self.assertEqual('[1337]', self.highest.t2_child_rel.get_param_repr()) def test_path(self): # Provides textual path all the way through self.assertEqual(self.lowest.path("self.t1"), "self.t1[1337].a") def test_change_of_path_root(self): self.assertEqual(self.lowest.path("root"), "root[1337].a") self.assertEqual(self.lowest.path(""), "[1337].a") def test_path_when_both_children_empty(self): """ This is a situation that should never happen. But we are creating it artificially. """ t1 = {1: 1} t2 = {1: 2} child_t1 = {} child_t2 = {} up = DiffLevel(t1, t2) down = up.down = DiffLevel(child_t1, child_t2) path = down.path() self.assertEqual(path, 'root') def test_repr_short(self): level = Verbose.level Verbose.level = 0 item_repr = repr(self.lowest) Verbose.level = level self.assertEqual(item_repr, '') def test_repr_long(self): level = Verbose.level Verbose.level = 1 item_repr = repr(self.lowest) Verbose.level = level self.assertEqual(item_repr, "") def test_repetition_attribute_and_repr(self): t1 = [1, 1] t2 = [1] some_repetition = 'some repetition' node = DiffLevel(t1, t2) node.additional['repetition'] = some_repetition self.assertEqual(node.repetition, some_repetition) self.assertEqual(repr(node), "") class ChildRelationshipTestCase(TestCase): def test_create_invalid_klass(self): with self.assertRaises(TypeError): ChildRelationship.create(DiffLevel, "hello", 42) def test_rel_repr_short(self): rel = WorkingChildRelationship(parent="that parent", child="this child", param="some param") rel_repr = repr(rel) expected = "" self.assertEqual(rel_repr, expected) def test_rel_repr_long(self): rel = WorkingChildRelationship( parent="that parent who has a long path", child="this child", param="some param") rel_repr = repr(rel) expected = "" self.assertEqual(rel_repr, expected) deepdiff-3.3.0/tests/test_search.py000066400000000000000000000262561312554330700173210ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- """ To run only the search tests: python -m unittest tests.search_tests Or to run all the tests with coverage: coverage run --source deepdiff setup.py test Or using Nose: nosetests --with-coverage --cover-package=deepdiff To run a specific test, run this from the root of repo: nosetests tests/test_search.py:DeepSearchTestCase.test_case_insensitive_of_str_in_list """ import unittest from deepdiff import DeepSearch, grep from datetime import datetime import logging logging.disable(logging.CRITICAL) item = "somewhere" class CustomClass: def __init__(self, a, b=None): self.a = a self.b = b def __str__(self): return "({}, {})".format(self.a, self.b) def __repr__(self): return self.__str__() class DeepSearchTestCase(unittest.TestCase): """DeepSearch Tests.""" def test_number_in_list(self): obj = ["a", 10, 20] item = 10 result = {"matched_values": {'root[1]'}} self.assertEqual(DeepSearch(obj, item, verbose_level=1), result) def test_string_in_root(self): obj = "long string somewhere" result = {"matched_values": {'root'}} self.assertEqual(DeepSearch(obj, item, verbose_level=1), result) def test_string_in_root_verbose(self): obj = "long string somewhere" result = {"matched_values": {'root': "long string somewhere"}} self.assertEqual(DeepSearch(obj, item, verbose_level=2), result) def test_string_in_tuple(self): obj = ("long", "string", 0, "somewhere") result = {"matched_values": {'root[3]'}} self.assertEqual(DeepSearch(obj, item, verbose_level=1), result) def test_string_in_list(self): obj = ["long", "string", 0, "somewhere"] result = {"matched_values": {'root[3]'}} self.assertEqual(DeepSearch(obj, item, verbose_level=1), result) def test_string_in_list_verbose(self): obj = ["long", "string", 0, "somewhere"] result = {"matched_values": {'root[3]': "somewhere"}} self.assertEqual(DeepSearch(obj, item, verbose_level=2), result) def test_string_in_list_verbose2(self): obj = ["long", "string", 0, "somewhere great!"] result = {"matched_values": {'root[3]': "somewhere great!"}} self.assertEqual(DeepSearch(obj, item, verbose_level=2), result) def test_string_in_list_verbose3(self): obj = ["long somewhere", "string", 0, "somewhere great!"] result = { "matched_values": { 'root[0]': 'long somewhere', 'root[3]': "somewhere great!" } } self.assertEqual(DeepSearch(obj, item, verbose_level=2), result) def test_int_in_dictionary(self): obj = {"long": "somewhere", "num": 2, 0: 0, "somewhere": "around"} item = 2 result = {'matched_values': {"root['num']"}} ds = DeepSearch(obj, item, verbose_level=1) self.assertEqual(ds, result) def test_string_in_dictionary(self): obj = {"long": "somewhere", "string": 2, 0: 0, "somewhere": "around"} result = { 'matched_paths': {"root['somewhere']"}, 'matched_values': {"root['long']"} } ds = DeepSearch(obj, item, verbose_level=1) self.assertEqual(ds, result) def test_string_in_dictionary_case_insensitive(self): obj = {"long": "Somewhere over there!", "string": 2, 0: 0, "SOMEWHERE": "around"} result = { 'matched_paths': {"root['SOMEWHERE']"}, 'matched_values': {"root['long']"} } ds = DeepSearch(obj, item, verbose_level=1, case_sensitive=False) self.assertEqual(ds, result) def test_string_in_dictionary_key_case_insensitive_partial(self): obj = {"SOMEWHERE here": "around"} result = { 'matched_paths': {"root['SOMEWHERE here']"} } ds = DeepSearch(obj, item, verbose_level=1, case_sensitive=False) self.assertEqual(ds, result) def test_string_in_dictionary_verbose(self): obj = {"long": "somewhere", "string": 2, 0: 0, "somewhere": "around"} result = { 'matched_paths': { "root['somewhere']": "around" }, 'matched_values': { "root['long']": "somewhere" } } ds = DeepSearch(obj, item, verbose_level=2) self.assertEqual(ds, result) def test_string_in_dictionary_in_list_verbose(self): obj = [ "something somewhere", { "long": "somewhere", "string": 2, 0: 0, "somewhere": "around" } ] result = { 'matched_paths': { "root[1]['somewhere']": "around" }, 'matched_values': { "root[1]['long']": "somewhere", "root[0]": "something somewhere" } } ds = DeepSearch(obj, item, verbose_level=2) self.assertEqual(ds, result) def test_custom_object(self): obj = CustomClass('here, something', 'somewhere') result = {'matched_values': {'root.b'}} ds = DeepSearch(obj, item, verbose_level=1) self.assertEqual(ds, result) def test_custom_object_verbose(self): obj = CustomClass('here, something', 'somewhere out there') result = {'matched_values': {'root.b': 'somewhere out there'}} ds = DeepSearch(obj, item, verbose_level=2) self.assertEqual(ds, result) def test_custom_object_in_dictionary_verbose(self): obj = {1: CustomClass('here, something', 'somewhere out there')} result = {'matched_values': {'root[1].b': 'somewhere out there'}} ds = DeepSearch(obj, item, verbose_level=2) self.assertEqual(ds, result) def test_named_tuples_verbose(self): from collections import namedtuple Point = namedtuple('Point', ['x', 'somewhere_good']) obj = Point(x="my keys are somewhere", somewhere_good=22) ds = DeepSearch(obj, item, verbose_level=2) result = { 'matched_values': { 'root.x': 'my keys are somewhere' }, 'matched_paths': { 'root.somewhere_good': 22 } } self.assertEqual(ds, result) def test_string_in_set_verbose(self): obj = {"long", "string", 0, "somewhere"} # result = {"matched_values": {'root[3]': "somewhere"}} ds = DeepSearch(obj, item, verbose_level=2) self.assertEqual(list(ds["matched_values"].values())[0], item) def test_loop(self): class LoopTest(object): def __init__(self, a): self.loop = self self.a = a obj = LoopTest("somewhere around here.") ds = DeepSearch(obj, item, verbose_level=1) result = {'matched_values': {'root.a'}} self.assertEqual(ds, result) def test_loop_in_lists(self): obj = [1, 2, 'somewhere'] obj.append(obj) ds = DeepSearch(obj, item, verbose_level=1) result = {'matched_values': {'root[2]'}} self.assertEqual(ds, result) def test_skip_path1(self): obj = { "for life": "vegan", "ingredients": ["no meat", "no eggs", "no dairy", "somewhere"] } ds = DeepSearch(obj, item, exclude_paths={"root['ingredients']"}) self.assertEqual(ds, {}) def test_custom_object_skip_path(self): obj = CustomClass('here, something', 'somewhere') result = {} ds = DeepSearch(obj, item, verbose_level=1, exclude_paths=['root.b']) self.assertEqual(ds, result) def test_skip_list_path(self): obj = ['a', 'somewhere'] ds = DeepSearch(obj, item, exclude_paths=['root[1]']) result = {} self.assertEqual(ds, result) def test_skip_dictionary_path(self): obj = {1: {2: "somewhere"}} ds = DeepSearch(obj, item, exclude_paths=['root[1][2]']) result = {} self.assertEqual(ds, result) def test_skip_type_str(self): obj = "long string somewhere" result = {} ds = DeepSearch(obj, item, verbose_level=1, exclude_types=[str]) self.assertEqual(ds, result) def test_unknown_parameters(self): with self.assertRaises(ValueError): DeepSearch(1, 1, wrong_param=2) def test_bad_attribute(self): class Bad(object): __slots__ = ['x', 'y'] def __getattr__(self, key): raise AttributeError("Bad item") def __str__(self): return "Bad Object" obj = Bad() ds = DeepSearch(obj, item, verbose_level=1) result = {'unprocessed': ['root']} self.assertEqual(ds, result) ds = DeepSearch(obj, item, verbose_level=2) self.assertEqual(ds, result) def test_case_insensitive_of_str_in_list(self): obj = ["a", "bb", "BBC", "aBbB"] item = "BB" result = {"matched_values": {'root[1]', 'root[2]', 'root[3]'}} self.assertEqual(DeepSearch(obj, item, verbose_level=1, case_sensitive=False), result) def test_case_sensitive_of_str_in_list(self): obj = ["a", "bb", "BBC", "aBbB"] item = "BB" result = {"matched_values": {'root[2]'}} self.assertEqual(DeepSearch(obj, item, verbose_level=1, case_sensitive=True), result) def test_case_sensitive_of_str_in_one_liner(self): obj = "Hello, what's up?" item = "WHAT" result = {} self.assertEqual(DeepSearch(obj, item, verbose_level=1, case_sensitive=True), result) def test_case_insensitive_of_str_in_one_liner(self): obj = "Hello, what's up?" item = "WHAT" result = {'matched_values': {'root'}} self.assertEqual(DeepSearch(obj, item, verbose_level=1, case_sensitive=False), result) def test_none(self): obj = item = None result = {'matched_values': {'root'}} self.assertEqual(DeepSearch(obj, item, verbose_level=1), result) def test_complex_obj(self): obj = datetime(2017, 5, 4, 1, 1, 1) item = datetime(2017, 5, 4, 1, 1, 1) result = {'matched_values': {'root'}} self.assertEqual(DeepSearch(obj, item, verbose_level=1), result) def test_keep_searching_after_obj_match(self): class AlwaysEqual: def __init__(self, recurse=True): if recurse: self.some_attr = AlwaysEqual(recurse=False) def __eq__(self, other): return True obj = AlwaysEqual() item = AlwaysEqual() result = {'matched_values': {'root', 'root.some_attr'}} def test_search_inherited_attributes(self): class Parent(object): a = 1 class Child(Parent): b = 2 obj = Child() item = 1 result = {'matched_values': {'root.a'}} self.assertEqual(DeepSearch(obj, item, verbose_level=1), result) class GrepTestCase(unittest.TestCase): def test_grep_dict(self): obj = { "for life": "vegan", "ingredients": ["no meat", "no eggs", "no dairy", "somewhere"] } ds = obj | grep(item) self.assertEqual(ds, {'matched_values': {"root['ingredients'][3]"}}) deepdiff-3.3.0/tests/test_serialization.py000066400000000000000000000044711312554330700207240ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- """ To run only the search tests: python -m unittest tests.test_serialization Or to run all the tests: python -m unittest discover Or to run all the tests with coverage: coverage run --source deepdiff setup.py test Or using Nose: nosetests --with-coverage --cover-package=deepdiff To run a specific test, run this from the root of repo: python -m unittest tests.test_serialization.DeepDiffTextTestCase.test_same_objects or using nosetests: nosetests tests/test_serialization.py:DeepDiffTestCase.test_diff_when_hash_fails """ import unittest from deepdiff import DeepDiff import logging logging.disable(logging.CRITICAL) class DeepAdditionsTestCase(unittest.TestCase): """Tests for Additions and Subtractions.""" def test_serialization_text(self): t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, 3]}} t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": "world\n\n\nEnd"}} ddiff = DeepDiff(t1, t2) self.assertTrue("deepdiff.helper.RemapDict" in ddiff.json) def test_deserialization(self): t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, 3]}} t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": "world\n\n\nEnd"}} ddiff = DeepDiff(t1, t2) jsoned = ddiff.json ddiff2 = DeepDiff.from_json(jsoned) self.assertEqual(ddiff, ddiff2) def test_serialization_tree(self): t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, 3]}} t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": "world\n\n\nEnd"}} ddiff = DeepDiff(t1, t2, view='tree') jsoned = ddiff.json self.assertTrue("world" in jsoned) def test_deserialization_tree(self): t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, 3]}} t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": "world\n\n\nEnd"}} ddiff = DeepDiff(t1, t2, view='tree') jsoned = ddiff.json ddiff2 = DeepDiff.from_json(jsoned) self.assertTrue('type_changes' in ddiff2) def test_deleting_serialization_cache(self): t1 = {1: 1} t2 = {1: 2} ddiff = DeepDiff(t1, t2) self.assertFalse(hasattr(ddiff, '_json')) ddiff.json self.assertTrue(hasattr(ddiff, '_json')) del ddiff.json self.assertFalse(hasattr(ddiff, '_json'))