././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1701082269.592042 pytest-arraydiff-0.6.1/0000755000175100001770000000000014531072236014423 5ustar00runnerdocker././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1701082269.584042 pytest-arraydiff-0.6.1/.github/0000755000175100001770000000000014531072236015763 5ustar00runnerdocker././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1701082269.588042 pytest-arraydiff-0.6.1/.github/workflows/0000755000175100001770000000000014531072236020020 5ustar00runnerdocker././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/.github/workflows/ci_workflows.yml0000644000175100001770000000216614531072215023255 0ustar00runnerdockername: CI on: pull_request: push: schedule: # Run every Sunday at 06:53 UTC - cron: 53 6 * * 0 concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true jobs: tests: uses: OpenAstronomy/github-actions-workflows/.github/workflows/tox.yml@v1 with: envs: | - linux: codestyle - linux: py37-test-pytestoldest - macos: py37-test-pytest50 - windows: py38-test-pytest52 - linux: py38-test-pytest53 - macos: py39-test-pytest60 - windows: py39-test-pytest61 - linux: py310-test-pytest62 - macos: py310-test-pytest70 - windows: py310-test-pytest71 - linux: py311-test-pytest72 - macos: py311-test-pytest73 - windows: py312-test-pytest74 - linux: py312-test-devdeps publish: needs: tests uses: OpenAstronomy/github-actions-workflows/.github/workflows/publish_pure_python.yml@v1 with: test_extras: test test_command: pytest $GITHUB_WORKSPACE/tests; pytest --arraydiff $GITHUB_WORKSPACE/tests secrets: pypi_token: ${{ secrets.pypi_password }} ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/.gitignore0000644000175100001770000000116714531072215016415 0ustar00runnerdocker# Compiled files *.py[cod] *.a *.o *.so *.pyd __pycache__ # Ignore .c files by default to avoid including generated code. If you want to # add a non-generated .c extension, use `git add -f filename.c`. *.c # Other generated files MANIFEST # Sphinx _build _generated docs/api docs/generated # Packages/installer info *.egg *.egg-info dist build eggs .eggs parts bin var sdist develop-eggs .installed.cfg distribute-*.tar.gz # Other .cache .tox .*.swp .*.swo *~ .project .pydevproject .settings .coverage cover htmlcov .pytest_cache # Env .venv venv .env # Mac OSX .DS_Store # PyCharm .idea */version.py pip-wheel-metadata/ ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/CHANGES.md0000644000175100001770000000161114531072215016011 0ustar00runnerdocker0.6.1 (2023-11-27) ------------------ - Fix broken ``single_reference=True`` usage. [#43] 0.6 (2023-11-15) ---------------- - Add ability to compare to Pandas DataFrames and store them as HDF5 files [#23] - Fix ``array_compare`` so that the ``atol`` parameter is correctly used with FITS files. [#33] - Test inside ``pytest_runtest_call`` hook. [#36] 0.5 (2022-01-12) ---------------- - Removed `astropy` as required dependency. [#31] - Formally register `array_compare` as marker. 0.4 (2021-12-31) ---------------- - Minimum Python version is now 3.7. [#30] - Various infrastructure updates. 0.3 (2018-12-05) ---------------- - Fixed compatibility with pytest 4+. [#15] 0.2 (2018-01-29) ---------------- - Fix compatibility with recent versions of Astropy and Numpy. [#8, #10] - Add back support for returning HDUs from tests. [#5] 0.1 (2016-11-26) ---------------- - Initial version ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/LICENSE0000644000175100001770000000265014531072215015430 0ustar00runnerdockerCopyright (c) 2016, Thomas P. Robitaille All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. This package was adapted from pytest-mpl, which is released under a BSD license and can be found here: https://github.com/astrofrog/pytest-mpl ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/MANIFEST.in0000644000175100001770000000025314531072215016156 0ustar00runnerdockerinclude LICENSE include README.rst include CHANGES.md include tox.ini include pyproject.toml include setup.cfg include setup.py recursive-include tests *.py *.fits *.txt ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1701082269.592042 pytest-arraydiff-0.6.1/PKG-INFO0000644000175100001770000002053514531072236015525 0ustar00runnerdockerMetadata-Version: 2.1 Name: pytest-arraydiff Version: 0.6.1 Summary: pytest plugin to help with comparing array output from tests Home-page: https://github.com/astropy/pytest-arraydiff Author: The Astropy Developers Author-email: astropy.team@gmail.com License: BSD Classifier: Development Status :: 4 - Beta Classifier: Framework :: Pytest Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3 :: Only Classifier: Programming Language :: Python :: 3.7 Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: 3.9 Classifier: Programming Language :: Python :: 3.10 Classifier: Programming Language :: Python :: 3.11 Classifier: Programming Language :: Python :: 3.12 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Topic :: Software Development :: Testing Classifier: Topic :: Utilities Requires-Python: >=3.7 Description-Content-Type: text/x-rst License-File: LICENSE Requires-Dist: pytest>=4.6 Requires-Dist: numpy Provides-Extra: test Requires-Dist: astropy; extra == "test" Requires-Dist: pandas; extra == "test" Requires-Dist: tables; extra == "test" .. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.5811772.svg :target: https://doi.org/10.5281/zenodo.5811772 :alt: 10.5281/zenodo.5811772 .. image:: https://github.com/astropy/pytest-arraydiff/workflows/CI/badge.svg :target: https://github.com/astropy/pytest-arraydiff/actions :alt: CI Status .. image:: https://img.shields.io/pypi/v/pytest-arraydiff.svg :target: https://pypi.org/project/pytest-arraydiff :alt: PyPI Status About ----- This is a `py.test `__ plugin to facilitate the generation and comparison of data arrays produced during tests, in particular in cases where the arrays are too large to conveniently hard-code them in the tests (e.g. ``np.testing.assert_allclose(x, [1, 2, 3])``). The basic idea is that you can write a test that generates a Numpy array (or other related objects depending on the format, e.g. pandas DataFrame). You can then either run the tests in a mode to **generate** reference files from the arrays, or you can run the tests in **comparison** mode, which will compare the results of the tests to the reference ones within some tolerance. At the moment, the supported file formats for the reference files are: - A plain text-based format (based on Numpy ``loadtxt`` output) - The FITS format (requires `astropy `__). With this format, tests can return either a Numpy array for a FITS HDU object. - A pandas HDF5 format using the pandas HDFStore For more information on how to write tests to do this, see the **Using** section below. Installing ---------- This plugin is compatible with Python 2.7, and 3.5 and later, and requires `pytest `__ and `numpy `__ to be installed. To install, you can do:: pip install pytest-arraydiff You can check that the plugin is registered with pytest by doing:: py.test --version which will show a list of plugins:: This is pytest version 2.7.1, imported from ... setuptools registered plugins: pytest-arraydiff-0.1 at ... Using ----- To use, you simply need to mark the function where you want to compare arrays using ``@pytest.mark.array_compare``, and make sure that the function returns a plain Numpy array:: python import pytest import numpy as np @pytest.mark.array_compare def test_succeeds(): return np.arange(3 * 5 * 4).reshape((3, 5, 4)) To generate the reference data files, run the tests with the ``--arraydiff-generate-path`` option with the name of the directory where the generated files should be placed:: py.test --arraydiff-generate-path=reference If the directory does not exist, it will be created. The directory will be interpreted as being relative to where you are running ``py.test``. Make sure you manually check the reference arrays to ensure they are correct. Once you are happy with the generated data files, you should move them to a sub-directory called ``reference`` relative to the test files (this name is configurable, see below). You can also generate the baseline arrays directly in the right directory. You can then run the tests simply with:: py.test --arraydiff and the tests will pass if the arrays are the same. If you omit the ``--arraydiff`` option, the tests will run but will only check that the code runs without checking the output arrays. Options ------- The ``@pytest.mark.array_compare`` marker take an argument to specify the format to use for the reference files: .. code:: python @pytest.mark.array_compare(file_format='text') def test_array(): ... The default file format can also be specified using the ``--arraydiff-default-format=`` flag when running ``py.test``, and ```` should be either ``fits`` or ``text``. The supported formats at this time are ``text`` and ``fits``, and contributions for other formats are welcome. The default format is ``text``. Additional arguments are the relative and absolute tolerances for floating point values (which default to 1e-7 and 0, respectively): .. code:: python @pytest.mark.array_compare(rtol=20, atol=0.1) def test_array(): ... You can also pass keyword arguments to the writers using the ``write_kwargs``. For the ``text`` format, these arguments are passed to ``savetxt`` while for the ``fits`` format they are passed to Astropy's ``fits.writeto`` function. .. code:: python @pytest.mark.array_compare(file_format='fits', write_kwargs={'output_verify': 'silentfix'}) def test_array(): ... Other options include the name of the reference directory (which defaults to ``reference`` ) and the filename for the reference file (which defaults to the name of the test with a format-dependent extension). .. code:: python @pytest.mark.array_compare(reference_dir='baseline_arrays', filename='other_name.fits') def test_array(): ... The reference directory in the decorator above will be interpreted as being relative to the test file. Note that the baseline directory can also be a URL (which should start with ``http://`` or ``https://`` and end in a slash). Finally, you can also set a custom baseline directory globally when running tests by running ``py.test`` with:: py.test --arraydiff --arraydiff-reference-path=baseline_arrays This directory will be interpreted as being relative to where the tests are run. In addition, if both this option and the ``reference_dir`` option in the ``array_compare`` decorator are used, the one in the decorator takes precedence. Test failure example -------------------- If the arrays produced by the tests are correct, then the test will pass, but if they are not, the test will fail with a message similar to the following:: E AssertionError: E E a: /var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpbvjkzt_q/test_to_mask_rect-mode_subpixels-subpixels_18.txt E b: /var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpbvjkzt_q/reference-test_to_mask_rect-mode_subpixels-subpixels_18.txt E E Not equal to tolerance rtol=1e-07, atol=0 E E (mismatch 47.22222222222222%) E x: array([[ 0. , 0. , 0. , 0. , 0.404012, 0.55 , E 0.023765, 0. , 0. ], E [ 0. , 0. , 0. , 0.112037, 1.028704, 1.1 ,... E y: array([[ 0. , 0. , 0. , 0. , 0.367284, 0.5 , E 0.021605, 0. , 0. ], E [ 0. , 0. , 0. , 0.101852, 0.935185, 1. ,... The file paths included in the exception are then available for inspection. Running the tests for pytest-arraydiff -------------------------------------- If you are contributing some changes and want to run the tests, first install the latest version of the plugin then do:: cd tests py.test --arraydiff The reason for having to install the plugin first is to ensure that the plugin is correctly loaded as part of the test suite. ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/README.rst0000644000175100001770000001601614531072215016113 0ustar00runnerdocker.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.5811772.svg :target: https://doi.org/10.5281/zenodo.5811772 :alt: 10.5281/zenodo.5811772 .. image:: https://github.com/astropy/pytest-arraydiff/workflows/CI/badge.svg :target: https://github.com/astropy/pytest-arraydiff/actions :alt: CI Status .. image:: https://img.shields.io/pypi/v/pytest-arraydiff.svg :target: https://pypi.org/project/pytest-arraydiff :alt: PyPI Status About ----- This is a `py.test `__ plugin to facilitate the generation and comparison of data arrays produced during tests, in particular in cases where the arrays are too large to conveniently hard-code them in the tests (e.g. ``np.testing.assert_allclose(x, [1, 2, 3])``). The basic idea is that you can write a test that generates a Numpy array (or other related objects depending on the format, e.g. pandas DataFrame). You can then either run the tests in a mode to **generate** reference files from the arrays, or you can run the tests in **comparison** mode, which will compare the results of the tests to the reference ones within some tolerance. At the moment, the supported file formats for the reference files are: - A plain text-based format (based on Numpy ``loadtxt`` output) - The FITS format (requires `astropy `__). With this format, tests can return either a Numpy array for a FITS HDU object. - A pandas HDF5 format using the pandas HDFStore For more information on how to write tests to do this, see the **Using** section below. Installing ---------- This plugin is compatible with Python 2.7, and 3.5 and later, and requires `pytest `__ and `numpy `__ to be installed. To install, you can do:: pip install pytest-arraydiff You can check that the plugin is registered with pytest by doing:: py.test --version which will show a list of plugins:: This is pytest version 2.7.1, imported from ... setuptools registered plugins: pytest-arraydiff-0.1 at ... Using ----- To use, you simply need to mark the function where you want to compare arrays using ``@pytest.mark.array_compare``, and make sure that the function returns a plain Numpy array:: python import pytest import numpy as np @pytest.mark.array_compare def test_succeeds(): return np.arange(3 * 5 * 4).reshape((3, 5, 4)) To generate the reference data files, run the tests with the ``--arraydiff-generate-path`` option with the name of the directory where the generated files should be placed:: py.test --arraydiff-generate-path=reference If the directory does not exist, it will be created. The directory will be interpreted as being relative to where you are running ``py.test``. Make sure you manually check the reference arrays to ensure they are correct. Once you are happy with the generated data files, you should move them to a sub-directory called ``reference`` relative to the test files (this name is configurable, see below). You can also generate the baseline arrays directly in the right directory. You can then run the tests simply with:: py.test --arraydiff and the tests will pass if the arrays are the same. If you omit the ``--arraydiff`` option, the tests will run but will only check that the code runs without checking the output arrays. Options ------- The ``@pytest.mark.array_compare`` marker take an argument to specify the format to use for the reference files: .. code:: python @pytest.mark.array_compare(file_format='text') def test_array(): ... The default file format can also be specified using the ``--arraydiff-default-format=`` flag when running ``py.test``, and ```` should be either ``fits`` or ``text``. The supported formats at this time are ``text`` and ``fits``, and contributions for other formats are welcome. The default format is ``text``. Additional arguments are the relative and absolute tolerances for floating point values (which default to 1e-7 and 0, respectively): .. code:: python @pytest.mark.array_compare(rtol=20, atol=0.1) def test_array(): ... You can also pass keyword arguments to the writers using the ``write_kwargs``. For the ``text`` format, these arguments are passed to ``savetxt`` while for the ``fits`` format they are passed to Astropy's ``fits.writeto`` function. .. code:: python @pytest.mark.array_compare(file_format='fits', write_kwargs={'output_verify': 'silentfix'}) def test_array(): ... Other options include the name of the reference directory (which defaults to ``reference`` ) and the filename for the reference file (which defaults to the name of the test with a format-dependent extension). .. code:: python @pytest.mark.array_compare(reference_dir='baseline_arrays', filename='other_name.fits') def test_array(): ... The reference directory in the decorator above will be interpreted as being relative to the test file. Note that the baseline directory can also be a URL (which should start with ``http://`` or ``https://`` and end in a slash). Finally, you can also set a custom baseline directory globally when running tests by running ``py.test`` with:: py.test --arraydiff --arraydiff-reference-path=baseline_arrays This directory will be interpreted as being relative to where the tests are run. In addition, if both this option and the ``reference_dir`` option in the ``array_compare`` decorator are used, the one in the decorator takes precedence. Test failure example -------------------- If the arrays produced by the tests are correct, then the test will pass, but if they are not, the test will fail with a message similar to the following:: E AssertionError: E E a: /var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpbvjkzt_q/test_to_mask_rect-mode_subpixels-subpixels_18.txt E b: /var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpbvjkzt_q/reference-test_to_mask_rect-mode_subpixels-subpixels_18.txt E E Not equal to tolerance rtol=1e-07, atol=0 E E (mismatch 47.22222222222222%) E x: array([[ 0. , 0. , 0. , 0. , 0.404012, 0.55 , E 0.023765, 0. , 0. ], E [ 0. , 0. , 0. , 0.112037, 1.028704, 1.1 ,... E y: array([[ 0. , 0. , 0. , 0. , 0.367284, 0.5 , E 0.021605, 0. , 0. ], E [ 0. , 0. , 0. , 0.101852, 0.935185, 1. ,... The file paths included in the exception are then available for inspection. Running the tests for pytest-arraydiff -------------------------------------- If you are contributing some changes and want to run the tests, first install the latest version of the plugin then do:: cd tests py.test --arraydiff The reason for having to install the plugin first is to ensure that the plugin is correctly loaded as part of the test suite. ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/pyproject.toml0000644000175100001770000000021414531072215017331 0ustar00runnerdocker[build-system] requires = ["setuptools>=30.3.0", "setuptools_scm", "wheel"] build-backend = 'setuptools.build_meta' ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1701082269.588042 pytest-arraydiff-0.6.1/pytest_arraydiff/0000755000175100001770000000000014531072236020002 5ustar00runnerdocker././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/pytest_arraydiff/__init__.py0000755000175100001770000000016514531072215022115 0ustar00runnerdocker# Licensed under a 3-clause BSD style license - see LICENSE.rst from .version import version as __version__ # noqa ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/pytest_arraydiff/plugin.py0000755000175100001770000003203614531072215021656 0ustar00runnerdocker# Copyright (c) 2016, Thomas P. Robitaille # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are met: # # 1. Redistributions of source code must retain the above copyright notice, # this list of conditions and the following disclaimer. # # 2. Redistributions in binary form must reproduce the above copyright notice, # this list of conditions and the following disclaimer in the documentation # and/or other materials provided with the distribution. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE # LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR # CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF # SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS # INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN # CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) # ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE # POSSIBILITY OF SUCH DAMAGE. # # This package was derived from pytest-mpl, which is released under a BSD # license and can be found here: # # https://github.com/astrofrog/pytest-mpl import os import abc import shutil import tempfile import warnings from urllib.request import urlopen import pytest import numpy as np abstractstaticmethod = abc.abstractstaticmethod abstractclassmethod = abc.abstractclassmethod class BaseDiff(object, metaclass=abc.ABCMeta): @abstractstaticmethod def read(filename): """ Given a filename, return a data object. """ raise NotImplementedError() @abstractstaticmethod def write(filename, data, **kwargs): """ Given a filename and a data object (and optional keyword arguments), write the data to a file. """ raise NotImplementedError() @abstractclassmethod def compare(self, reference_file, test_file, atol=None, rtol=None): """ Given a reference and test filename, compare the data to the specified absolute (``atol``) and relative (``rtol``) tolerances. Should return two arguments: a boolean indicating whether the data are identical, and a string giving the full error message if not. """ raise NotImplementedError() class SimpleArrayDiff(BaseDiff): @classmethod def compare(cls, reference_file, test_file, atol=None, rtol=None): array_ref = cls.read(reference_file) array_new = cls.read(test_file) try: np.testing.assert_allclose(array_ref, array_new, atol=atol, rtol=rtol) except AssertionError as exc: message = "\n\na: {0}".format(test_file) + '\n' message += "b: {0}".format(reference_file) + '\n' message += exc.args[0] return False, message else: return True, "" class FITSDiff(BaseDiff): extension = 'fits' @staticmethod def read(filename): from astropy.io import fits return fits.getdata(filename) @staticmethod def write(filename, data, **kwargs): from astropy.io import fits if isinstance(data, np.ndarray): data = fits.PrimaryHDU(data) return data.writeto(filename, **kwargs) @classmethod def compare(cls, reference_file, test_file, atol=None, rtol=None): import astropy from astropy.io.fits.diff import FITSDiff from astropy.utils.introspection import minversion if minversion(astropy, '2.0'): diff = FITSDiff(reference_file, test_file, rtol=rtol, atol=atol) else: # `atol` is not supported prior to Astropy 2.0 diff = FITSDiff(reference_file, test_file, tolerance=rtol) return diff.identical, diff.report() class TextDiff(SimpleArrayDiff): extension = 'txt' @staticmethod def read(filename): return np.loadtxt(filename) @staticmethod def write(filename, data, **kwargs): fmt = kwargs.get('fmt', '%g') kwargs['fmt'] = fmt return np.savetxt(filename, data, **kwargs) class PDHDFDiff(BaseDiff): extension = 'h5' @staticmethod def read(filename): import pandas as pd return pd.read_hdf(filename) @staticmethod def write(filename, data, **kwargs): import pandas as pd # noqa: F401 key = os.path.basename(filename).replace('.h5', '') return data.to_hdf(filename, key=key, **kwargs) @classmethod def compare(cls, reference_file, test_file, atol=None, rtol=None): import pandas.testing as pdt import pandas as pd ref_data = pd.read_hdf(reference_file) test_data = pd.read_hdf(test_file) try: pdt.assert_frame_equal(ref_data, test_data) except AssertionError as exc: message = "\n\na: {0}".format(test_file) + '\n' message += "b: {0}".format(reference_file) + '\n' message += exc.args[0] return False, message else: return True, "" FORMATS = {} FORMATS['fits'] = FITSDiff FORMATS['text'] = TextDiff FORMATS['pd_hdf'] = PDHDFDiff def _download_file(url): u = urlopen(url) result_dir = tempfile.mkdtemp() filename = os.path.join(result_dir, 'downloaded') with open(filename, 'wb') as tmpfile: tmpfile.write(u.read()) return filename def pytest_addoption(parser): group = parser.getgroup("general") group.addoption('--arraydiff', action='store_true', help="Enable comparison of arrays to reference arrays stored in files") group.addoption('--arraydiff-generate-path', help="directory to generate reference files in, relative to location where py.test is run", action='store') group.addoption('--arraydiff-reference-path', help="directory containing reference files, relative to location where py.test is run", action='store') group.addoption('--arraydiff-default-format', help="Default format for the reference arrays (can be 'fits' or 'text' currently)") def pytest_configure(config): config.getini('markers').append( 'array_compare: for functions using array comparison') if config.getoption("--arraydiff") or config.getoption("--arraydiff-generate-path") is not None: reference_dir = config.getoption("--arraydiff-reference-path") generate_dir = config.getoption("--arraydiff-generate-path") if reference_dir is not None and generate_dir is not None: warnings.warn("Ignoring --arraydiff-reference-path since --arraydiff-generate-path is set") if reference_dir is not None: reference_dir = os.path.abspath(reference_dir) if generate_dir is not None: reference_dir = os.path.abspath(generate_dir) default_format = config.getoption("--arraydiff-default-format") or 'text' config.pluginmanager.register(ArrayComparison(config, reference_dir=reference_dir, generate_dir=generate_dir, default_format=default_format)) else: config.pluginmanager.register(ArrayInterceptor(config)) def generate_test_name(item): """ Generate a unique name for this test. """ if item.cls is not None: name = f"{item.module.__name__}.{item.cls.__name__}.{item.name}" else: name = f"{item.module.__name__}.{item.name}" return name def wrap_array_interceptor(plugin, item): """ Intercept and store arrays returned by test functions. """ # Only intercept array on marked array tests if item.get_closest_marker('array_compare') is not None: # Use the full test name as a key to ensure correct array is being retrieved test_name = generate_test_name(item) def array_interceptor(store, obj): def wrapper(*args, **kwargs): store.return_value[test_name] = obj(*args, **kwargs) return wrapper item.obj = array_interceptor(plugin, item.obj) class ArrayComparison(object): def __init__(self, config, reference_dir=None, generate_dir=None, default_format='text'): self.config = config self.reference_dir = reference_dir self.generate_dir = generate_dir self.default_format = default_format self.return_value = {} @pytest.hookimpl(hookwrapper=True) def pytest_runtest_call(self, item): compare = item.get_closest_marker('array_compare') if compare is None: yield return file_format = compare.kwargs.get('file_format', self.default_format) if file_format not in FORMATS: raise ValueError("Unknown format: {0}".format(file_format)) if 'extension' in compare.kwargs: extension = compare.kwargs['extension'] else: extension = FORMATS[file_format].extension atol = compare.kwargs.get('atol', 0.) rtol = compare.kwargs.get('rtol', 1e-7) single_reference = compare.kwargs.get('single_reference', False) write_kwargs = compare.kwargs.get('write_kwargs', {}) reference_dir = compare.kwargs.get('reference_dir', None) if reference_dir is None: if self.reference_dir is None: reference_dir = os.path.join(os.path.dirname(item.fspath.strpath), 'reference') else: reference_dir = self.reference_dir else: if not reference_dir.startswith(('http://', 'https://')): reference_dir = os.path.join(os.path.dirname(item.fspath.strpath), reference_dir) baseline_remote = reference_dir.startswith('http') # Run test and get array object wrap_array_interceptor(self, item) yield test_name = generate_test_name(item) if test_name not in self.return_value: # Test function did not complete successfully return array = self.return_value[test_name] # Find test name to use as plot name filename = compare.kwargs.get('filename', None) if filename is None: if single_reference: filename = item.originalname + '.' + extension else: filename = item.name + '.' + extension filename = filename.replace('[', '_').replace(']', '_') filename = filename.replace('_.' + extension, '.' + extension) # What we do now depends on whether we are generating the reference # files or simply running the test. if self.generate_dir is None: # Save the figure result_dir = tempfile.mkdtemp() test_array = os.path.abspath(os.path.join(result_dir, filename)) FORMATS[file_format].write(test_array, array, **write_kwargs) # Find path to baseline array if baseline_remote: baseline_file_ref = _download_file(reference_dir + filename) else: baseline_file_ref = os.path.abspath(os.path.join(os.path.dirname(item.fspath.strpath), reference_dir, filename)) if not os.path.exists(baseline_file_ref): raise Exception("""File not found for comparison test Generated file: \t{test} This is expected for new tests.""".format( test=test_array)) # setuptools may put the baseline arrays in non-accessible places, # copy to our tmpdir to be sure to keep them in case of failure baseline_file = os.path.abspath(os.path.join(result_dir, 'reference-' + filename)) shutil.copyfile(baseline_file_ref, baseline_file) identical, msg = FORMATS[file_format].compare(baseline_file, test_array, atol=atol, rtol=rtol) if identical: shutil.rmtree(result_dir) else: raise Exception(msg) else: if not os.path.exists(self.generate_dir): os.makedirs(self.generate_dir) FORMATS[file_format].write(os.path.abspath(os.path.join(self.generate_dir, filename)), array, **write_kwargs) pytest.skip("Skipping test, since generating data") class ArrayInterceptor: """ This is used in place of ArrayComparison when the array comparison option is not used, to make sure that we still intercept arrays returned by tests. """ def __init__(self, config): self.config = config self.return_value = {} @pytest.hookimpl(hookwrapper=True) def pytest_runtest_call(self, item): if item.get_closest_marker('array_compare') is not None: wrap_array_interceptor(self, item) yield return ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082269.0 pytest-arraydiff-0.6.1/pytest_arraydiff/version.py0000644000175100001770000000063314531072235022042 0ustar00runnerdocker# file generated by setuptools_scm # don't change, don't track in version control TYPE_CHECKING = False if TYPE_CHECKING: from typing import Tuple, Union VERSION_TUPLE = Tuple[Union[int, str], ...] else: VERSION_TUPLE = object version: str __version__: str __version_tuple__: VERSION_TUPLE version_tuple: VERSION_TUPLE __version__ = version = '0.6.1' __version_tuple__ = version_tuple = (0, 6, 1) ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1701082269.592042 pytest-arraydiff-0.6.1/pytest_arraydiff.egg-info/0000755000175100001770000000000014531072236021474 5ustar00runnerdocker././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082269.0 pytest-arraydiff-0.6.1/pytest_arraydiff.egg-info/PKG-INFO0000644000175100001770000002053514531072235022575 0ustar00runnerdockerMetadata-Version: 2.1 Name: pytest-arraydiff Version: 0.6.1 Summary: pytest plugin to help with comparing array output from tests Home-page: https://github.com/astropy/pytest-arraydiff Author: The Astropy Developers Author-email: astropy.team@gmail.com License: BSD Classifier: Development Status :: 4 - Beta Classifier: Framework :: Pytest Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3 :: Only Classifier: Programming Language :: Python :: 3.7 Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: 3.9 Classifier: Programming Language :: Python :: 3.10 Classifier: Programming Language :: Python :: 3.11 Classifier: Programming Language :: Python :: 3.12 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Topic :: Software Development :: Testing Classifier: Topic :: Utilities Requires-Python: >=3.7 Description-Content-Type: text/x-rst License-File: LICENSE Requires-Dist: pytest>=4.6 Requires-Dist: numpy Provides-Extra: test Requires-Dist: astropy; extra == "test" Requires-Dist: pandas; extra == "test" Requires-Dist: tables; extra == "test" .. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.5811772.svg :target: https://doi.org/10.5281/zenodo.5811772 :alt: 10.5281/zenodo.5811772 .. image:: https://github.com/astropy/pytest-arraydiff/workflows/CI/badge.svg :target: https://github.com/astropy/pytest-arraydiff/actions :alt: CI Status .. image:: https://img.shields.io/pypi/v/pytest-arraydiff.svg :target: https://pypi.org/project/pytest-arraydiff :alt: PyPI Status About ----- This is a `py.test `__ plugin to facilitate the generation and comparison of data arrays produced during tests, in particular in cases where the arrays are too large to conveniently hard-code them in the tests (e.g. ``np.testing.assert_allclose(x, [1, 2, 3])``). The basic idea is that you can write a test that generates a Numpy array (or other related objects depending on the format, e.g. pandas DataFrame). You can then either run the tests in a mode to **generate** reference files from the arrays, or you can run the tests in **comparison** mode, which will compare the results of the tests to the reference ones within some tolerance. At the moment, the supported file formats for the reference files are: - A plain text-based format (based on Numpy ``loadtxt`` output) - The FITS format (requires `astropy `__). With this format, tests can return either a Numpy array for a FITS HDU object. - A pandas HDF5 format using the pandas HDFStore For more information on how to write tests to do this, see the **Using** section below. Installing ---------- This plugin is compatible with Python 2.7, and 3.5 and later, and requires `pytest `__ and `numpy `__ to be installed. To install, you can do:: pip install pytest-arraydiff You can check that the plugin is registered with pytest by doing:: py.test --version which will show a list of plugins:: This is pytest version 2.7.1, imported from ... setuptools registered plugins: pytest-arraydiff-0.1 at ... Using ----- To use, you simply need to mark the function where you want to compare arrays using ``@pytest.mark.array_compare``, and make sure that the function returns a plain Numpy array:: python import pytest import numpy as np @pytest.mark.array_compare def test_succeeds(): return np.arange(3 * 5 * 4).reshape((3, 5, 4)) To generate the reference data files, run the tests with the ``--arraydiff-generate-path`` option with the name of the directory where the generated files should be placed:: py.test --arraydiff-generate-path=reference If the directory does not exist, it will be created. The directory will be interpreted as being relative to where you are running ``py.test``. Make sure you manually check the reference arrays to ensure they are correct. Once you are happy with the generated data files, you should move them to a sub-directory called ``reference`` relative to the test files (this name is configurable, see below). You can also generate the baseline arrays directly in the right directory. You can then run the tests simply with:: py.test --arraydiff and the tests will pass if the arrays are the same. If you omit the ``--arraydiff`` option, the tests will run but will only check that the code runs without checking the output arrays. Options ------- The ``@pytest.mark.array_compare`` marker take an argument to specify the format to use for the reference files: .. code:: python @pytest.mark.array_compare(file_format='text') def test_array(): ... The default file format can also be specified using the ``--arraydiff-default-format=`` flag when running ``py.test``, and ```` should be either ``fits`` or ``text``. The supported formats at this time are ``text`` and ``fits``, and contributions for other formats are welcome. The default format is ``text``. Additional arguments are the relative and absolute tolerances for floating point values (which default to 1e-7 and 0, respectively): .. code:: python @pytest.mark.array_compare(rtol=20, atol=0.1) def test_array(): ... You can also pass keyword arguments to the writers using the ``write_kwargs``. For the ``text`` format, these arguments are passed to ``savetxt`` while for the ``fits`` format they are passed to Astropy's ``fits.writeto`` function. .. code:: python @pytest.mark.array_compare(file_format='fits', write_kwargs={'output_verify': 'silentfix'}) def test_array(): ... Other options include the name of the reference directory (which defaults to ``reference`` ) and the filename for the reference file (which defaults to the name of the test with a format-dependent extension). .. code:: python @pytest.mark.array_compare(reference_dir='baseline_arrays', filename='other_name.fits') def test_array(): ... The reference directory in the decorator above will be interpreted as being relative to the test file. Note that the baseline directory can also be a URL (which should start with ``http://`` or ``https://`` and end in a slash). Finally, you can also set a custom baseline directory globally when running tests by running ``py.test`` with:: py.test --arraydiff --arraydiff-reference-path=baseline_arrays This directory will be interpreted as being relative to where the tests are run. In addition, if both this option and the ``reference_dir`` option in the ``array_compare`` decorator are used, the one in the decorator takes precedence. Test failure example -------------------- If the arrays produced by the tests are correct, then the test will pass, but if they are not, the test will fail with a message similar to the following:: E AssertionError: E E a: /var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpbvjkzt_q/test_to_mask_rect-mode_subpixels-subpixels_18.txt E b: /var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmpbvjkzt_q/reference-test_to_mask_rect-mode_subpixels-subpixels_18.txt E E Not equal to tolerance rtol=1e-07, atol=0 E E (mismatch 47.22222222222222%) E x: array([[ 0. , 0. , 0. , 0. , 0.404012, 0.55 , E 0.023765, 0. , 0. ], E [ 0. , 0. , 0. , 0.112037, 1.028704, 1.1 ,... E y: array([[ 0. , 0. , 0. , 0. , 0.367284, 0.5 , E 0.021605, 0. , 0. ], E [ 0. , 0. , 0. , 0.101852, 0.935185, 1. ,... The file paths included in the exception are then available for inspection. Running the tests for pytest-arraydiff -------------------------------------- If you are contributing some changes and want to run the tests, first install the latest version of the plugin then do:: cd tests py.test --arraydiff The reason for having to install the plugin first is to ensure that the plugin is correctly loaded as part of the test suite. ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082269.0 pytest-arraydiff-0.6.1/pytest_arraydiff.egg-info/SOURCES.txt0000644000175100001770000000162714531072235023365 0ustar00runnerdocker.gitignore CHANGES.md LICENSE MANIFEST.in README.rst pyproject.toml setup.cfg setup.py tox.ini .github/workflows/ci_workflows.yml pytest_arraydiff/__init__.py pytest_arraydiff/plugin.py pytest_arraydiff/version.py pytest_arraydiff.egg-info/PKG-INFO pytest_arraydiff.egg-info/SOURCES.txt pytest_arraydiff.egg-info/dependency_links.txt pytest_arraydiff.egg-info/entry_points.txt pytest_arraydiff.egg-info/not-zip-safe pytest_arraydiff.egg-info/requires.txt pytest_arraydiff.egg-info/top_level.txt tests/test_pytest_arraydiff.py tests/baseline/test_absolute_tolerance.fits tests/baseline/test_relative_tolerance.fits tests/baseline/test_single_reference.fits tests/baseline/test_succeeds_class.fits tests/baseline/test_succeeds_func_default.txt tests/baseline/test_succeeds_func_fits.fits tests/baseline/test_succeeds_func_fits_hdu.fits tests/baseline/test_succeeds_func_pdhdf.h5 tests/baseline/test_succeeds_func_text.txt././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082269.0 pytest-arraydiff-0.6.1/pytest_arraydiff.egg-info/dependency_links.txt0000644000175100001770000000000114531072235025541 0ustar00runnerdocker ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082269.0 pytest-arraydiff-0.6.1/pytest_arraydiff.egg-info/entry_points.txt0000644000175100001770000000006614531072235024773 0ustar00runnerdocker[pytest11] pytest_arraydiff = pytest_arraydiff.plugin ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082269.0 pytest-arraydiff-0.6.1/pytest_arraydiff.egg-info/not-zip-safe0000644000175100001770000000000114531072235023721 0ustar00runnerdocker ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082269.0 pytest-arraydiff-0.6.1/pytest_arraydiff.egg-info/requires.txt0000644000175100001770000000006014531072235024067 0ustar00runnerdockerpytest>=4.6 numpy [test] astropy pandas tables ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082269.0 pytest-arraydiff-0.6.1/pytest_arraydiff.egg-info/top_level.txt0000644000175100001770000000002114531072235024216 0ustar00runnerdockerpytest_arraydiff ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1701082269.592042 pytest-arraydiff-0.6.1/setup.cfg0000644000175100001770000000277714531072236016261 0ustar00runnerdocker[metadata] name = pytest-arraydiff url = https://github.com/astropy/pytest-arraydiff author = The Astropy Developers author_email = astropy.team@gmail.com classifiers = Development Status :: 4 - Beta Framework :: Pytest Intended Audience :: Developers License :: OSI Approved :: BSD License Operating System :: OS Independent Programming Language :: Python Programming Language :: Python :: 3 Programming Language :: Python :: 3 :: Only Programming Language :: Python :: 3.7 Programming Language :: Python :: 3.8 Programming Language :: Python :: 3.9 Programming Language :: Python :: 3.10 Programming Language :: Python :: 3.11 Programming Language :: Python :: 3.12 Programming Language :: Python :: Implementation :: CPython Topic :: Software Development :: Testing Topic :: Utilities license = BSD description = pytest plugin to help with comparing array output from tests long_description = file: README.rst long_description_content_type = text/x-rst [options] zip_safe = False packages = find: python_requires = >=3.7 setup_requires = setuptools_scm install_requires = pytest>=4.6 numpy [options.extras_require] test = astropy pandas tables [options.entry_points] pytest11 = pytest_arraydiff = pytest_arraydiff.plugin [tool:pytest] minversion = 4.6 testpaths = tests xfail_strict = true markers = array_compare: for functions using array comparison filterwarnings = error ignore:distutils Version classes are deprecated [flake8] max-line-length = 150 [egg_info] tag_build = tag_date = 0 ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/setup.py0000755000175100001770000000022314531072215016132 0ustar00runnerdocker#!/usr/bin/env python import os from setuptools import setup setup(use_scm_version={'write_to': os.path.join('pytest_arraydiff', 'version.py')}) ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1701082269.588042 pytest-arraydiff-0.6.1/tests/0000755000175100001770000000000014531072236015565 5ustar00runnerdocker././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1701082269.592042 pytest-arraydiff-0.6.1/tests/baseline/0000755000175100001770000000000014531072236017347 5ustar00runnerdocker././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/tests/baseline/test_absolute_tolerance.fits0000644000175100001770000001320014531072215025140 0ustar00runnerdockerSIMPLE = T / conforms to FITS standard BITPIX = -64 / array data type NAXIS = 2 / number of array dimensions NAXIS1 = 4 NAXIS2 = 3 EXTEND = T END ?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/tests/baseline/test_relative_tolerance.fits0000644000175100001770000001320014531072215025135 0ustar00runnerdockerSIMPLE = T / conforms to FITS standard BITPIX = -64 / array data type NAXIS = 2 / number of array dimensions NAXIS1 = 4 NAXIS2 = 3 EXTEND = T END ?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/tests/baseline/test_single_reference.fits0000644000175100001770000001320014531072215024565 0ustar00runnerdockerSIMPLE = T / conforms to FITS standard BITPIX = -64 / array data type NAXIS = 2 / number of array dimensions NAXIS1 = 4 NAXIS2 = 3 EXTEND = T END ?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š?ù™™™™™š././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/tests/baseline/test_succeeds_class.fits0000644000175100001770000001320014531072215024251 0ustar00runnerdockerSIMPLE = T / conforms to FITS standard BITPIX = 64 / array data type NAXIS = 3 / number of array dimensions NAXIS1 = 3 NAXIS2 = 4 NAXIS3 = 2 EXTEND = T END  ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/tests/baseline/test_succeeds_func_default.txt0000644000175100001770000000004314531072215025456 0ustar00runnerdocker0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/tests/baseline/test_succeeds_func_fits.fits0000644000175100001770000001320014531072215025124 0ustar00runnerdockerSIMPLE = T / conforms to FITS standard BITPIX = 64 / array data type NAXIS = 2 / number of array dimensions NAXIS1 = 5 NAXIS2 = 3 EXTEND = T END  ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/tests/baseline/test_succeeds_func_fits_hdu.fits0000644000175100001770000001320014531072215025764 0ustar00runnerdockerSIMPLE = T / conforms to FITS standard BITPIX = 64 / array data type NAXIS = 2 / number of array dimensions NAXIS1 = 5 NAXIS2 = 3 EXTEND = T END  ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/tests/baseline/test_succeeds_func_pdhdf.h50000644000175100001770000001557014531072215024627 0ustar00runnerdocker‰HDF  ÿÿÿÿÿÿÿÿxÿÿÿÿÿÿÿÿ`ˆ¨ àTREEÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÀHEAPX(Ètest_succeeds_func_pdhdf0ˆ¨ TITLE (CLASSGROUP (VERSION1.0 8PYTABLES_FORMAT_VERSION2.1èTREEÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ HEAPX8haxis0axis1block0_valuesblock0_items SNOD(H8P TITLE (CLASSGROUP (VERSION1.0 0 pandas_typeframe 0pandas_version0.15.2 0 encodingUTF-8 (errorsstrict 0 ndim@ 0axis0_varietyregularHð H ¢_ (CLASSARRAY (VERSION2.4 TITLESNODð ˆ( ¸test_data  test_data (FLAVORnumpy 8 transposed (kindstring (nameN.hP 0axis1_varietyregular Ð@Q  ¢_ (CLASSARRAY (VERSION2.4 TITLE (FLAVORnumpy 8 transposed (kindinteger (nameN.HX 0 nblocks@ €(@ñ  ¢_ (CLASSARRAY (VERSION2.4 TITLE (FLAVORnumpy 8 transposed(H 8block0_items_varietyregular È ‘ ¢_ (CLASSARRAY (VERSION2.4 TITLE (FLAVORnumpy 8 transposed (kindstring (nameN.././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/tests/baseline/test_succeeds_func_text.txt0000644000175100001770000000004314531072215025016 0ustar00runnerdocker0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/tests/test_pytest_arraydiff.py0000644000175100001770000001240014531072215022547 0ustar00runnerdockerimport os import subprocess import tempfile import pytest import numpy as np from packaging.version import Version NUMPY_LT_2_0 = Version(np.__version__) < Version("2.0.dev") reference_dir = 'baseline' @pytest.mark.array_compare(reference_dir=reference_dir) def test_succeeds_func_default(): return np.arange(3 * 5).reshape((3, 5)) @pytest.mark.array_compare(file_format='text', reference_dir=reference_dir) def test_succeeds_func_text(): return np.arange(3 * 5).reshape((3, 5)) @pytest.mark.skipif(not NUMPY_LT_2_0, reason="AttributeError: `np.unicode_` was removed in the NumPy 2.0 release. Use `np.str_` instead.") @pytest.mark.array_compare(file_format='pd_hdf', reference_dir=reference_dir) def test_succeeds_func_pdhdf(): pd = pytest.importorskip('pandas') return pd.DataFrame(data=np.arange(20, dtype='int64'), columns=['test_data']) @pytest.mark.array_compare(file_format='fits', reference_dir=reference_dir) def test_succeeds_func_fits(): return np.arange(3 * 5).reshape((3, 5)).astype(np.int64) @pytest.mark.array_compare(file_format='fits', reference_dir=reference_dir) def test_succeeds_func_fits_hdu(): from astropy.io import fits return fits.PrimaryHDU(np.arange(3 * 5).reshape((3, 5)).astype(np.int64)) class TestClass(object): @pytest.mark.array_compare(file_format='fits', reference_dir=reference_dir) def test_succeeds_class(self): return np.arange(2 * 4 * 3).reshape((2, 4, 3)).astype(np.int64) TEST_FAILING = """ import pytest import numpy as np from astropy.io import fits @pytest.mark.array_compare def test_fail(): return np.ones((3, 4)) """ def test_fails(): tmpdir = tempfile.mkdtemp() test_file = os.path.join(tmpdir, 'test.py') with open(test_file, 'w') as f: f.write(TEST_FAILING) # If we use --arraydiff, it should detect that the file is missing code = subprocess.call('pytest --arraydiff {0}'.format(test_file), shell=True) assert code != 0 # If we don't use --arraydiff option, the test should succeed code = subprocess.call('pytest {0}'.format(test_file), shell=True) assert code == 0 TEST_GENERATE = """ import pytest import numpy as np from astropy.io import fits @pytest.mark.array_compare(file_format='{file_format}') def test_gen(): return np.arange(6 * 5).reshape((6, 5)) """ @pytest.mark.parametrize('file_format', ('fits', 'text')) def test_generate(file_format): tmpdir = tempfile.mkdtemp() test_file = os.path.join(tmpdir, 'test.py') with open(test_file, 'w') as f: f.write(TEST_GENERATE.format(file_format=file_format)) gen_dir = os.path.join(tmpdir, 'spam', 'egg') # If we don't generate, the test will fail try: subprocess.check_output(['pytest', '--arraydiff', test_file], timeout=10) except subprocess.CalledProcessError as grepexc: assert b'File not found for comparison test' in grepexc.output # If we do generate, the test should succeed and a new file will appear code = subprocess.call(['pytest', '--arraydiff-generate-path={0}'.format(gen_dir), test_file], timeout=10) assert code == 0 assert os.path.exists(os.path.join(gen_dir, 'test_gen.' + ('fits' if file_format == 'fits' else 'txt'))) TEST_DEFAULT = """ import pytest import numpy as np from astropy.io import fits @pytest.mark.array_compare def test_default(): return np.arange(6 * 5).reshape((6, 5)) """ @pytest.mark.parametrize('file_format', ('fits', 'text')) def test_default_format(file_format): tmpdir = tempfile.mkdtemp() test_file = os.path.join(tmpdir, 'test.py') with open(test_file, 'w') as f: f.write(TEST_DEFAULT) gen_dir = os.path.join(tmpdir, 'spam', 'egg') # If we do generate, the test should succeed and a new file will appear code = subprocess.call('pytest -s --arraydiff-default-format={0}' ' --arraydiff-generate-path={1} {2}'.format(file_format, gen_dir, test_file), shell=True) assert code == 0 assert os.path.exists(os.path.join(gen_dir, 'test_default.' + ('fits' if file_format == 'fits' else 'txt'))) @pytest.mark.array_compare(reference_dir=reference_dir, rtol=0.5, file_format='fits') def test_relative_tolerance(): # Scale up the output values by 1.5 to ensure the large `rtol` value is # needed. (The comparison file contains all 1.6.) return np.ones((3, 4)) * 1.6 * 1.5 @pytest.mark.array_compare(reference_dir=reference_dir, atol=1.5, file_format='fits') def test_absolute_tolerance(): # Increase the output values by 1.4 to ensure the large `atol` value is # needed. (The comparison file contains all 1.6.) return np.ones((3, 4)) * 1.6 + 1.4 @pytest.mark.array_compare( reference_dir=reference_dir, atol=1.5, file_format='fits', single_reference=True) @pytest.mark.parametrize('spam', ('egg', 'bacon')) def test_single_reference(spam): return np.ones((3, 4)) * 1.6 + 1.4 class TestSingleReferenceClass: @pytest.mark.array_compare( reference_dir=reference_dir, atol=1.5, file_format='fits', single_reference=True) @pytest.mark.parametrize('spam', ('egg', 'bacon')) def test_single_reference(self, spam): return np.ones((3, 4)) * 1.6 + 1.4 def test_nofile(): pass ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701082253.0 pytest-arraydiff-0.6.1/tox.ini0000644000175100001770000000304414531072215015734 0ustar00runnerdocker[tox] envlist = py{37,38,39,310,311,312}-test{,-pytestoldest,-pytest50,-pytest52,-pytest53,-pytest60,-pytest61,-pytest62,-pytest70,-pytest71,-pytest72,-pytest73,-pytest74,-devdeps} codestyle requires = setuptools >= 30.3.0 pip >= 19.3.1 isolated_build = true [testenv] changedir = .tmp/{envname} setenv = devdeps: PIP_EXTRA_INDEX_URL = https://pypi.anaconda.org/astropy/simple https://pypi.anaconda.org/liberfa/simple https://pypi.anaconda.org/scientific-python-nightly-wheels/simple description = run tests deps = pytestoldest: pytest==4.6.0 pytest50: pytest==5.0.* pytest52: pytest==5.2.* pytest53: pytest==5.3.* pytest60: pytest==6.0.* pytest61: pytest==6.1.* pytest62: pytest==6.2.* pytest70: pytest==7.0.* pytest71: pytest==7.1.* pytest72: pytest==7.2.* pytest73: pytest==7.3.* pytest74: pytest==7.4.* devdeps: git+https://github.com/pytest-dev/pytest#egg=pytest devdeps: numpy>=0.0.dev0 devdeps: pandas>=0.0.dev0 devdeps: pyerfa>=0.0.dev0 devdeps: astropy>=0.0.dev0 extras = test commands = # Force numpy-dev after something in the stack downgrades it devdeps: python -m pip install --pre --upgrade --extra-index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple numpy pip freeze pytest {toxinidir}/tests {posargs} pytest {toxinidir}/tests --arraydiff {posargs} [testenv:codestyle] skip_install = true changedir = {toxinidir} description = check code style, e.g. with flake8 deps = flake8 commands = flake8 pytest_arraydiff --count