././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1640125399.8034217 pytest-remotedata-0.3.3/0000755000175100001710000000000000000000000015770 5ustar00runnerdocker00000000000000././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1640125399.8034217 pytest-remotedata-0.3.3/.github/0000755000175100001710000000000000000000000017330 5ustar00runnerdocker00000000000000././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1640125399.8034217 pytest-remotedata-0.3.3/.github/workflows/0000755000175100001710000000000000000000000021365 5ustar00runnerdocker00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/.github/workflows/publish.yml0000644000175100001710000000232700000000000023562 0ustar00runnerdocker00000000000000name: Release on: pull_request: push: tags: - '*' jobs: build-n-publish: name: Build and publish Python 🐍 distributions 📦 to PyPI runs-on: ubuntu-latest if: ((github.event_name == 'push' && startsWith(github.ref, 'refs/tags')) || contains(github.event.pull_request.labels.*.name, 'Build wheels')) steps: - uses: actions/checkout@v2 with: fetch-depth: 0 - uses: actions/setup-python@v2 with: python-version: 3.8 - name: Install python-build and twine run: python -m pip install pip build "twine>=3.3" -U - name: Build package run: python -m build --sdist --wheel . - name: List result run: ls -l dist - name: Check long_description run: python -m twine check --strict dist/* - name: Test package run: | cd .. python -m venv testenv testenv/bin/pip install pytest pytest-remotedata/dist/*.whl testenv/bin/pytest pytest-remotedata/tests --remote-data - name: Publish distribution 📦 to PyPI if: startsWith(github.ref, 'refs/tags') uses: pypa/gh-action-pypi-publish@release/v1 with: user: __token__ password: ${{ secrets.pypi_password }} ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/.github/workflows/python-tests.yml0000644000175100001710000000322200000000000024570 0ustar00runnerdocker00000000000000name: Run unit tests on: pull_request: push: branches: [ main ] tags: workflow_dispatch: schedule: # Run every Sunday at 03:53 UTC - cron: 53 3 * * 0 jobs: tests: runs-on: ${{ matrix.os }} strategy: fail-fast: false matrix: include: - os: ubuntu-latest python-version: 3.6 toxenv: py36-test-pytest46 - os: windows-latest python-version: 3.6 toxenv: py36-test-pytest50 - os: macos-latest python-version: 3.7 toxenv: py37-test-pytest51 - os: ubuntu-latest python-version: 3.7 toxenv: py37-test-pytest52 - os: windows-latest python-version: 3.8 toxenv: py38-test-pytest53 - os: ubuntu-latest python-version: 3.8 toxenv: py38-test-pytest60 - os: ubuntu-latest python-version: 3.9 toxenv: py39-test-pytest61 - os: macos-latest python-version: 3.8 toxenv: py38-test-pytestdev steps: - uses: actions/checkout@v2 with: fetch-depth: 0 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v2 with: python-version: ${{ matrix.python-version }} - name: Install Tox run: python -m pip install tox - name: Run Tox run: tox -v -e ${{ matrix.toxenv }} # - name: Slack Notification # uses: 8398a7/action-slack@v3 # with: # status: ${{ job.status }} # env: # SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }} # if: always() # TODO: cron ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/.gitignore0000644000175100001710000000116000000000000017756 0ustar00runnerdocker00000000000000# Compiled files *.py[cod] *.a *.o *.so *.pyd __pycache__ # Ignore .c files by default to avoid including generated code. If you want to # add a non-generated .c extension, use `git add -f filename.c`. *.c # Other generated files MANIFEST # Sphinx _build _generated docs/api docs/generated # Packages/installer info *.egg *.egg-info dist build eggs .eggs parts bin var sdist develop-eggs .installed.cfg distribute-*.tar.gz # Other .cache .tox .*.swp .*.swo *~ .project .pydevproject .settings .coverage cover htmlcov .pytest_cache # Mac OSX .DS_Store # PyCharm .idea pytest_remotedata/version.py pip-wheel-metadata/ ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/CHANGES.rst0000644000175100001710000000213600000000000017574 0ustar00runnerdocker000000000000000.3.3 (2021-12-21) ================== - Various infrastructure and packaging updates. If you have trouble installing an older version, try upgrading to this one. 0.3.2 (2019-07-20) ================== - Fixed compatibility with pytest 4.2 and later. [#38, #40] 0.3.1 (2018-10-23) ================== - Fix warnings that occur with pytest 3.7 and later. [#34] 0.3.0 (2018-05-29) ================== - Allow ``remote_data`` argument to be passed without ``source`` keyword. [#27] - Add ``source='github'`` option. Also fix IP lookup error for ``source='astropy'`` option. [#28] 0.2.1 (2018-04-20) ================== - Fix packaging error: tests are no longer included in package distribution. [#24] 0.2.0 (2017-11-10) ================== - Do not accept invalid values passed to ``--remote-data`` argument. [#15] - Update list of valid hosts URLs recognized as astropy data sources. [#13] - Remove test dependency on astropy core. [#14] - Allow strict remote data checking to be configurable. This introduces a dependency on ``pytest>=3.1``. [#18] 0.1 (2017-10-10) ================ - Alpha release. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/LICENSE.rst0000644000175100001710000000273000000000000017606 0ustar00runnerdocker00000000000000Copyright (c) 2011-2017, Astropy Developers All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the Astropy Team nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/MANIFEST.in0000644000175100001710000000020300000000000017521 0ustar00runnerdocker00000000000000include LICENSE.rst include README.rst include CHANGES.rst include setup.cfg recursive-include tests * global-exclude *.pyc *.o ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1640125399.8034217 pytest-remotedata-0.3.3/PKG-INFO0000644000175100001710000001202000000000000017060 0ustar00runnerdocker00000000000000Metadata-Version: 2.1 Name: pytest-remotedata Version: 0.3.3 Summary: Pytest plugin for controlling remote data access. Home-page: https://github.com/astropy/pytest-remotedata Author: The Astropy Developers Author-email: astropy.team@gmail.com License: BSD Keywords: remote,data,pytest,py.test Platform: UNKNOWN Classifier: Development Status :: 3 - Alpha Classifier: Framework :: Pytest Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3 :: Only Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Topic :: Software Development :: Testing Classifier: Topic :: Utilities Requires-Python: >=3.6 Description-Content-Type: text/x-rst License-File: LICENSE.rst ================= pytest-remotedata ================= .. image:: https://github.com/astropy/pytest-remotedata/workflows/Run%20unit%20tests/badge.svg :target: https://github.com/astropy/pytest-remotedata/actions :alt: CI Status This package provides a plugin for the `pytest`_ framework that allows developers to control unit tests that require access to data from the internet. It was originally part of the `astropy`_ core package, but has been moved to a separate package in order to be of more general use. .. _pytest: https://pytest.org/en/latest/ .. _astropy: https://astropy.org/ Motivation ---------- Many software packages provide features that require access to data from the internet. These features need to be tested, but unit tests that access the internet can dominate the overall runtime of a test suite. The ``pytest-remotedata`` plugin allows developers to indicate which unit tests require access to the internet, and to control when and whether such tests should execute as part of any given run of the test suite. Installation ------------ The ``pytest-remotedata`` plugin can be installed using ``pip``:: $ pip install pytest-remotedata It is also possible to install the latest development version from the source repository:: $ git clone https://github.com/astropy/pytest-remotedata $ cd pytest-remotedata $ pip install . In either case, the plugin will automatically be registered for use with ``pytest``. Usage ----- Installing this plugin makes two decorators available for use with ``pytest``: * ``remote_data`` for marking tests that require data from the internet * ``internet_off`` for marking tests that should run only when internet access is disabled These decorators can be used to mark test functions, methods, and classes using ``@pytest.mark``. For example, consider the following test function that requires access to data from the internet: .. code-block:: python import pytest from urllib.request import urlopen @pytest.mark.remote_data def test_remote_data(): urlopen('https://astropy.org') Marking the ``test_remote_data`` function with ``@pytest.mark.remote_data`` indicates to ``pytest`` that this test should be run only when access to remote data sources is explicitly requested. When this plugin is installed, the ``--remote-data`` command line option is added to the ``pytest`` command line interface. The default behavior is to skip tests that are marked with ``remote_data``. If the ``--remote-data`` option is not provided to the ``pytest`` command, or if ``--remote-data=none`` is provided, all tests that are marked with ``remote_data`` will be skipped. All tests that are marked with ``internet_off`` will be executed. Sometimes it is useful to check that certain tests do not unexpectedly access the internet. Strict remote data access checking can be enabled by setting ``remote_data_strict = true`` in the tested package's ``setup.cfg`` file. If this option is enabled, any test that attempts to access the network but is not marked with ``@pytest.mark.remote_data`` will fail. Providing either the ``--remote-data`` option, or ``--remote-data=any`` to the ``pytest`` command line interface will cause all tests that are marked with ``remote-data`` to execute. Any tests that are marked with ``internet_off`` will be skipped. Running the tests with ``--remote-data=astropy`` will cause only tests that receive remote data from Astropy data sources to be run. Tests with any other data sources will be skipped. This is indicated in the test code by marking test functions with ``@pytest.mark.remote_data(source='astropy')``. In the future, we intend to support a configurable way to indicate specific remote data sources in addition to ``astropy``. Development Status ------------------ Questions, bug reports, and feature requests can be submitted on `github`_. .. _github: https://github.com/astropy/pytest-remotedata License ------- This plugin is licensed under a 3-clause BSD style license - see the ``LICENSE.rst`` file. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/README.rst0000644000175100001710000000774000000000000017467 0ustar00runnerdocker00000000000000================= pytest-remotedata ================= .. image:: https://github.com/astropy/pytest-remotedata/workflows/Run%20unit%20tests/badge.svg :target: https://github.com/astropy/pytest-remotedata/actions :alt: CI Status This package provides a plugin for the `pytest`_ framework that allows developers to control unit tests that require access to data from the internet. It was originally part of the `astropy`_ core package, but has been moved to a separate package in order to be of more general use. .. _pytest: https://pytest.org/en/latest/ .. _astropy: https://astropy.org/ Motivation ---------- Many software packages provide features that require access to data from the internet. These features need to be tested, but unit tests that access the internet can dominate the overall runtime of a test suite. The ``pytest-remotedata`` plugin allows developers to indicate which unit tests require access to the internet, and to control when and whether such tests should execute as part of any given run of the test suite. Installation ------------ The ``pytest-remotedata`` plugin can be installed using ``pip``:: $ pip install pytest-remotedata It is also possible to install the latest development version from the source repository:: $ git clone https://github.com/astropy/pytest-remotedata $ cd pytest-remotedata $ pip install . In either case, the plugin will automatically be registered for use with ``pytest``. Usage ----- Installing this plugin makes two decorators available for use with ``pytest``: * ``remote_data`` for marking tests that require data from the internet * ``internet_off`` for marking tests that should run only when internet access is disabled These decorators can be used to mark test functions, methods, and classes using ``@pytest.mark``. For example, consider the following test function that requires access to data from the internet: .. code-block:: python import pytest from urllib.request import urlopen @pytest.mark.remote_data def test_remote_data(): urlopen('https://astropy.org') Marking the ``test_remote_data`` function with ``@pytest.mark.remote_data`` indicates to ``pytest`` that this test should be run only when access to remote data sources is explicitly requested. When this plugin is installed, the ``--remote-data`` command line option is added to the ``pytest`` command line interface. The default behavior is to skip tests that are marked with ``remote_data``. If the ``--remote-data`` option is not provided to the ``pytest`` command, or if ``--remote-data=none`` is provided, all tests that are marked with ``remote_data`` will be skipped. All tests that are marked with ``internet_off`` will be executed. Sometimes it is useful to check that certain tests do not unexpectedly access the internet. Strict remote data access checking can be enabled by setting ``remote_data_strict = true`` in the tested package's ``setup.cfg`` file. If this option is enabled, any test that attempts to access the network but is not marked with ``@pytest.mark.remote_data`` will fail. Providing either the ``--remote-data`` option, or ``--remote-data=any`` to the ``pytest`` command line interface will cause all tests that are marked with ``remote-data`` to execute. Any tests that are marked with ``internet_off`` will be skipped. Running the tests with ``--remote-data=astropy`` will cause only tests that receive remote data from Astropy data sources to be run. Tests with any other data sources will be skipped. This is indicated in the test code by marking test functions with ``@pytest.mark.remote_data(source='astropy')``. In the future, we intend to support a configurable way to indicate specific remote data sources in addition to ``astropy``. Development Status ------------------ Questions, bug reports, and feature requests can be submitted on `github`_. .. _github: https://github.com/astropy/pytest-remotedata License ------- This plugin is licensed under a 3-clause BSD style license - see the ``LICENSE.rst`` file. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/pyproject.toml0000644000175100001710000000021400000000000020701 0ustar00runnerdocker00000000000000[build-system] requires = ["setuptools>=30.3.0", "setuptools_scm", "wheel"] build-backend = 'setuptools.build_meta' ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1640125399.8034217 pytest-remotedata-0.3.3/pytest_remotedata/0000755000175100001710000000000000000000000021525 5ustar00runnerdocker00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/pytest_remotedata/__init__.py0000644000175100001710000000016500000000000023640 0ustar00runnerdocker00000000000000# Licensed under a 3-clause BSD style license - see LICENSE.rst from .version import version as __version__ # noqa ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/pytest_remotedata/disable_internet.py0000644000175100001710000001415400000000000025417 0ustar00runnerdocker00000000000000# Licensed under a 3-clause BSD style license - see LICENSE.rst import contextlib import socket import urllib.request # save original socket method for restoration # These are global so that re-calling the turn_off_internet function doesn't # overwrite them again socket_original = socket.socket socket_create_connection = socket.create_connection socket_bind = socket.socket.bind socket_connect = socket.socket.connect GITHUB_HOSTS = ['www.github.io'] ASTROPY_HOSTS = (['data.astropy.org', 'astropy.stsci.edu', 'www.astropy.org'] + GITHUB_HOSTS) INTERNET_OFF = False # urllib2 uses a global variable to cache its default "opener" for opening # connections for various protocols; we store it off here so we can restore to # the default after re-enabling internet use _orig_opener = None def _resolve_host_ips(hostname, port=80): """ Obtain all the IPs, including aliases, in a way that supports IPv4/v6 dual stack. """ try: ips = set([s[-1][0] for s in socket.getaddrinfo(hostname, port)]) except socket.gaierror: ips = set([]) ips.add(hostname) return ips # ::1 is apparently another valid name for localhost? # it is returned by getaddrinfo when that function is given localhost def check_internet_off(original_function, allow_astropy_data=False, allow_github_data=False): """ Wraps ``original_function``, which in most cases is assumed to be a `socket.socket` method, to raise an `IOError` for any operations on non-local AF_INET sockets. Allowing Astropy data also automatically allow GitHub data. """ def new_function(*args, **kwargs): if isinstance(args[0], socket.socket): if not args[0].family in (socket.AF_INET, socket.AF_INET6): # Should be fine in all but some very obscure cases # More to the point, we don't want to affect AF_UNIX # sockets. return original_function(*args, **kwargs) host = args[1][0] addr_arg = 1 valid_hosts = set(['localhost', '127.0.0.1', '::1']) else: # The only other function this is used to wrap currently is # socket.create_connection, which should be passed a 2-tuple, but # we'll check just in case if not (isinstance(args[0], tuple) and len(args[0]) == 2): return original_function(*args, **kwargs) host = args[0][0] addr_arg = 0 valid_hosts = set(['localhost', '127.0.0.1']) # Astropy + GitHub data if allow_astropy_data: for valid_host in ASTROPY_HOSTS: valid_hosts = valid_hosts.union(_resolve_host_ips(valid_host)) # Only GitHub data elif allow_github_data: for valid_host in GITHUB_HOSTS: valid_hosts = valid_hosts.union(_resolve_host_ips(valid_host)) hostname = socket.gethostname() fqdn = socket.getfqdn() if host in (hostname, fqdn): host = 'localhost' host_ips = set([host]) new_addr = (host, args[addr_arg][1]) args = args[:addr_arg] + (new_addr,) + args[addr_arg + 1:] else: host_ips = _resolve_host_ips(host) if len(host_ips & valid_hosts) > 0: # Any overlap is acceptable return original_function(*args, **kwargs) else: raise IOError("An attempt was made to connect to the internet " "by a test that was not marked `remote_data`. The " "requested host was: {0}".format(host)) return new_function def turn_off_internet(verbose=False, allow_astropy_data=False, allow_github_data=False): """ Disable internet access via python by preventing connections from being created using the socket module. Presumably this could be worked around by using some other means of accessing the internet, but all default python modules (urllib, requests, etc.) use socket [citation needed]. """ global INTERNET_OFF global _orig_opener if INTERNET_OFF: return INTERNET_OFF = True __tracebackhide__ = True if verbose: print("Internet access disabled") # Update urllib2 to force it not to use any proxies # Must use {} here (the default of None will kick off an automatic search # for proxies) _orig_opener = urllib.request.build_opener() no_proxy_handler = urllib.request.ProxyHandler({}) opener = urllib.request.build_opener(no_proxy_handler) urllib.request.install_opener(opener) socket.create_connection = check_internet_off( socket_create_connection, allow_astropy_data=allow_astropy_data, allow_github_data=allow_github_data) socket.socket.bind = check_internet_off( socket_bind, allow_astropy_data=allow_astropy_data, allow_github_data=allow_github_data) socket.socket.connect = check_internet_off( socket_connect, allow_astropy_data=allow_astropy_data, allow_github_data=allow_github_data) return socket def turn_on_internet(verbose=False): """ Restore internet access. Not used, but kept in case it is needed. """ global INTERNET_OFF global _orig_opener if not INTERNET_OFF: return INTERNET_OFF = False if verbose: print("Internet access enabled") urllib.request.install_opener(_orig_opener) socket.create_connection = socket_create_connection socket.socket.bind = socket_bind socket.socket.connect = socket_connect return socket @contextlib.contextmanager def no_internet(verbose=False): """Context manager to temporarily disable internet access (if not already disabled). If it was already disabled before entering the context manager (i.e. `turn_off_internet` was called previously) then this is a no-op and leaves internet access disabled until a manual call to `turn_on_internet`. """ already_disabled = INTERNET_OFF turn_off_internet(verbose=verbose) try: yield finally: if not already_disabled: turn_on_internet(verbose=verbose) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/pytest_remotedata/plugin.py0000644000175100001710000000742200000000000023402 0ustar00runnerdocker00000000000000# Licensed under a 3-clause BSD style license - see LICENSE.rst """ This plugin provides command-line options for controlling whether and how tests make use of online data. """ import pytest from packaging.version import Version from .disable_internet import turn_off_internet, turn_on_internet def pytest_addoption(parser): # The following means that if --remote-data is not specified, the default # is 'none', but if it is specified without arguments (--remote-data), it # defaults to '--remote-data=any'. parser.addoption( "--remote-data", nargs="?", const='any', default='none', help="run tests with online data") parser.addini( 'remote_data_strict', "If 'True', tests will fail if they attempt to access the internet " "but are not explicitly marked with 'remote_data'", type="bool", default=False) def pytest_configure(config): config.getini('markers').append( 'remote_data: Apply to tests that require data from remote servers') config.getini('markers').append( 'internet_off: Apply to tests that should only run when ' 'network access is deactivated') strict_check = bool(config.getini('remote_data_strict')) remote_data = config.getoption('remote_data') options = ['astropy', 'any', 'github', 'none'] if remote_data not in options: raise pytest.UsageError( "'{}' is not a valid source for remote data, " "use one of '{}'".format(remote_data, "', '".join(options))) # Monkeypatch to deny access to remote resources unless explicitly told # otherwise if strict_check: if remote_data == 'github': # GitHub only turn_off_internet( verbose=config.option.verbose, allow_github_data=True) elif remote_data != 'any': # Astropy + GitHub or nothing turn_off_internet( verbose=config.option.verbose, allow_astropy_data=(remote_data == 'astropy')) def pytest_unconfigure(): """ Cleanup post-testing """ # restore internet connectivity (only lost if remote_data=False and # turn_off_internet previously called) # this is harmless / does nothing if socket connections were never disabled turn_on_internet() def pytest_runtest_setup(item): if Version(pytest.__version__) < Version("3.6"): remote_data = item.get_marker('remote_data') internet_off = item.get_marker('internet_off') else: remote_data = item.get_closest_marker('remote_data') internet_off = item.get_closest_marker('internet_off') remote_data_config = item.config.getvalue("remote_data") if remote_data is not None and internet_off is not None: raise ValueError("remote_data and internet_off are not compatible") if remote_data is not None: if len(remote_data.args) > 0: source = remote_data.args[0] else: source = remote_data.kwargs.get('source', 'any') if source not in ('astropy', 'any', 'github'): raise ValueError("source should be 'astropy', 'any', or 'github'") if remote_data_config == 'none': pytest.skip("need --remote-data option to run") elif remote_data_config == 'github': if source in ('any', 'astropy'): pytest.skip("need --remote-data or " "--remote-data=astropy option to run") elif remote_data_config == 'astropy': # When --remote-data=astropy is given, skip tests simply # marked as --remote-data if source == 'any': pytest.skip("need --remote-data option to run") if internet_off is not None: if remote_data_config != 'none': pytest.skip("run this test only when network access is disabled") ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125399.0 pytest-remotedata-0.3.3/pytest_remotedata/version.py0000644000175100001710000000021600000000000023563 0ustar00runnerdocker00000000000000# coding: utf-8 # file generated by setuptools_scm # don't change, don't track in version control version = '0.3.3' version_tuple = (0, 3, 3) ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1640125399.8034217 pytest-remotedata-0.3.3/pytest_remotedata.egg-info/0000755000175100001710000000000000000000000023217 5ustar00runnerdocker00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125399.0 pytest-remotedata-0.3.3/pytest_remotedata.egg-info/PKG-INFO0000644000175100001710000001202000000000000024307 0ustar00runnerdocker00000000000000Metadata-Version: 2.1 Name: pytest-remotedata Version: 0.3.3 Summary: Pytest plugin for controlling remote data access. Home-page: https://github.com/astropy/pytest-remotedata Author: The Astropy Developers Author-email: astropy.team@gmail.com License: BSD Keywords: remote,data,pytest,py.test Platform: UNKNOWN Classifier: Development Status :: 3 - Alpha Classifier: Framework :: Pytest Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3 :: Only Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Topic :: Software Development :: Testing Classifier: Topic :: Utilities Requires-Python: >=3.6 Description-Content-Type: text/x-rst License-File: LICENSE.rst ================= pytest-remotedata ================= .. image:: https://github.com/astropy/pytest-remotedata/workflows/Run%20unit%20tests/badge.svg :target: https://github.com/astropy/pytest-remotedata/actions :alt: CI Status This package provides a plugin for the `pytest`_ framework that allows developers to control unit tests that require access to data from the internet. It was originally part of the `astropy`_ core package, but has been moved to a separate package in order to be of more general use. .. _pytest: https://pytest.org/en/latest/ .. _astropy: https://astropy.org/ Motivation ---------- Many software packages provide features that require access to data from the internet. These features need to be tested, but unit tests that access the internet can dominate the overall runtime of a test suite. The ``pytest-remotedata`` plugin allows developers to indicate which unit tests require access to the internet, and to control when and whether such tests should execute as part of any given run of the test suite. Installation ------------ The ``pytest-remotedata`` plugin can be installed using ``pip``:: $ pip install pytest-remotedata It is also possible to install the latest development version from the source repository:: $ git clone https://github.com/astropy/pytest-remotedata $ cd pytest-remotedata $ pip install . In either case, the plugin will automatically be registered for use with ``pytest``. Usage ----- Installing this plugin makes two decorators available for use with ``pytest``: * ``remote_data`` for marking tests that require data from the internet * ``internet_off`` for marking tests that should run only when internet access is disabled These decorators can be used to mark test functions, methods, and classes using ``@pytest.mark``. For example, consider the following test function that requires access to data from the internet: .. code-block:: python import pytest from urllib.request import urlopen @pytest.mark.remote_data def test_remote_data(): urlopen('https://astropy.org') Marking the ``test_remote_data`` function with ``@pytest.mark.remote_data`` indicates to ``pytest`` that this test should be run only when access to remote data sources is explicitly requested. When this plugin is installed, the ``--remote-data`` command line option is added to the ``pytest`` command line interface. The default behavior is to skip tests that are marked with ``remote_data``. If the ``--remote-data`` option is not provided to the ``pytest`` command, or if ``--remote-data=none`` is provided, all tests that are marked with ``remote_data`` will be skipped. All tests that are marked with ``internet_off`` will be executed. Sometimes it is useful to check that certain tests do not unexpectedly access the internet. Strict remote data access checking can be enabled by setting ``remote_data_strict = true`` in the tested package's ``setup.cfg`` file. If this option is enabled, any test that attempts to access the network but is not marked with ``@pytest.mark.remote_data`` will fail. Providing either the ``--remote-data`` option, or ``--remote-data=any`` to the ``pytest`` command line interface will cause all tests that are marked with ``remote-data`` to execute. Any tests that are marked with ``internet_off`` will be skipped. Running the tests with ``--remote-data=astropy`` will cause only tests that receive remote data from Astropy data sources to be run. Tests with any other data sources will be skipped. This is indicated in the test code by marking test functions with ``@pytest.mark.remote_data(source='astropy')``. In the future, we intend to support a configurable way to indicate specific remote data sources in addition to ``astropy``. Development Status ------------------ Questions, bug reports, and feature requests can be submitted on `github`_. .. _github: https://github.com/astropy/pytest-remotedata License ------- This plugin is licensed under a 3-clause BSD style license - see the ``LICENSE.rst`` file. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125399.0 pytest-remotedata-0.3.3/pytest_remotedata.egg-info/SOURCES.txt0000644000175100001710000000127300000000000025106 0ustar00runnerdocker00000000000000.gitignore CHANGES.rst LICENSE.rst MANIFEST.in README.rst pyproject.toml setup.cfg setup.py tox.ini .github/workflows/publish.yml .github/workflows/python-tests.yml pytest_remotedata/__init__.py pytest_remotedata/disable_internet.py pytest_remotedata/plugin.py pytest_remotedata/version.py pytest_remotedata.egg-info/PKG-INFO pytest_remotedata.egg-info/SOURCES.txt pytest_remotedata.egg-info/dependency_links.txt pytest_remotedata.egg-info/entry_points.txt pytest_remotedata.egg-info/not-zip-safe pytest_remotedata.egg-info/requires.txt pytest_remotedata.egg-info/top_level.txt tests/__init__.py tests/conftest.py tests/test_skip_remote_data.py tests/test_socketblocker.py tests/test_strict_check.py././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125399.0 pytest-remotedata-0.3.3/pytest_remotedata.egg-info/dependency_links.txt0000644000175100001710000000000100000000000027265 0ustar00runnerdocker00000000000000 ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125399.0 pytest-remotedata-0.3.3/pytest_remotedata.egg-info/entry_points.txt0000644000175100001710000000007100000000000026513 0ustar00runnerdocker00000000000000[pytest11] pytest_remotedata = pytest_remotedata.plugin ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125399.0 pytest-remotedata-0.3.3/pytest_remotedata.egg-info/not-zip-safe0000644000175100001710000000000100000000000025445 0ustar00runnerdocker00000000000000 ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125399.0 pytest-remotedata-0.3.3/pytest_remotedata.egg-info/requires.txt0000644000175100001710000000002600000000000025615 0ustar00runnerdocker00000000000000pytest>=4.6 packaging ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125399.0 pytest-remotedata-0.3.3/pytest_remotedata.egg-info/top_level.txt0000644000175100001710000000002200000000000025743 0ustar00runnerdocker00000000000000pytest_remotedata ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1640125399.8074217 pytest-remotedata-0.3.3/setup.cfg0000644000175100001710000000237200000000000017615 0ustar00runnerdocker00000000000000[metadata] name = pytest-remotedata url = https://github.com/astropy/pytest-remotedata author = The Astropy Developers author_email = astropy.team@gmail.com classifiers = Development Status :: 3 - Alpha Framework :: Pytest Intended Audience :: Developers License :: OSI Approved :: BSD License Operating System :: OS Independent Programming Language :: Python Programming Language :: Python :: 3 Programming Language :: Python :: 3 :: Only Programming Language :: Python :: 3.6 Programming Language :: Python :: 3.7 Programming Language :: Python :: 3.8 Programming Language :: Python :: Implementation :: CPython Topic :: Software Development :: Testing Topic :: Utilities license = BSD description = Pytest plugin for controlling remote data access. long_description = file: README.rst long_description_content_type = text/x-rst keywords = remote, data, pytest, py.test [options] zip_safe = False packages = find: python_requires = >=3.6 setup_requires = setuptools_scm install_requires = pytest>=4.6 packaging [options.entry_points] pytest11 = pytest_remotedata = pytest_remotedata.plugin [options.packages.find] exclude = tests [tool:pytest] minversion = 4.6 testpaths = tests remote_data_strict = true [egg_info] tag_build = tag_date = 0 ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/setup.py0000755000175100001710000000022400000000000017503 0ustar00runnerdocker00000000000000#!/usr/bin/env python import os from setuptools import setup setup(use_scm_version={'write_to': os.path.join('pytest_remotedata', 'version.py')}) ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1640125399.8034217 pytest-remotedata-0.3.3/tests/0000755000175100001710000000000000000000000017132 5ustar00runnerdocker00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/tests/__init__.py0000644000175100001710000000015500000000000021244 0ustar00runnerdocker00000000000000from __future__ import (absolute_import, division, print_function, unicode_literals) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/tests/conftest.py0000644000175100001710000000014500000000000021331 0ustar00runnerdocker00000000000000# This enables the meta-testing of this plugin via the testdir fixture pytest_plugins = ['pytester'] ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/tests/test_skip_remote_data.py0000644000175100001710000000727400000000000024067 0ustar00runnerdocker00000000000000# Licensed under a 3-clause BSD style license - see LICENSE.rst # this test doesn't actually use any online data, it should just be skipped # by run_tests because it has the remote_data decorator. from contextlib import closing from urllib.error import URLError from urllib.request import urlopen import pytest ASTROPY_DATA_URL = "http://data.astropy.org/" GITHUB_DATA_URL = "http://astropy.github.io/" EXTERNAL_URL = "http://www.google.com" TIMEOUT = 10 def download_file(remote_url): with closing(urlopen(remote_url, timeout=TIMEOUT)) as remote: remote.read() @pytest.mark.remote_data def test_skip_remote_data(pytestconfig): # astropy.test() has remote_data=none or remote_data=astropy but we still # got here somehow, so fail with a helpful message if pytestconfig.getoption('remote_data') == 'none': pytest.fail('@remote_data was not skipped with remote_data=none') elif pytestconfig.getoption('remote_data') == 'astropy': pytest.fail('@remote_data was not skipped with remote_data=astropy') elif pytestconfig.getoption('remote_data') == 'github': pytest.fail('@remote_data was not skipped with remote_data=github') # Test Astropy URL download_file(ASTROPY_DATA_URL + 'galactic_center/gc_2mass_k.fits') # Test GitHub URL download_file(GITHUB_DATA_URL) # Test unrelated URL download_file(EXTERNAL_URL) @pytest.mark.remote_data(source='astropy') def test_skip_remote_data_astropy(pytestconfig): # astropy.test() has remote_data=none but we still got here somehow, # so fail with a helpful message if pytestconfig.getoption('remote_data') in ('none', 'github'): pytest.fail('@remote_data was not skipped with remote_data=none' 'or remote_data=github') # Test Astropy URL download_file(ASTROPY_DATA_URL + 'galactic_center/gc_2mass_k.fits') # Test non-Astropy URL if pytestconfig.getoption('remote_data') == 'astropy': with pytest.raises(Exception) as exc: download_file(EXTERNAL_URL) assert "An attempt was made to connect to the internet" in str(exc.value) # noqa else: # remote_data=any download_file(EXTERNAL_URL) @pytest.mark.remote_data(source='github') def test_skip_remote_data_github(pytestconfig): if pytestconfig.getoption('remote_data') == 'none': pytest.fail('@remote_data was not skipped with remote_data=none') # Test GitHub URL download_file(GITHUB_DATA_URL) # Test non-GitHub URL if pytestconfig.getoption('remote_data') == 'github': with pytest.raises(Exception) as exc: download_file(ASTROPY_DATA_URL) assert "An attempt was made to connect to the internet" in str(exc.value) else: # remote_data=any or remote_data=astropy download_file(ASTROPY_DATA_URL) @pytest.mark.internet_off def test_internet_off_decorator(pytestconfig): # This test should only run when internet access has been disabled if pytestconfig.getoption('remote_data') != 'none': pytest.fail('@internet_off test ran when remote_data!=none') def test_block_internet_connection(pytestconfig): if pytestconfig.getoption('remote_data') == 'none': with pytest.raises(URLError): download_file(EXTERNAL_URL) elif pytestconfig.getoption('remote_data') == 'astropy': with pytest.raises(URLError): download_file(EXTERNAL_URL) elif pytestconfig.getoption('remote_data') == 'github': with pytest.raises(URLError): download_file(EXTERNAL_URL) else: download_file(EXTERNAL_URL) @pytest.mark.internet_off def test_block_internet_connection_internet_off(): with pytest.raises(URLError): download_file(EXTERNAL_URL) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/tests/test_socketblocker.py0000644000175100001710000000447200000000000023404 0ustar00runnerdocker00000000000000# Licensed under a 3-clause BSD style license - see LICENSE.rst import sys import time from http.server import HTTPServer, SimpleHTTPRequestHandler from threading import Thread from urllib.request import urlopen import pytest from pytest_remotedata.disable_internet import no_internet def test_outgoing_fails(): with pytest.raises(IOError): with no_internet(): urlopen('http://www.python.org') class StoppableHTTPServer(HTTPServer, object): def __init__(self, *args): super(StoppableHTTPServer, self).__init__(*args) self.stop = False def handle_request(self): self.stop = True super(StoppableHTTPServer, self).handle_request() def serve_forever(self): """ Serve until stop set, which will happen if any request is handled """ while not self.stop: self.handle_request() @pytest.mark.parametrize(('localhost'), ('localhost', '127.0.0.1')) def test_localconnect_succeeds(localhost): """ Ensure that connections to localhost are allowed, since these are genuinely not remotedata. """ # port "0" means find open port # see http://stackoverflow.com/questions/1365265/on-localhost-how-to-pick-a-free-port-number httpd = StoppableHTTPServer(('localhost', 0), SimpleHTTPRequestHandler) port = httpd.socket.getsockname()[1] server = Thread(target=httpd.serve_forever) server.setDaemon(True) server.start() time.sleep(0.1) urlopen('http://{localhost:s}:{port:d}'.format(localhost=localhost, port=port)).close() PY3_4 = sys.version_info[:2] >= (3, 4) # Used for the below test--inline functions aren't pickleable # by multiprocessing? def _square(x): return x ** 2 @pytest.mark.skipif('not PY3_4 or sys.platform == "win32" or sys.platform.startswith("gnu0")') def test_multiprocessing_forkserver(): """ Test that using multiprocessing with forkserver works. Perhaps a simpler more direct test would be to just open some local sockets and pass something through them. Regression test for https://github.com/astropy/astropy/pull/3713 """ import multiprocessing ctx = multiprocessing.get_context('forkserver') pool = ctx.Pool(1) result = pool.map(_square, [1, 2, 3, 4, 5]) pool.close() pool.join() assert result == [1, 4, 9, 16, 25] ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/tests/test_strict_check.py0000644000175100001710000000331000000000000023205 0ustar00runnerdocker00000000000000# Licensed under a 3-clause BSD style license - see LICENSE.rst import os import pytest PYFILE_CONTENTS = """ import pytest from urllib.request import urlopen def test_config_setting(pytestconfig): assert pytestconfig.getini('remote_data_strict') == {} {} def test_internet_access(): urlopen('http://astropy.org') """ def _write_config_file(testdir, entry): config = testdir.tmpdir.join('setup.cfg') config.write(""" [tool:pytest] {} """.format(entry)) # Just a sanity check to make sure we actually write the config file assert os.path.exists(str(testdir.tmpdir.join('setup.cfg'))) def test_local_config(pytestconfig): assert pytestconfig.getini('remote_data_strict') == True def test_default_behavior(testdir): _write_config_file(testdir, '') testdir.makepyfile(PYFILE_CONTENTS.format('False', '')) result = testdir.runpytest_subprocess() result.assert_outcomes(passed=2) def test_strict_behavior(testdir): _write_config_file(testdir, 'remote_data_strict = true') testdir.makepyfile(PYFILE_CONTENTS.format('True', '')) result = testdir.runpytest_subprocess() result.assert_outcomes(passed=1, failed=1) @pytest.mark.parametrize('source', ['none', 'any']) def test_strict_with_decorator(testdir, source): _write_config_file(testdir, 'remote_data_strict = true') decorator = '@pytest.mark.remote_data' testdir.makepyfile(PYFILE_CONTENTS.format('True', decorator)) clarg = '--remote-data=' + source result = testdir.runpytest_subprocess(clarg) if source == 'none': outcomes = dict(passed=1, skipped=1) else: outcomes = dict(passed=2) result.assert_outcomes(**outcomes) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1640125385.0 pytest-remotedata-0.3.3/tox.ini0000644000175100001710000000173500000000000017311 0ustar00runnerdocker00000000000000[tox] envlist = py{36,37,38,39}-test{,-devdeps} codestyle requires = setuptools >= 30.3.0 pip >= 19.3.1 isolated_build = true [testenv] changedir = .tmp/{envname} description = run tests deps = pytest46: pytest==4.6.* pytest50: pytest==5.0.* pytest51: pytest==5.1.* pytest52: pytest==5.2.* pytest53: pytest==5.3.* pytest60: pytest==6.0.* pytest61: pytest==6.1.* pytestdev: git+https://github.com/pytest-dev/pytest#egg=pytest commands = pip freeze pytest {toxinidir}/tests {posargs} pytest {toxinidir}/tests --remote-data {posargs} pytest {toxinidir}/tests --remote-data=none {posargs} pytest {toxinidir}/tests --remote-data=github {posargs} pytest {toxinidir}/tests --remote-data=astropy {posargs} pytest {toxinidir}/tests --remote-data=any {posargs} [testenv:codestyle] changedir = skip_install = true description = check code style, e.g. with flake8 deps = flake8 commands = flake8 pytest_remotedata --count