pax_global_header 0000666 0000000 0000000 00000000064 15142726461 0014522 g ustar 00root root 0000000 0000000 52 comment=e5f8f0172ba97490ac67f971d040e70686d862f6
tkem-cachetools-78b44a8/ 0000775 0000000 0000000 00000000000 15142726461 0015147 5 ustar 00root root 0000000 0000000 tkem-cachetools-78b44a8/.github/ 0000775 0000000 0000000 00000000000 15142726461 0016507 5 ustar 00root root 0000000 0000000 tkem-cachetools-78b44a8/.github/FUNDING.yml 0000664 0000000 0000000 00000000066 15142726461 0020326 0 ustar 00root root 0000000 0000000 github: [tkem]
custom: ["https://www.paypal.me/tkem"]
tkem-cachetools-78b44a8/.github/ISSUE_TEMPLATE/ 0000775 0000000 0000000 00000000000 15142726461 0020672 5 ustar 00root root 0000000 0000000 tkem-cachetools-78b44a8/.github/ISSUE_TEMPLATE/bug_report.md 0000664 0000000 0000000 00000001027 15142726461 0023364 0 ustar 00root root 0000000 0000000 ---
name: Bug report
about: Create a report to help us improve
title: ''
labels: bug
assignees: tkem
---
Before reporting a bug, please make sure you have the latest `cachetools` version installed:
```
pip install --upgrade cachetools
```
**Describe the bug**
A clear and concise description of what the bug is.
**Expected result**
A clear and concise description of what you expected to happen.
**Actual result**
A clear and concise description of what happened instead.
**Reproduction steps**
```python
import cachetools
```
tkem-cachetools-78b44a8/.github/ISSUE_TEMPLATE/config.yml 0000664 0000000 0000000 00000000034 15142726461 0022657 0 ustar 00root root 0000000 0000000 blank_issues_enabled: false
tkem-cachetools-78b44a8/.github/ISSUE_TEMPLATE/feature_request.md 0000664 0000000 0000000 00000000276 15142726461 0024424 0 ustar 00root root 0000000 0000000 ---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: enhancement
assignees: tkem
---
Sorry, but `cachetools` is not accepting feature requests at this time.
tkem-cachetools-78b44a8/.github/PULL_REQUEST_TEMPLATE.md 0000664 0000000 0000000 00000001740 15142726461 0022312 0 ustar 00root root 0000000 0000000 # Description
A clear and concise description of your changes.
## Motivation and Context
Why is this change required? What problem does it solve?
## Related Issue
Except for trivial changes like correcting typos, this project only
accepts pull requests related to open issues. If suggesting a new
feature or change, please discuss it in an issue first. If fixing a
bug, there should be an issue describing the bug, with steps to
reproduce.
Please link to the issue here: https://github.com/tkem/cachetools/issues/###
## Has this been tested and documented?
This project uses [tox](https://tox.wiki) for test automization.
Before submitting a pull request, run `tox` in the project's root and
make sure all tests are passing.
When adding new features or code not already covered by existing unit
tests, please also add some tests covering your changes under the
`tests` directory. For new features, please consider adding to the
existing documentation under the `docs` directory.
tkem-cachetools-78b44a8/.github/SECURITY.md 0000664 0000000 0000000 00000001407 15142726461 0020302 0 ustar 00root root 0000000 0000000 # Security Policy
## Supported Versions
Security updates are applied only to the latest release.
## Reporting a Vulnerability
If you have discovered a security vulnerability in this project, please report it privately. **Do not disclose it as a public issue.** This gives us time to work with you to fix the issue before public exposure, reducing the chance that the exploit will be used before a patch is released.
Please disclose it at [security advisory](https://github.com/tkem/cachetools/security/advisories/new).
This project is maintained by a single person on a best effort basis. As such, vulnerability reports will be investigated and fixed or disclosed as soon as possible, but there may be delays in response time due to the maintainer's other commitments.
tkem-cachetools-78b44a8/.github/copilot-instructions.md 0000664 0000000 0000000 00000012257 15142726461 0023253 0 ustar 00root root 0000000 0000000 # Copilot Instructions for cachetools
## Architecture Overview
**cachetools** provides extensible memoizing collections and decorators, including variants of Python's `@lru_cache`. All cache implementations live in a single ~730-line file (`src/cachetools/__init__.py`) with decorator helpers in separate modules.
### Core Design Pattern
- All caches inherit from `Cache` (a `MutableMapping` with `maxsize`, `currsize`, and `getsizeof`)
- Subclasses override `__setitem__`, `__getitem__`, `__delitem__`, and `popitem()` to implement eviction policies
- **Critical:** Subclasses use default parameter trick (e.g., `cache_setitem=Cache.__setitem__`) to call parent methods efficiently while avoiding recursion
### Cache Types & Eviction Policies
- `FIFOCache`: Evicts oldest inserted (uses `OrderedDict`)
- `LRUCache`: Evicts least recently used (uses `OrderedDict.move_to_end()`)
- `LFUCache`: Evicts least frequently used (doubly-linked list of frequency buckets)
- `RRCache`: Random eviction (maintains `__keys` list with `__index` dict for O(1) removal)
- `TTLCache`/`TLRUCache`: Time-based eviction with `_TimedCache` base (uses `_Timer` context manager to freeze time during operations)
### Decorator Architecture
- `@cached`: Function memoization via `src/cachetools/_cached.py` (separate wrappers for each combination of lock/condition/info, plus uncached variants when `cache=None`)
- `@cachedmethod`: Method memoization via `src/cachetools/_cachedmethod.py` (uses descriptor protocol with `__set_name__`/`__get__` to replace itself in instance `__dict__` with a per-instance `Wrapper`; `weakref.WeakKeyDictionary` only used for deprecated `@classmethod` pending sets)
- Both support custom `key` functions, `lock` objects, and `condition` variables for thread safety
- `info=True` adds `cache_info()` and `cache_clear()` methods (see `_CacheInfo` namedtuple)
## Critical Implementation Details
### Key Generation (`src/cachetools/keys.py`)
- `_HashedTuple` caches hash values to avoid recomputation on cache misses
- `methodkey(self, *args, **kwargs)` drops `self` from cache key (instance methods share cache)
- `typedkey` adds type information: `key += tuple(type(v) for v in args)`
### Thread Safety Pattern
Decorators use 3-tier locking strategy:
1. **Unlocked:** No synchronization (fastest, single-threaded)
2. **Locked:** Lock around cache access, compute outside lock to avoid holding during expensive operations
3. **Condition:** Lock + pending set + wait_for/notify_all (prevents thundering herd on cache misses)
### Time-Based Caches (`_TimedCache`)
- `_Timer` wrapper freezes time during multi-operation methods using `__enter__/__exit__`
- Prevents time-of-check-time-of-use bugs during iteration/expiration
- `expire()` method returns list of `(key, value)` pairs (allows logging/cleanup hooks)
## Developer Workflows
### Testing
```bash
pytest # Run all tests
pytest --cov=cachetools --cov-report term-missing # With coverage
tox # All environments
tox -e py # Just tests
tox -e flake8 # Linting
tox -e docs # Build docs
tox -e doctest # Doctest validation
```
### Code Style
- **Black** formatter (max line length: 80 via flake8)
- flake8 with `flake8-black`, `flake8-bugbear`, `flake8-import-order`
- Ignore: F401 (submodule shims), E501 (line length handled by black)
### Test Patterns
- `tests/__init__.py` defines `CacheTestMixin` with standard cache behavior tests
- Each cache type test inherits: `class LRUCacheTest(unittest.TestCase, CacheTestMixin)`
- Mixin provides 13 standard tests; cache-specific tests added per file
- Decorator tests use `CountedLock` and `CountedCondition` helpers to verify synchronization
## Project-Specific Conventions
### Adding New Cache Types
1. Inherit from `Cache` or `_TimedCache`
2. Override `__setitem__`, `__delitem__`, `popitem()` (and optionally `__getitem__`)
3. Use default parameters to call parent: `def __setitem__(self, key, value, cache_setitem=Cache.__setitem__)`
4. Handle `__missing__` edge case: check `if key in self` after parent call (see LRU/LFU `__getitem__`)
5. Add test class inheriting `CacheTestMixin` in `tests/`
### Module Structure
- All cache classes in `src/cachetools/__init__.py` (~730 lines)
- Decorator wrappers split: `_cached.py` (functions), `_cachedmethod.py` (methods/descriptors, ~410 lines)
- Functools-compatible wrappers in `src/cachetools/func.py` (`lru_cache`, `ttl_cache`, etc.)
- No external dependencies at runtime (pure Python 3.10+)
### Version Management
- Version in `src/cachetools/__init__.py` as `__version__`
- Extracted by `pyproject.toml`: `version = {attr = "cachetools.__version__"}`
- Docs extract version via `docs/conf.py` parsing
## Key Files
- `src/cachetools/__init__.py` - All cache implementations (~730 lines)
- `src/cachetools/keys.py` - Key generation with hash optimization
- `src/cachetools/_cached.py` - Function decorator variants
- `src/cachetools/_cachedmethod.py` - Method descriptor variants (~410 lines)
- `tests/__init__.py` - `CacheTestMixin` for standard cache tests
- `pyproject.toml` - Package metadata, build config, and flake8 config tkem-cachetools-78b44a8/.github/dependabot.yml 0000664 0000000 0000000 00000000167 15142726461 0021343 0 ustar 00root root 0000000 0000000 version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
tkem-cachetools-78b44a8/.github/workflows/ 0000775 0000000 0000000 00000000000 15142726461 0020544 5 ustar 00root root 0000000 0000000 tkem-cachetools-78b44a8/.github/workflows/ci.yml 0000664 0000000 0000000 00000001472 15142726461 0021666 0 ustar 00root root 0000000 0000000 name: CI
on: [push, pull_request, workflow_dispatch]
permissions:
contents: read
jobs:
main:
name: Python ${{ matrix.python }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python: ["3.10", "3.11", "3.12", "3.13", "3.13t", "3.14", "3.14t", "pypy3.10", "pypy3.11"]
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: ${{ matrix.python }}
allow-prereleases: true
- run: pip install coverage tox
- run: tox
- uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
with:
name: ${{ matrix.python }}
token: ${{ secrets.CODECOV_TOKEN }}
tkem-cachetools-78b44a8/.gitignore 0000664 0000000 0000000 00000000142 15142726461 0017134 0 ustar 00root root 0000000 0000000 *.egg-info
*.pyc
*.swp
.cache/
.coverage
.pytest_cache/
.tox/
MANIFEST
build/
dist/
docs/_build/
tkem-cachetools-78b44a8/.readthedocs.yaml 0000664 0000000 0000000 00000000207 15142726461 0020375 0 ustar 00root root 0000000 0000000 # Configure ReadTheDocs.
version: 2
build:
os: "ubuntu-22.04"
tools:
python: "3.11"
sphinx:
configuration: "docs/conf.py"
tkem-cachetools-78b44a8/CHANGELOG.rst 0000664 0000000 0000000 00000032310 15142726461 0017167 0 ustar 00root root 0000000 0000000 v7.0.1 (2026-02-10)
===================
- Various test improvements.
- Update Copilot Instructions.
v7.0.0 (2026-02-01)
===================
- Require Python 3.10 or later (breaking change).
- Drop support for passing ``info`` as fourth positional parameter to
``@cached`` (breaking change).
- Drop support for ``cache(self)`` returning ``None`` with
``@cachedmethod`` (breaking change).
- Convert the ``@cachedmethod`` wrappers to descriptors, deprecating
its use with class methods and instances that do not provide a
mutable ``__dict__`` attribute (potentially breaking change).
- Convert the previously undocumented ``@cachedmethod`` attributes
(``cache``, ``cache_lock``, etc.) to properties for instance
methods, providing official support and documentation (potentially
breaking change).
- Add an optional ``info`` parameter to the ``@cachedmethod``
decorator for reporting per-instance cache statistics. Note that
this requires the instance's ``__dict__`` attribute to be a mutable
mapping.
v6.2.6 (2026-01-27)
===================
- Improve typedkey performance.
- Minor documentation improvements.
- Minor testing improvements.
- Minor code readability improvements.
v6.2.5 (2026-01-25)
===================
- Improve documentation regarding ``@cachedmethod`` with ``lock``
parameter.
- Add test cases for cache stampede scenarios.
- Update CI environment.
v6.2.4 (2025-12-15)
===================
- Fix license information displayed on PyPI be using an updated
version of ``twine`` for uploading.
v6.2.3 (2025-12-12)
===================
- Improve documentation regarding ``@cachedmethod`` with ``condition``
parameter.
- Minor style and readability improvements.
- Modernize build environment.
- Update CI environment.
v6.2.2 (2025-11-13)
===================
- Minor improvements from GitHub Copilot code review.
- Improve documentation.
v6.2.1 (2025-10-12)
===================
- Add support for Python 3.14.
- Improve documentation.
- Update CI environment.
v6.2.0 (2025-08-25)
===================
- Improve general ``RRCache`` performance by storing cache keys in an
additional sequence container. Note that this will increase memory
consumption.
- Add more unit tests.
v6.1.0 (2025-06-16)
===================
- Improve ``LFUCache`` insertion performance by switching to an
implementation based on the `cacheing
`_ library.
- Update CI environment.
v6.0.0 (2025-05-23)
===================
- Require Python 3.9 or later (breaking change).
- Remove ``MRUCache`` and the ``@func.mru_cache`` decorator (breaking
change).
- Add an optional ``condition`` parameter to the ``@cached`` and
``@cachedmethod`` decorators, which, when used with a
``threading.Condition`` instance, should improve `cache stampede
`_ issues in massively
parallel environments. Note that this will inflict some performance
penalty, and therefore has to be enabled explicitly.
- Convert the ``cachetools.func`` decorators to use a
``threading.Condition`` instance to deal with `cache stampede
`_ issues. Note that
this *may* result in a noticable performance degradation, depending
on your actual use case.
- Deprecate support for ``cache(self)`` returning ``None`` to suppress
caching with the ``@cachedmethod`` decorator.
- Improve documentation.
- Update CI environment.
v5.5.2 (2025-02-20)
===================
- Reduce number of ``@cached`` lock/unlock operations.
- Improve documentation.
- Update CI environment.
v5.5.1 (2025-01-21)
===================
- Add documentation regarding caching of exceptions.
- Officially support Python 3.13.
- Update CI environment.
v5.5.0 (2024-08-18)
===================
- ``TTLCache.expire()`` returns iterable of expired ``(key, value)``
pairs.
- ``TLRUCache.expire()`` returns iterable of expired ``(key, value)``
pairs.
- Documentation improvements.
- Update CI environment.
v5.4.0 (2024-07-15)
===================
- Add the ``keys.typedmethodkey`` decorator.
- Deprecate ``MRUCache`` class.
- Deprecate ``@func.mru_cache`` decorator.
- Update CI environment.
v5.3.3 (2024-02-26)
===================
- Documentation improvements.
- Update CI environment.
v5.3.2 (2023-10-24)
===================
- Add support for Python 3.12.
- Various documentation improvements.
v5.3.1 (2023-05-27)
===================
- Depend on Python >= 3.7.
v5.3.0 (2023-01-22)
===================
- Add ``cache_info()`` function to ``@cached`` decorator.
v5.2.1 (2023-01-08)
===================
- Add support for Python 3.11.
- Correct version information in RTD documentation.
- ``badges/shields``: Change to GitHub workflow badge routes.
v5.2.0 (2022-05-29)
===================
- Add ``cachetools.keys.methodkey()``.
- Add ``cache_clear()`` function to decorators.
- Add ``src`` directory to ``sys.path`` for Sphinx autodoc.
- Modernize ``func`` wrappers.
v5.1.0 (2022-05-15)
===================
- Add cache decorator parameters as wrapper function attributes.
v5.0.0 (2021-12-21)
===================
- Require Python 3.7 or later (breaking change).
- Remove deprecated submodules (breaking change).
The ``cache``, ``fifo``, ``lfu``, ``lru``, ``mru``, ``rr`` and
``ttl`` submodules have been deleted. Therefore, statements like
``from cachetools.ttl import TTLCache``
will no longer work. Use
``from cachetools import TTLCache``
instead.
- Pass ``self`` to ``@cachedmethod`` key function (breaking change).
The ``key`` function passed to the ``@cachedmethod`` decorator is
now called as ``key(self, *args, **kwargs)``.
The default key function has been changed to ignore its first
argument, so this should only affect applications using custom key
functions with the ``@cachedmethod`` decorator.
- Change exact time of expiration in ``TTLCache`` (breaking change).
``TTLCache`` items now get expired if their expiration time is less
than *or equal to* ``timer()``. For applications using the default
``timer()``, this should be barely noticeable, but it may affect the
use of custom timers with larger tick intervals. Note that this
also implies that a ``TTLCache`` with ``ttl=0`` can no longer hold
any items, since they will expire immediately.
- Change ``Cache.__repr__()`` format (breaking change).
String representations of cache instances now use a more compact and
efficient format, e.g.
``LRUCache({1: 1, 2: 2}, maxsize=10, currsize=2)``
- Add TLRU cache implementation.
- Documentation improvements.
v4.2.4 (2021-09-30)
===================
- Add submodule shims for backward compatibility.
v4.2.3 (2021-09-29)
===================
- Add documentation and tests for using ``TTLCache`` with
``datetime``.
- Link to typeshed typing stubs.
- Flatten package file hierarchy.
v4.2.2 (2021-04-27)
===================
- Update build environment.
- Remove Python 2 remnants.
- Format code with Black.
v4.2.1 (2021-01-24)
===================
- Handle ``__missing__()`` not storing cache items.
- Clean up ``__missing__()`` example.
v4.2.0 (2020-12-10)
===================
- Add FIFO cache implementation.
- Add MRU cache implementation.
- Improve behavior of decorators in case of race conditions.
- Improve documentation regarding mutability of caches values and use
of key functions with decorators.
- Officially support Python 3.9.
v4.1.1 (2020-06-28)
===================
- Improve ``popitem()`` exception context handling.
- Replace ``float('inf')`` with ``math.inf``.
- Improve "envkey" documentation example.
v4.1.0 (2020-04-08)
===================
- Support ``user_function`` with ``cachetools.func`` decorators
(Python 3.8 compatibility).
- Support ``cache_parameters()`` with ``cachetools.func`` decorators
(Python 3.9 compatibility).
v4.0.0 (2019-12-15)
===================
- Require Python 3.5 or later.
v3.1.1 (2019-05-23)
===================
- Document how to use shared caches with ``@cachedmethod``.
- Fix pickling/unpickling of cache keys
v3.1.0 (2019-01-29)
===================
- Fix Python 3.8 compatibility issue.
- Use ``time.monotonic`` as default timer if available.
- Improve documentation regarding thread safety.
v3.0.0 (2018-11-04)
===================
- Officially support Python 3.7.
- Drop Python 3.3 support (breaking change).
- Remove ``missing`` cache constructor parameter (breaking change).
- Remove ``self`` from ``@cachedmethod`` key arguments (breaking
change).
- Add support for ``maxsize=None`` in ``cachetools.func`` decorators.
v2.1.0 (2018-05-12)
===================
- Deprecate ``missing`` cache constructor parameter.
- Handle overridden ``getsizeof()`` method in subclasses.
- Fix Python 2.7 ``RRCache`` pickling issues.
- Various documentation improvements.
v2.0.1 (2017-08-11)
===================
- Officially support Python 3.6.
- Move documentation to RTD.
- Documentation: Update import paths for key functions (courtesy of
slavkoja).
v2.0.0 (2016-10-03)
===================
- Drop Python 3.2 support (breaking change).
- Drop support for deprecated features (breaking change).
- Move key functions to separate package (breaking change).
- Accept non-integer ``maxsize`` in ``Cache.__repr__()``.
v1.1.6 (2016-04-01)
===================
- Reimplement ``LRUCache`` and ``TTLCache`` using
``collections.OrderedDict``. Note that this will break pickle
compatibility with previous versions.
- Fix ``TTLCache`` not calling ``__missing__()`` of derived classes.
- Handle ``ValueError`` in ``Cache.__missing__()`` for consistency
with caching decorators.
- Improve how ``TTLCache`` handles expired items.
- Use ``Counter.most_common()`` for ``LFUCache.popitem()``.
v1.1.5 (2015-10-25)
===================
- Refactor ``Cache`` base class. Note that this will break pickle
compatibility with previous versions.
- Clean up ``LRUCache`` and ``TTLCache`` implementations.
v1.1.4 (2015-10-24)
===================
- Refactor ``LRUCache`` and ``TTLCache`` implementations. Note that
this will break pickle compatibility with previous versions.
- Document pending removal of deprecated features.
- Minor documentation improvements.
v1.1.3 (2015-09-15)
===================
- Fix pickle tests.
v1.1.2 (2015-09-15)
===================
- Fix pickling of large ``LRUCache`` and ``TTLCache`` instances.
v1.1.1 (2015-09-07)
===================
- Improve key functions.
- Improve documentation.
- Improve unit test coverage.
v1.1.0 (2015-08-28)
===================
- Add ``@cached`` function decorator.
- Add ``hashkey`` and ``typedkey`` functions.
- Add `key` and `lock` arguments to ``@cachedmethod``.
- Set ``__wrapped__`` attributes for Python versions < 3.2.
- Move ``functools`` compatible decorators to ``cachetools.func``.
- Deprecate ``@cachedmethod`` `typed` argument.
- Deprecate `cache` attribute for ``@cachedmethod`` wrappers.
- Deprecate `getsizeof` and `lock` arguments for `cachetools.func`
decorator.
v1.0.3 (2015-06-26)
===================
- Clear cache statistics when calling ``clear_cache()``.
v1.0.2 (2015-06-18)
===================
- Allow simple cache instances to be pickled.
- Refactor ``Cache.getsizeof`` and ``Cache.missing`` default
implementation.
v1.0.1 (2015-06-06)
===================
- Code cleanup for improved PEP 8 conformance.
- Add documentation and unit tests for using ``@cachedmethod`` with
generic mutable mappings.
- Improve documentation.
v1.0.0 (2014-12-19)
===================
- Provide ``RRCache.choice`` property.
- Improve documentation.
v0.8.2 (2014-12-15)
===================
- Use a ``NestedTimer`` for ``TTLCache``.
v0.8.1 (2014-12-07)
===================
- Deprecate ``Cache.getsize()``.
v0.8.0 (2014-12-03)
===================
- Ignore ``ValueError`` raised on cache insertion in decorators.
- Add ``Cache.getsize()``.
- Add ``Cache.__missing__()``.
- Feature freeze for `v1.0`.
v0.7.1 (2014-11-22)
===================
- Fix `MANIFEST.in`.
v0.7.0 (2014-11-12)
===================
- Deprecate ``TTLCache.ExpiredError``.
- Add `choice` argument to ``RRCache`` constructor.
- Refactor ``LFUCache``, ``LRUCache`` and ``TTLCache``.
- Use custom ``NullContext`` implementation for unsynchronized
function decorators.
v0.6.0 (2014-10-13)
===================
- Raise ``TTLCache.ExpiredError`` for expired ``TTLCache`` items.
- Support unsynchronized function decorators.
- Allow ``@cachedmethod.cache()`` to return None
v0.5.1 (2014-09-25)
===================
- No formatting of ``KeyError`` arguments.
- Update ``README.rst``.
v0.5.0 (2014-09-23)
===================
- Do not delete expired items in TTLCache.__getitem__().
- Add ``@ttl_cache`` function decorator.
- Fix public ``getsizeof()`` usage.
v0.4.0 (2014-06-16)
===================
- Add ``TTLCache``.
- Add ``Cache`` base class.
- Remove ``@cachedmethod`` `lock` parameter.
v0.3.1 (2014-05-07)
===================
- Add proper locking for ``cache_clear()`` and ``cache_info()``.
- Report `size` in ``cache_info()``.
v0.3.0 (2014-05-06)
===================
- Remove ``@cache`` decorator.
- Add ``size``, ``getsizeof`` members.
- Add ``@cachedmethod`` decorator.
v0.2.0 (2014-04-02)
===================
- Add ``@cache`` decorator.
- Update documentation.
v0.1.0 (2014-03-27)
===================
- Initial release.
tkem-cachetools-78b44a8/LICENSE 0000664 0000000 0000000 00000002075 15142726461 0016160 0 ustar 00root root 0000000 0000000 The MIT License (MIT)
Copyright (c) 2014-2026 Thomas Kemmer
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
tkem-cachetools-78b44a8/MANIFEST.in 0000664 0000000 0000000 00000000301 15142726461 0016677 0 ustar 00root root 0000000 0000000 include CHANGELOG.rst
include LICENSE
include MANIFEST.in
include README.rst
include tox.ini
exclude .readthedocs.yaml
recursive-include docs *
prune docs/_build
recursive-include tests *.py
tkem-cachetools-78b44a8/README.rst 0000664 0000000 0000000 00000011005 15142726461 0016633 0 ustar 00root root 0000000 0000000 cachetools
========================================================================
.. image:: https://img.shields.io/pypi/v/cachetools
:target: https://pypi.org/project/cachetools/
:alt: Latest PyPI version
.. image:: https://img.shields.io/github/actions/workflow/status/tkem/cachetools/ci.yml
:target: https://github.com/tkem/cachetools/actions/workflows/ci.yml
:alt: CI build status
.. image:: https://img.shields.io/readthedocs/cachetools
:target: https://cachetools.readthedocs.io/
:alt: Documentation build status
.. image:: https://img.shields.io/codecov/c/github/tkem/cachetools/master.svg
:target: https://codecov.io/gh/tkem/cachetools
:alt: Test coverage
.. image:: https://img.shields.io/librariesio/sourcerank/pypi/cachetools
:target: https://libraries.io/pypi/cachetools
:alt: Libraries.io SourceRank
.. image:: https://img.shields.io/github/license/tkem/cachetools
:target: https://raw.github.com/tkem/cachetools/master/LICENSE
:alt: License
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/psf/black
:alt: Code style: black
This module provides various memoizing collections and decorators,
including variants of the Python Standard Library's `@lru_cache`_
function decorator.
.. code-block:: python
from cachetools import cached, LRUCache, TTLCache
# speed up calculating Fibonacci numbers with dynamic programming
@cached(cache={})
def fib(n):
return n if n < 2 else fib(n - 1) + fib(n - 2)
# cache least recently used Python Enhancement Proposals
@cached(cache=LRUCache(maxsize=32))
def get_pep(num):
url = 'http://www.python.org/dev/peps/pep-%04d/' % num
with urllib.request.urlopen(url) as s:
return s.read()
# cache weather data for no longer than ten minutes
@cached(cache=TTLCache(maxsize=1024, ttl=600))
def get_weather(place):
return owm.weather_at_place(place).get_weather()
For the purpose of this module, a *cache* is a mutable_ mapping_ of a
fixed maximum size. When the cache is full, i.e. by adding another
item the cache would exceed its maximum size, the cache must choose
which item(s) to discard based on a suitable `cache algorithm`_.
This module provides multiple cache classes based on different cache
algorithms, as well as decorators for easily memoizing function and
method calls.
Installation
------------------------------------------------------------------------
cachetools is available from PyPI_ and can be installed by running::
pip install cachetools
Typing stubs for this package are provided by typeshed_ and can be
installed by running::
pip install types-cachetools
Project Resources
------------------------------------------------------------------------
- `Documentation`_
- `Issue tracker`_
- `Source code`_
- `Change log`_
Related Projects
------------------------------------------------------------------------
- asyncache_: Helpers to use cachetools_ with asyncio.
- cachetools-async_: Helpers to use cachetools_ with asyncio.
- cacheing_: Pure Python Cacheing Library.
- CacheToolsUtils_: Stackable cache classes for sharing, encryption,
statistics *and more* on top of cachetools_, redis_ and memcached_.
- shelved-cache_: Persistent cache implementation for Python
cachetools_.
License
------------------------------------------------------------------------
Copyright (c) 2014-2026 Thomas Kemmer.
Licensed under the `MIT License`_.
.. _@lru_cache: https://docs.python.org/3/library/functools.html#functools.lru_cache
.. _mutable: https://docs.python.org/dev/glossary.html#term-mutable
.. _mapping: https://docs.python.org/dev/glossary.html#term-mapping
.. _cache algorithm: https://en.wikipedia.org/wiki/Cache_algorithms
.. _PyPI: https://pypi.org/project/cachetools/
.. _typeshed: https://github.com/python/typeshed/
.. _Documentation: https://cachetools.readthedocs.io/
.. _Issue tracker: https://github.com/tkem/cachetools/issues/
.. _Source code: https://github.com/tkem/cachetools/
.. _Change log: https://github.com/tkem/cachetools/blob/master/CHANGELOG.rst
.. _MIT License: https://raw.github.com/tkem/cachetools/master/LICENSE
.. _asyncache: https://pypi.org/project/asyncache/
.. _cachetools-async: https://pypi.org/project/cachetools-async/
.. _cacheing: https://pypi.org/project/cacheing/
.. _CacheToolsUtils: https://pypi.org/project/CacheToolsUtils/
.. _shelved-cache: https://pypi.org/project/shelved-cache/
.. _cachetools: https://pypi.org/project/cachetools/
.. _redis: https://redis.io/
.. _memcached: https://memcached.org/
tkem-cachetools-78b44a8/docs/ 0000775 0000000 0000000 00000000000 15142726461 0016077 5 ustar 00root root 0000000 0000000 tkem-cachetools-78b44a8/docs/conf.py 0000664 0000000 0000000 00000001763 15142726461 0017405 0 ustar 00root root 0000000 0000000 import pathlib
import sys
src_directory = (pathlib.Path(__file__).parent.parent / "src").resolve()
sys.path.insert(0, str(src_directory))
# Extract the current version from the source.
def get_version():
"""Get the version and release from the source code."""
text = (src_directory / "cachetools/__init__.py").read_text()
for line in text.splitlines():
if not line.strip().startswith("__version__"):
continue
full_version = line.partition("=")[2].strip().strip("\"'")
partial_version = ".".join(full_version.split(".")[:2])
return full_version, partial_version
project = "cachetools"
copyright = "2014-2026 Thomas Kemmer"
release, version = get_version()
extensions = [
"sphinx.ext.autodoc",
"sphinx.ext.coverage",
"sphinx.ext.doctest",
"sphinx.ext.intersphinx",
"sphinx.ext.todo",
]
exclude_patterns = ["_build"]
master_doc = "index"
html_theme = "classic"
intersphinx_mapping = {"python": ("https://docs.python.org/3", None)}
tkem-cachetools-78b44a8/docs/index.rst 0000664 0000000 0000000 00000074112 15142726461 0017745 0 ustar 00root root 0000000 0000000 :tocdepth: 3
*********************************************************************
:mod:`cachetools` --- Extensible memoizing collections and decorators
*********************************************************************
.. module:: cachetools
This module provides various memoizing collections and decorators,
including variants of the Python Standard Library's `@lru_cache`_
function decorator.
For the purpose of this module, a *cache* is a mutable_ mapping_ of a
fixed maximum size. When the cache is full, i.e. by adding another
item the cache would exceed its maximum size, the cache must choose
which item(s) to discard based on a suitable `cache algorithm`_.
This module provides multiple cache classes based on different cache
algorithms, as well as decorators for easily memoizing function and
method calls.
.. testsetup:: *
from cachetools import cached, cachedmethod, LRUCache, TLRUCache, TTLCache
from unittest import mock
urllib = mock.MagicMock()
import threading
import time
Cache implementations
=====================
This module provides several classes implementing caches using
different cache algorithms. All these classes derive from class
:class:`Cache`, which in turn derives from
:class:`collections.abc.MutableMapping`, and provide :attr:`maxsize`
and :attr:`currsize` properties to retrieve the maximum and current
size of the cache. When a cache is full, :meth:`Cache.__setitem__()`
calls :meth:`self.popitem()` repeatedly until there is enough room for
the item to be added.
.. note::
Please be aware that `maxsize` must be a positive number. If you
really want your cache to grow without bounds, use
:const:`math.inf` or something similar.
In general, a cache's size is the total size of its item's values.
Therefore, :class:`Cache` provides a :meth:`getsizeof` method, which
returns the size of a given `value`. The default implementation of
:meth:`getsizeof` returns :const:`1` irrespective of its argument,
making the cache's size equal to the number of its items, or
`len(cache)`. For convenience, all cache classes accept an optional
named constructor parameter `getsizeof`, which may specify a callable
``getsizeof(value) -> int`` used to retrieve the size of an item's
value.
The values of a :class:`Cache` are mutable by default, as are e.g. the
values of a :class:`dict`. It is the user's responsibility to take
care that cached values are not accidentally modified. This is
especially important when using a custom `getsizeof` function, since
the size of an item's value will only be computed when the item is
inserted into the cache.
.. note::
Please be aware that all these classes are *not* thread-safe.
Access to a shared cache from multiple threads must be properly
synchronized, e.g. by using one of the memoizing decorators with a
suitable `lock` object.
.. autoclass:: Cache(maxsize, getsizeof=None)
:members: currsize, getsizeof, maxsize
This class discards arbitrary items using :meth:`popitem` to make
space when necessary. Derived classes may override :meth:`popitem`
to implement specific caching strategies. If a subclass has to
keep track of item access, insertion or deletion, it may
additionally need to override :meth:`__getitem__`,
:meth:`__setitem__` and :meth:`__delitem__`.
.. autoclass:: FIFOCache(maxsize, getsizeof=None)
:members: popitem
This class evicts items in the order they were added to make space
when necessary.
.. autoclass:: LFUCache(maxsize, getsizeof=None)
:members: popitem
This class counts how often an item is retrieved, and discards the
items used least often to make space when necessary.
.. autoclass:: LRUCache(maxsize, getsizeof=None)
:members: popitem
This class discards the least recently used items first to make
space when necessary.
.. autoclass:: RRCache(maxsize, choice=random.choice, getsizeof=None)
:members: choice, popitem
This class randomly selects candidate items and discards them to
make space when necessary.
By default, items are selected from the list of cache keys using
:func:`random.choice`. The optional argument `choice` may specify
an alternative function that returns an arbitrary element from a
non-empty sequence.
.. autoclass:: TTLCache(maxsize, ttl, timer=time.monotonic, getsizeof=None)
:members: popitem, timer, ttl
This class associates a time-to-live value with each item. Items
that expire because they have exceeded their time-to-live will be
no longer accessible, and will be removed eventually. If no
expired items are there to remove, the least recently used items
will be discarded first to make space when necessary.
By default, the time-to-live is specified in seconds and
:func:`time.monotonic` is used to retrieve the current time.
.. testcode::
cache = TTLCache(maxsize=10, ttl=60)
A custom `timer` function can also be supplied, which does not have
to return seconds, or even a numeric value. The expression
`timer() + ttl` at the time of insertion defines the expiration
time of a cache item and must be comparable against later results
of `timer()`, but `ttl` does not necessarily have to be a number,
either.
.. testcode::
from datetime import datetime, timedelta
cache = TTLCache(maxsize=10, ttl=timedelta(hours=12), timer=datetime.now)
.. method:: expire(self, time=None)
Expired items will be removed from a cache only at the next
mutating operation, e.g. :meth:`__setitem__` or
:meth:`__delitem__`, and therefore may still claim memory.
Calling this method removes all items whose time-to-live would
have expired by `time`, so garbage collection is free to reuse
their memory. If `time` is :const:`None`, this removes all
items that have expired by the current value returned by
:attr:`timer`.
:returns: An iterable of expired `(key, value)` pairs.
.. autoclass:: TLRUCache(maxsize, ttu, timer=time.monotonic, getsizeof=None)
:members: popitem, timer, ttu
Similar to :class:`TTLCache`, this class also associates an
expiration time with each item. However, for :class:`TLRUCache`
items, expiration time is calculated by a user-provided time-to-use
(`ttu`) function, which is passed three arguments at the time of
insertion: the new item's key and value, as well as the current
value of `timer()`.
.. testcode::
def my_ttu(_key, value, now):
# assume value.ttu contains the item's time-to-use in seconds
# note that the _key argument is ignored in this example
return now + value.ttu
cache = TLRUCache(maxsize=10, ttu=my_ttu)
The expression `ttu(key, value, timer())` defines the expiration
time of a cache item, and must be comparable against later results
of `timer()`. As with :class:`TTLCache`, a custom `timer` function
can be supplied, which does not have to return a numeric value.
.. testcode::
from datetime import datetime, timedelta
def datetime_ttu(_key, value, now):
# assume now to be of type datetime.datetime, and
# value.hours to contain the item's time-to-use in hours
return now + timedelta(hours=value.hours)
cache = TLRUCache(maxsize=10, ttu=datetime_ttu, timer=datetime.now)
Items that expire because they have exceeded their time-to-use will
be no longer accessible, and will be removed eventually. If no
expired items are there to remove, the least recently used items
will be discarded first to make space when necessary.
.. method:: expire(self, time=None)
Expired items will be removed from a cache only at the next
mutating operation, e.g. :meth:`__setitem__` or
:meth:`__delitem__`, and therefore may still claim memory.
Calling this method removes all items whose time-to-use would
have expired by `time`, so garbage collection is free to reuse
their memory. If `time` is :const:`None`, this removes all
items that have expired by the current value returned by
:attr:`timer`.
:returns: An iterable of expired `(key, value)` pairs.
Extending cache classes
=======================
Sometimes it may be desirable to notice when and what cache items are
evicted, i.e. removed from a cache to make room for new items. Since
all cache implementations call :meth:`popitem` to evict items from the
cache, this can be achieved by overriding this method in a subclass:
.. doctest::
>>> class MyCache(LRUCache):
... def popitem(self):
... key, value = super().popitem()
... print('Key "%s" evicted with value "%s"' % (key, value))
... return key, value
>>> c = MyCache(maxsize=2)
>>> c['a'] = 1
>>> c['b'] = 2
>>> c['c'] = 3
Key "a" evicted with value "1"
With :class:`TTLCache` and :class:`TLRUCache`, items may also be
removed after they expire. In this case, :meth:`popitem` will *not*
be called, but :meth:`expire` will be called from the next mutating
operation and will return an iterable of the expired `(key, value)`
pairs. By overriding :meth:`expire`, a subclass will be able to track
expired items:
.. doctest::
>>> class ExpCache(TTLCache):
... def expire(self, time=None):
... items = super().expire(time)
... print(f"Expired items: {items}")
... return items
>>> c = ExpCache(maxsize=10, ttl=1.0)
>>> c['a'] = 1
Expired items: []
>>> c['b'] = 2
Expired items: []
>>> time.sleep(1.5)
>>> c['c'] = 3
Expired items: [('a', 1), ('b', 2)]
Similar to the standard library's :class:`collections.defaultdict`,
subclasses of :class:`Cache` may implement a :meth:`__missing__`
method which is called by :meth:`Cache.__getitem__` if the requested
key is not found:
.. doctest::
>>> class PepStore(LRUCache):
... def __missing__(self, key):
... """Retrieve text of a Python Enhancement Proposal"""
... url = 'http://www.python.org/dev/peps/pep-%04d/' % key
... with urllib.request.urlopen(url) as s:
... pep = s.read()
... self[key] = pep # store text in cache
... return pep
>>> peps = PepStore(maxsize=4)
>>> for n in 8, 9, 290, 308, 320, 8, 218, 320, 279, 289, 320:
... pep = peps[n]
>>> print(sorted(peps.keys()))
[218, 279, 289, 320]
Note, though, that such a class does not really behave like a *cache*
any more, and will lead to surprising results when used with any of
the memoizing decorators described below. However, it may be useful
in its own right.
Memoizing decorators
====================
The :mod:`cachetools` module provides decorators for memoizing
function and method calls. This can save time when a function is
often called with the same arguments:
.. doctest::
>>> @cached(cache={})
... def fib(n):
... 'Compute the nth number in the Fibonacci sequence'
... return n if n < 2 else fib(n - 1) + fib(n - 2)
>>> fib(42)
267914296
.. decorator:: cached(cache, key=cachetools.keys.hashkey, lock=None, condition=None, info=False)
Decorator to wrap a function with a memoizing callable that saves
results in a cache.
The `cache` argument specifies a cache object to store previous
function arguments and return values. Note that `cache` need not
be an instance of the cache implementations provided by the
:mod:`cachetools` module. :func:`cached` will work with any
mutable mapping type, including plain :class:`dict` and
:class:`weakref.WeakValueDictionary`.
`key` specifies a function that will be called with the same
positional and keyword arguments as the wrapped function itself,
and which has to return a suitable cache key. Since caches are
mappings, the object returned by `key` must be hashable. The
default is to call :func:`cachetools.keys.hashkey`.
If `lock` is not :const:`None`, it must specify an object
implementing the `context manager`_ protocol. Any access to the
cache will then be nested in a ``with lock:`` statement. This can
be used for synchronizing thread access to the cache by providing a
:class:`threading.Lock` or :class:`threading.RLock` instance, for
example.
.. note::
The `lock` context manager is used only to guard access to the
cache object. The underlying wrapped function will be called
outside the `with` statement to allow concurrent execution, and
therefore must be `thread-safe`_ by itself.
If `condition` is not :const:`None`, it must specify a `condition
variable`_, i.e. an object providing :func:`wait()`,
:func:`wait_for()`, :func:`notify()` and :func:`notify_all()`
methods as defined by :class:`threading.Condition`. Using a
`condition` variable will prevent concurrent execution of the
wrapped function with *identical* parameters, or cache keys.
Instead, a calling thread will check if an identical function call
is already executing, and will then :func:`wait()` for the pending
call to finish. The executing thread will :func:`notify()` any
waiting threads as soon as the function completes, which will then
return the cached function result.
.. note::
Although providing a `lock` alone is generally sufficient to
make :func:`cached` `thread-safe`_, it may still be subject to
`cache stampede`_ issues under high load, depending on your
actual use case. Providing a `condition` variable will mitigate
these situations, but will inflict some performance penalty.
If no separate `lock` parameter is provided, `condition` must also
implement the `context manager`_ protocol, and will also be used to
guard access to the cache.
The decorator's `cache`, `key`, `lock` and `condition` parameters
are also available as :attr:`cache`, :attr:`cache_key`,
:attr:`cache_lock` and :attr:`cache_condition` attributes of the
memoizing wrapper function. These can be used for clearing the
cache or invalidating individual cache items, for example.
.. testcode::
# 640K should be enough for anyone...
@cached(cache=LRUCache(maxsize=640*1024, getsizeof=len), lock=threading.Lock())
def get_pep(num):
'Retrieve text of a Python Enhancement Proposal'
url = 'http://www.python.org/dev/peps/pep-%04d/' % num
with urllib.request.urlopen(url) as s:
return s.read()
# make sure access to cache is synchronized
with get_pep.cache_lock:
get_pep.cache.clear()
# always use the key function for accessing cache items
with get_pep.cache_lock:
get_pep.cache.pop(get_pep.cache_key(42), None)
For the common use case of clearing or invalidating the cache, the
decorator also provides a :func:`cache_clear()` function which
takes care of locking automatically, if needed:
.. testcode::
# no need for get_pep.cache_lock here
get_pep.cache_clear()
If `info` is set to :const:`True`, the wrapped function is
instrumented with a :func:`cache_info()` function that returns a
named tuple showing `hits`, `misses`, `maxsize` and `currsize`, to
help measure the effectiveness of the cache.
.. note::
Note that this will inflict some performance penalty, so it has
to be enabled explicitly.
.. doctest::
>>> @cached(cache=LRUCache(maxsize=32), info=True)
... def get_pep(num):
... url = 'http://www.python.org/dev/peps/pep-%04d/' % num
... with urllib.request.urlopen(url) as s:
... return s.read()
>>> for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991:
... pep = get_pep(n)
>>> get_pep.cache_info()
CacheInfo(hits=3, misses=8, maxsize=32, currsize=8)
The original underlying function is accessible through the
:attr:`__wrapped__` attribute. This can be used for introspection
or for bypassing the cache.
It is also possible to use a single shared cache object with
multiple functions. However, care must be taken that different
cache keys are generated for each function, even for identical
function arguments:
.. doctest::
:options: +ELLIPSIS
>>> from cachetools.keys import hashkey
>>> from functools import partial
>>> # shared cache for integer sequences
>>> numcache = {}
>>> # compute Fibonacci numbers
>>> @cached(numcache, key=partial(hashkey, 'fib'))
... def fib(n):
... return n if n < 2 else fib(n - 1) + fib(n - 2)
>>> # compute Lucas numbers
>>> @cached(numcache, key=partial(hashkey, 'luc'))
... def luc(n):
... return 2 - n if n < 2 else luc(n - 1) + luc(n - 2)
>>> fib(42)
267914296
>>> luc(42)
599074578
>>> list(sorted(numcache.items()))
[..., (('fib', 42), 267914296), ..., (('luc', 42), 599074578)]
Function invocations are *not* cached if any exceptions are raised.
To cache some (or all) calls raising exceptions, additional
function wrappers may be introduced which wrap exceptions as
regular function results for caching purposes:
.. testcode::
@cached(cache=LRUCache(maxsize=10), info=True)
def _get_pep_wrapped(num):
url = "http://www.python.org/dev/peps/pep-%04d/" % num
try:
with urllib.request.urlopen(url) as s:
return s.read()
except urllib.error.HTTPError as e:
# note that only HTTPError instances are cached
return e
def get_pep(num):
"Retrieve text of a Python Enhancement Proposal"
res = _get_pep_wrapped(num)
if isinstance(res, Exception):
raise res
else:
return res
try:
get_pep(100_000_000)
except Exception as e:
print(e, "-", _get_pep_wrapped.cache_info())
try:
get_pep(100_000_000)
except Exception as e:
print(e, "-", _get_pep_wrapped.cache_info())
Curiously, default function arguments are not quite handled as one
might expect, and also the use of positional vs. keyword arguments
may lead to surprising results. In the example below, `foo()`,
`foo(1)` and `foo(a=1)` are treated as different function
invocations, with separately cached results:
.. doctest::
>>> @cached(LRUCache(maxsize=100))
... def foo(a=1):
... print(f"foo({a}) called")
...
>>> foo()
foo(1) called
>>> foo()
>>> foo(1)
foo(1) called
>>> foo(1)
>>> foo(a=1)
foo(1) called
>>> foo(a=1)
If consistent behavior is required, a private helper function may
be introduced to avoid ambiguities, e.g.:
.. doctest::
>>> def foo(a=1):
... _foo(a)
...
>>> @cached(LRUCache(maxsize=100))
... def _foo(a):
... print(f"_foo({a}) called")
...
>>> foo()
_foo(1) called
>>> foo()
>>> foo(1)
>>> foo(a=1)
.. decorator:: cachedmethod(cache, key=cachetools.keys.methodkey, lock=None, condition=None, info=False)
Decorator to wrap an instance method with a memoizing callable that
saves results in a cache.
The main difference between this and the :func:`cached` function
decorator is that `cache`, `lock` and `condition` are not passed
objects, but functions. Those will be called with :const:`self` as
their sole argument to retrieve a valid cache, lock, or condition
object for the respective instance.
.. note::
As with :func:`cached`, the context manager obtained by calling
`lock(self)` will only guard access to the cache itself. It is
the user's responsibility to handle concurrent calls to the
underlying wrapped method in a multithreaded environment.
Similarly to the :func:`cached` function decorator, the `cache`,
`key`, `lock` and `condition` parameters are also available as
:attr:`cache`, :attr:`cache_key`, :attr:`cache_lock` and
:attr:`cache_condition` attributes of the wrapped instance method.
A :func:`cache_clear()` function will also be provided, as well as
an optional :func:`cache_info()` function reporting per-instance
cache statistics.
Also, mostly for efficiency, this decorator requires that the
:attr:`__dict__` attribute on each instance be a mutable mapping.
This means it will not work with some types, such as metaclasses,
and those that specify :attr:`__slots__` without including
:attr:`__dict__` as one of the defined slots.
One advantage of :func:`cachedmethod` over the :func:`cached`
function decorator is that cache properties such as `maxsize` can
be set at runtime:
.. doctest::
>>> class CachedPEPs:
... def __init__(self, cachesize):
... self.cache = LRUCache(maxsize=cachesize)
... self.lock = threading.Lock()
...
... @cachedmethod(lambda self: self.cache, lock=lambda self: self.lock, info=True)
... def get(self, num):
... """Retrieve text of a Python Enhancement Proposal"""
... url = 'http://www.python.org/dev/peps/pep-%04d/' % num
... with urllib.request.urlopen(url) as s:
... return s.read()
>>> peps = CachedPEPs(cachesize=32)
>>> for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991:
... pep = peps.get(n)
>>> peps.get.cache_info()
CacheInfo(hits=3, misses=8, maxsize=32, currsize=8)
>>> peps.get.cache_clear()
>>> peps.get.cache_info()
CacheInfo(hits=0, misses=0, maxsize=32, currsize=0)
The `key` function will be called as `key(self, *args, **kwargs)`
to retrieve a suitable cache key. Note that the default `key`
function, :func:`cachetools.keys.methodkey`, ignores its first
argument, i.e. :const:`self`. This has mostly historical reasons,
but also ensures that :const:`self` does not have to be hashable.
You may provide a different `key` function,
e.g. :func:`cachetools.keys.hashkey`, if you need :const:`self` to
be part of the cache key.
When using a shared cache for multiple methods, be aware that
different cache keys must be created for each method even when
function arguments are the same, just as with the :func:`cached`
decorator:
.. testcode::
from cachetools.keys import methodkey
from functools import partial
class CachedReferences:
def __init__(self, cachesize):
self.cache = LRUCache(maxsize=cachesize)
@cachedmethod(lambda self: self.cache, key=partial(methodkey, method='pep'))
def get_pep(self, num):
"""Retrieve text of a Python Enhancement Proposal"""
url = 'http://www.python.org/dev/peps/pep-%04d/' % num
with urllib.request.urlopen(url) as s:
return s.read()
@cachedmethod(lambda self: self.cache, key=partial(methodkey, method='rfc'))
def get_rfc(self, num):
"""Retrieve text of an IETF Request for Comments"""
url = 'https://tools.ietf.org/rfc/rfc%d.txt' % num
with urllib.request.urlopen(url) as s:
return s.read()
docs = CachedReferences(cachesize=100)
print("PEP #20: %s" % docs.get_pep(20))
print("RFC #20: %s" % docs.get_rfc(20))
assert len(docs.cache) == 2
.. testoutput::
:hide:
:options: +ELLIPSIS
PEP #20: ...
RFC #20: ...
Note how keyword arguments are used with :func:`functools.partial`
to create distinct cache keys, to avoid issues with
:func:`methodkey` skipping its initial `self` argument.
.. versionchanged:: 7.0
Added the `info` option for reporting per-instance cache
statistics.
:func:`cachedmethod` attributes (:attr:`cache`, :attr:`cache_lock`,
etc.) are now implemented as properties for instance methods, and
are finally officially supported.
Require :attr:`__dict__` to be a mutable mapping to support
per-instance :func:`cache_info()`.
Returning :const:`None` from `cache(self)` is no longer supported.
.. deprecated:: 7.0
Using :func:`cachedmethod` with :func:`classmethod` is deprecated,
and support will be removed in the next major version. This is
mostly due to the fact that chaining descriptors, i.e. using
descriptors with :func:`classmethod`, was deprecated in Python 3.11
and removed in Python 3.13. When using :func:`cache_info()`, an
instance method is already required.
Using :func:`cachedmethod` with an instance that does not provide a
mutable :attr:`__dict__` attribute is deprecated, and may inflict a
noticeable performance penalty. When using :func:`cache_info()`,
:attr:`__dict__` already has to be a mutable mapping.
*****************************************************************
:mod:`cachetools.keys` --- Key functions for memoizing decorators
*****************************************************************
.. module:: cachetools.keys
This module provides several functions that can be used as key
functions with the :func:`cached` and :func:`cachedmethod` decorators:
.. autofunction:: hashkey
This function returns a :class:`tuple` instance suitable as a cache
key, provided the positional and keywords arguments are hashable.
.. autofunction:: methodkey
This function is similar to :func:`hashkey`, but ignores its
first positional argument, i.e. `self` when used with the
:func:`cachedmethod` decorator.
.. autofunction:: typedkey
This function is similar to :func:`hashkey`, but arguments of
different types will yield distinct cache keys. For example,
`typedkey(3)` and `typedkey(3.0)` will return different results.
.. autofunction:: typedmethodkey
This function is similar to :func:`typedkey`, but ignores its
first positional argument, i.e. `self` when used with the
:func:`cachedmethod` decorator.
These functions can also be helpful when implementing custom key
functions for handling some non-hashable arguments. For example,
calling the following function with a dictionary as its `env` argument
will raise a :class:`TypeError`, since :class:`dict` is not hashable::
@cached(LRUCache(maxsize=128))
def foo(x, y, z, env={}):
pass
However, if `env` always holds only hashable values itself, a custom
key function can be written that handles the `env` keyword argument
specially::
def envkey(*args, env={}, **kwargs):
key = hashkey(*args, **kwargs)
key += tuple(sorted(env.items()))
return key
The :func:`envkey` function can then be used in decorator declarations
like this::
@cached(LRUCache(maxsize=128), key=envkey)
def foo(x, y, z, env={}):
pass
foo(1, 2, 3, env=dict(a='a', b='b'))
****************************************************************************
:mod:`cachetools.func` --- :func:`functools.lru_cache` compatible decorators
****************************************************************************
.. module:: cachetools.func
To ease migration from (or to) Python 3's :func:`functools.lru_cache`,
this module provides several memoizing function decorators with a
similar API. All these decorators wrap a function with a memoizing
callable that saves up to the `maxsize` most recent calls, using
different caching strategies. If `maxsize` is set to :const:`None`,
the caching strategy is effectively disabled and the cache can grow
without bound.
If the optional argument `typed` is set to :const:`True`, function
arguments of different types will be cached separately. For example,
`f(3)` and `f(3.0)` will be treated as distinct calls with distinct
results.
If a `user_function` is specified instead, it must be a callable.
This allows the decorator to be applied directly to a user function,
leaving the `maxsize` at its default value of 128::
@cachetools.func.lru_cache
def count_vowels(sentence):
sentence = sentence.casefold()
return sum(sentence.count(vowel) for vowel in 'aeiou')
The wrapped function is instrumented with a :func:`cache_parameters`
function that returns a new :class:`dict` showing the values for
`maxsize` and `typed`. This is for information purposes only.
Mutating the values has no effect.
The wrapped function is also instrumented with :func:`cache_info` and
:func:`cache_clear` functions to provide information about cache
performance and clear the cache. Please see the
:func:`functools.lru_cache` documentation for details. Also note that
all the decorators in this module are thread-safe by default.
.. decorator:: fifo_cache(user_function)
fifo_cache(maxsize=128, typed=False)
Decorator that wraps a function with a memoizing callable that
saves up to `maxsize` results based on a First In First Out
(FIFO) algorithm.
.. decorator:: lfu_cache(user_function)
lfu_cache(maxsize=128, typed=False)
Decorator that wraps a function with a memoizing callable that
saves up to `maxsize` results based on a Least Frequently Used
(LFU) algorithm.
.. decorator:: lru_cache(user_function)
lru_cache(maxsize=128, typed=False)
Decorator that wraps a function with a memoizing callable that
saves up to `maxsize` results based on a Least Recently Used (LRU)
algorithm.
.. decorator:: rr_cache(user_function)
rr_cache(maxsize=128, choice=random.choice, typed=False)
Decorator that wraps a function with a memoizing callable that
saves up to `maxsize` results based on a Random Replacement (RR)
algorithm.
.. decorator:: ttl_cache(user_function)
ttl_cache(maxsize=128, ttl=600, timer=time.monotonic, typed=False)
Decorator to wrap a function with a memoizing callable that saves
up to `maxsize` results based on a Least Recently Used (LRU)
algorithm with a per-item time-to-live (TTL) value. By default,
the time-to-live is set to 600 seconds and :func:`time.monotonic`
is used to retrieve the current time.
.. _@lru_cache: https://docs.python.org/3/library/functools.html#functools.lru_cache
.. _cache algorithm: https://en.wikipedia.org/wiki/Cache_algorithms
.. _cache stampede: https://en.wikipedia.org/wiki/Cache_stampede
.. _condition variable: https://docs.python.org/3/library/threading.html#condition-objects
.. _context manager: https://docs.python.org/dev/glossary.html#term-context-manager
.. _mapping: https://docs.python.org/dev/glossary.html#term-mapping
.. _mutable: https://docs.python.org/dev/glossary.html#term-mutable
.. _thread-safe: https://en.wikipedia.org/wiki/Thread_safety
tkem-cachetools-78b44a8/pyproject.toml 0000664 0000000 0000000 00000002654 15142726461 0020072 0 ustar 00root root 0000000 0000000 [build-system]
requires = ["setuptools >= 61.0.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "cachetools"
dynamic = ["version"]
description = "Extensible memoizing collections and decorators"
authors = [
{name = "Thomas Kemmer", email = "tkemmer@computer.org"}
]
maintainers = [
{name = "Thomas Kemmer", email = "tkemmer@computer.org"}
]
readme = "README.rst"
license = "MIT"
license-files = ["LICENSE"]
requires-python = ">= 3.10"
classifiers = [
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Topic :: Software Development :: Libraries :: Python Modules",
]
[project.urls]
Homepage = "https://github.com/tkem/cachetools/"
[tool.setuptools]
package-dir = {"" = "src"}
[tool.setuptools.packages.find]
where = ["src"]
[tool.setuptools.dynamic]
version = {attr = "cachetools.__version__"}
[tool.flake8]
max-line-length = 80
exclude = [".git", ".tox", "build"]
select = ["C", "E", "F", "W", "B", "B950", "I", "N"]
# F401: imported but unused (submodule shims)
# E501: line too long (black)
ignore = ["F401", "E501"]
tkem-cachetools-78b44a8/src/ 0000775 0000000 0000000 00000000000 15142726461 0015736 5 ustar 00root root 0000000 0000000 tkem-cachetools-78b44a8/src/cachetools/ 0000775 0000000 0000000 00000000000 15142726461 0020062 5 ustar 00root root 0000000 0000000 tkem-cachetools-78b44a8/src/cachetools/__init__.py 0000664 0000000 0000000 00000052563 15142726461 0022206 0 ustar 00root root 0000000 0000000 """Extensible memoizing collections and decorators."""
__all__ = (
"Cache",
"FIFOCache",
"LFUCache",
"LRUCache",
"RRCache",
"TLRUCache",
"TTLCache",
"cached",
"cachedmethod",
)
__version__ = "7.0.1"
import collections
import collections.abc
import functools
import heapq
import random
import time
from . import keys
# Typing stubs for this package are provided by typeshed:
# https://github.com/python/typeshed/tree/main/stubs/cachetools
class _DefaultSize:
"""A minimal "fake" dict that returns a constant size 1 for any key."""
__slots__ = ()
def __getitem__(self, _key):
return 1
def __setitem__(self, _key, _value):
pass
def pop(self, _key):
return 1
class Cache(collections.abc.MutableMapping):
"""Mutable mapping to serve as a simple cache or cache base class."""
__marker = object()
__size = _DefaultSize()
def __init__(self, maxsize, getsizeof=None):
if getsizeof:
self.getsizeof = getsizeof
if self.getsizeof is not Cache.getsizeof:
self.__size = dict()
self.__data = dict()
self.__currsize = 0
self.__maxsize = maxsize
def __repr__(self):
return "%s(%s, maxsize=%r, currsize=%r)" % (
type(self).__name__,
repr(self.__data),
self.__maxsize,
self.__currsize,
)
def __getitem__(self, key):
try:
return self.__data[key]
except KeyError:
return self.__missing__(key)
def __setitem__(self, key, value):
maxsize = self.__maxsize
size = self.getsizeof(value)
if size > maxsize:
raise ValueError("value too large")
if key not in self.__data or self.__size[key] < size:
while self.__currsize + size > maxsize:
self.popitem()
if key in self.__data:
diffsize = size - self.__size[key]
else:
diffsize = size
self.__data[key] = value
self.__size[key] = size
self.__currsize += diffsize
def __delitem__(self, key):
size = self.__size.pop(key)
del self.__data[key]
self.__currsize -= size
def __contains__(self, key):
return key in self.__data
def __missing__(self, key):
raise KeyError(key)
def __iter__(self):
return iter(self.__data)
def __len__(self):
return len(self.__data)
# Note that we cannot simply inherit get(), pop() and setdefault()
# from MutableMapping, since these rely on __getitem__ throwing a
# KeyError on cache miss. This is not the case if __missing__ is
# implemented for a Cache subclass, so we have to roll our own,
# somewhat less elegant versions.
def get(self, key, default=None):
if key in self:
return self[key]
else:
return default
def pop(self, key, default=__marker):
if key in self:
value = self[key]
del self[key]
elif default is self.__marker:
raise KeyError(key)
else:
value = default
return value
def setdefault(self, key, default=None):
if key in self:
value = self[key]
else:
self[key] = value = default
return value
@property
def maxsize(self):
"""The maximum size of the cache."""
return self.__maxsize
@property
def currsize(self):
"""The current size of the cache."""
return self.__currsize
@staticmethod
def getsizeof(value):
"""Return the size of a cache element's value."""
return 1
class FIFOCache(Cache):
"""First In First Out (FIFO) cache implementation."""
def __init__(self, maxsize, getsizeof=None):
Cache.__init__(self, maxsize, getsizeof)
self.__order = collections.OrderedDict()
def __setitem__(self, key, value, cache_setitem=Cache.__setitem__):
cache_setitem(self, key, value)
if key in self.__order:
self.__order.move_to_end(key)
else:
self.__order[key] = None
def __delitem__(self, key, cache_delitem=Cache.__delitem__):
cache_delitem(self, key)
del self.__order[key]
def popitem(self):
"""Remove and return the `(key, value)` pair first inserted."""
try:
key = next(iter(self.__order))
except StopIteration:
raise KeyError("%s is empty" % type(self).__name__) from None
else:
return (key, self.pop(key))
class LFUCache(Cache):
"""Least Frequently Used (LFU) cache implementation."""
class _Link:
__slots__ = ("count", "keys", "next", "prev")
def __init__(self, count):
self.count = count
self.keys = set()
def unlink(self):
next = self.next
prev = self.prev
prev.next = next
next.prev = prev
def __init__(self, maxsize, getsizeof=None):
Cache.__init__(self, maxsize, getsizeof)
self.__root = root = LFUCache._Link(0) # sentinel
root.prev = root.next = root
self.__links = {}
def __getitem__(self, key, cache_getitem=Cache.__getitem__):
value = cache_getitem(self, key)
if key in self: # __missing__ may not store item
self.__touch(key)
return value
def __setitem__(self, key, value, cache_setitem=Cache.__setitem__):
cache_setitem(self, key, value)
if key in self.__links:
self.__touch(key)
return
root = self.__root
link = root.next
if link.count != 1:
link = LFUCache._Link(1)
link.next = root.next
root.next = link.next.prev = link
link.prev = root
link.keys.add(key)
self.__links[key] = link
def __delitem__(self, key, cache_delitem=Cache.__delitem__):
cache_delitem(self, key)
link = self.__links.pop(key)
link.keys.remove(key)
if not link.keys:
link.unlink()
def popitem(self):
"""Remove and return the `(key, value)` pair least frequently used."""
root = self.__root
curr = root.next
if curr is root:
raise KeyError("%s is empty" % type(self).__name__) from None
key = next(iter(curr.keys)) # remove an arbitrary element
return (key, self.pop(key))
def __touch(self, key):
"""Increment use count"""
link = self.__links[key]
curr = link.next
if curr.count != link.count + 1:
if len(link.keys) == 1:
link.count += 1
return
curr = LFUCache._Link(link.count + 1)
curr.next = link.next
link.next = curr.next.prev = curr
curr.prev = link
curr.keys.add(key)
link.keys.remove(key)
if not link.keys:
link.unlink()
self.__links[key] = curr
class LRUCache(Cache):
"""Least Recently Used (LRU) cache implementation."""
def __init__(self, maxsize, getsizeof=None):
Cache.__init__(self, maxsize, getsizeof)
self.__order = collections.OrderedDict()
def __getitem__(self, key, cache_getitem=Cache.__getitem__):
value = cache_getitem(self, key)
if key in self: # __missing__ may not store item
self.__touch(key)
return value
def __setitem__(self, key, value, cache_setitem=Cache.__setitem__):
cache_setitem(self, key, value)
self.__touch(key)
def __delitem__(self, key, cache_delitem=Cache.__delitem__):
cache_delitem(self, key)
del self.__order[key]
def popitem(self):
"""Remove and return the `(key, value)` pair least recently used."""
try:
key = next(iter(self.__order))
except StopIteration:
raise KeyError("%s is empty" % type(self).__name__) from None
else:
return (key, self.pop(key))
def __touch(self, key):
"""Mark as recently used"""
try:
self.__order.move_to_end(key)
except KeyError:
self.__order[key] = None
class RRCache(Cache):
"""Random Replacement (RR) cache implementation."""
def __init__(self, maxsize, choice=random.choice, getsizeof=None):
Cache.__init__(self, maxsize, getsizeof)
self.__choice = choice
self.__index = {}
self.__keys = []
@property
def choice(self):
"""The `choice` function used by the cache."""
return self.__choice
def __setitem__(self, key, value, cache_setitem=Cache.__setitem__):
cache_setitem(self, key, value)
if key not in self.__index:
self.__index[key] = len(self.__keys)
self.__keys.append(key)
def __delitem__(self, key, cache_delitem=Cache.__delitem__):
cache_delitem(self, key)
index = self.__index.pop(key)
if index != len(self.__keys) - 1:
last = self.__keys[-1]
self.__keys[index] = last
self.__index[last] = index
self.__keys.pop()
def popitem(self):
"""Remove and return a random `(key, value)` pair."""
try:
key = self.__choice(self.__keys)
except IndexError:
raise KeyError("%s is empty" % type(self).__name__) from None
else:
return (key, self.pop(key))
class _TimedCache(Cache):
"""Base class for time aware cache implementations."""
class _Timer:
def __init__(self, timer):
self.__timer = timer
self.__nesting = 0
def __call__(self):
if self.__nesting == 0:
return self.__timer()
else:
return self.__time
def __enter__(self):
if self.__nesting == 0:
self.__time = time = self.__timer()
else:
time = self.__time
self.__nesting += 1
return time
def __exit__(self, *exc):
self.__nesting -= 1
def __reduce__(self):
return _TimedCache._Timer, (self.__timer,)
def __getattr__(self, name):
return getattr(self.__timer, name)
def __init__(self, maxsize, timer=time.monotonic, getsizeof=None):
Cache.__init__(self, maxsize, getsizeof)
self.__timer = _TimedCache._Timer(timer)
def __repr__(self, cache_repr=Cache.__repr__):
with self.__timer as time:
self.expire(time)
return cache_repr(self)
def __len__(self, cache_len=Cache.__len__):
with self.__timer as time:
self.expire(time)
return cache_len(self)
@property
def currsize(self):
with self.__timer as time:
self.expire(time)
return super().currsize
@property
def timer(self):
"""The timer function used by the cache."""
return self.__timer
def clear(self):
with self.__timer as time:
self.expire(time)
Cache.clear(self)
def get(self, *args, **kwargs):
with self.__timer:
return Cache.get(self, *args, **kwargs)
def pop(self, *args, **kwargs):
with self.__timer:
return Cache.pop(self, *args, **kwargs)
def setdefault(self, *args, **kwargs):
with self.__timer:
return Cache.setdefault(self, *args, **kwargs)
class TTLCache(_TimedCache):
"""LRU Cache implementation with per-item time-to-live (TTL) value."""
class _Link:
__slots__ = ("key", "expires", "next", "prev")
def __init__(self, key=None, expires=None):
self.key = key
self.expires = expires
def __reduce__(self):
return TTLCache._Link, (self.key, self.expires)
def unlink(self):
next = self.next
prev = self.prev
prev.next = next
next.prev = prev
def __init__(self, maxsize, ttl, timer=time.monotonic, getsizeof=None):
_TimedCache.__init__(self, maxsize, timer, getsizeof)
self.__root = root = TTLCache._Link()
root.prev = root.next = root
self.__links = collections.OrderedDict()
self.__ttl = ttl
def __contains__(self, key):
try:
link = self.__links[key] # no reordering
except KeyError:
return False
else:
return self.timer() < link.expires
def __getitem__(self, key, cache_getitem=Cache.__getitem__):
try:
link = self.__getlink(key)
except KeyError:
expired = False
else:
expired = not (self.timer() < link.expires)
if expired:
return self.__missing__(key)
else:
return cache_getitem(self, key)
def __setitem__(self, key, value, cache_setitem=Cache.__setitem__):
with self.timer as time:
self.expire(time)
cache_setitem(self, key, value)
try:
link = self.__getlink(key)
except KeyError:
self.__links[key] = link = TTLCache._Link(key)
else:
link.unlink()
link.expires = time + self.__ttl
link.next = root = self.__root
link.prev = prev = root.prev
prev.next = root.prev = link
def __delitem__(self, key, cache_delitem=Cache.__delitem__):
cache_delitem(self, key)
link = self.__links.pop(key)
link.unlink()
if not (self.timer() < link.expires):
raise KeyError(key)
def __iter__(self):
root = self.__root
curr = root.next
while curr is not root:
# "freeze" time for iterator access
with self.timer as time:
if time < curr.expires:
yield curr.key
curr = curr.next
def __setstate__(self, state):
self.__dict__.update(state)
root = self.__root
root.prev = root.next = root
for link in sorted(self.__links.values(), key=lambda obj: obj.expires):
link.next = root
link.prev = prev = root.prev
prev.next = root.prev = link
self.expire(self.timer())
@property
def ttl(self):
"""The time-to-live value of the cache's items."""
return self.__ttl
def expire(self, time=None):
"""Remove expired items from the cache and return an iterable of the
expired `(key, value)` pairs.
"""
if time is None:
time = self.timer()
root = self.__root
curr = root.next
links = self.__links
expired = []
cache_delitem = Cache.__delitem__
cache_getitem = Cache.__getitem__
while curr is not root and not (time < curr.expires):
expired.append((curr.key, cache_getitem(self, curr.key)))
cache_delitem(self, curr.key)
del links[curr.key]
next = curr.next
curr.unlink()
curr = next
return expired
def popitem(self):
"""Remove and return the `(key, value)` pair least recently used that
has not already expired.
"""
with self.timer as time:
self.expire(time)
try:
key = next(iter(self.__links))
except StopIteration:
raise KeyError("%s is empty" % type(self).__name__) from None
else:
return (key, self.pop(key))
def __getlink(self, key):
value = self.__links[key]
self.__links.move_to_end(key)
return value
class TLRUCache(_TimedCache):
"""Time aware Least Recently Used (TLRU) cache implementation."""
__HEAP_CLEANUP_FACTOR = 2 # clean up the heap if size > N * len(items)
@functools.total_ordering
class _Item:
__slots__ = ("key", "expires", "removed")
def __init__(self, key=None, expires=None):
self.key = key
self.expires = expires
self.removed = False
def __lt__(self, other):
return self.expires < other.expires
def __init__(self, maxsize, ttu, timer=time.monotonic, getsizeof=None):
_TimedCache.__init__(self, maxsize, timer, getsizeof)
self.__items = collections.OrderedDict()
self.__order = []
self.__ttu = ttu
def __contains__(self, key):
try:
item = self.__items[key] # no reordering
except KeyError:
return False
else:
return self.timer() < item.expires
def __getitem__(self, key, cache_getitem=Cache.__getitem__):
try:
item = self.__getitem(key)
except KeyError:
expired = False
else:
expired = not (self.timer() < item.expires)
if expired:
return self.__missing__(key)
else:
return cache_getitem(self, key)
def __setitem__(self, key, value, cache_setitem=Cache.__setitem__):
with self.timer as time:
expires = self.__ttu(key, value, time)
if not (time < expires):
return # skip expired items
self.expire(time)
cache_setitem(self, key, value)
# removing an existing item would break the heap structure, so
# only mark it as removed for now
try:
self.__getitem(key).removed = True
except KeyError:
pass
self.__items[key] = item = TLRUCache._Item(key, expires)
heapq.heappush(self.__order, item)
def __delitem__(self, key, cache_delitem=Cache.__delitem__):
with self.timer as time:
# no self.expire() for performance reasons, e.g. self.clear() [#67]
cache_delitem(self, key)
item = self.__items.pop(key)
item.removed = True
if not (time < item.expires):
raise KeyError(key)
def __iter__(self):
for curr in self.__order:
# "freeze" time for iterator access
with self.timer as time:
if time < curr.expires and not curr.removed:
yield curr.key
@property
def ttu(self):
"""The local time-to-use function used by the cache."""
return self.__ttu
def expire(self, time=None):
"""Remove expired items from the cache and return an iterable of the
expired `(key, value)` pairs.
"""
if time is None:
time = self.timer()
items = self.__items
order = self.__order
# clean up the heap if too many items are marked as removed
if len(order) > len(items) * self.__HEAP_CLEANUP_FACTOR:
self.__order = order = [item for item in order if not item.removed]
heapq.heapify(order)
expired = []
cache_delitem = Cache.__delitem__
cache_getitem = Cache.__getitem__
while order and (order[0].removed or not (time < order[0].expires)):
item = heapq.heappop(order)
if not item.removed:
expired.append((item.key, cache_getitem(self, item.key)))
cache_delitem(self, item.key)
del items[item.key]
return expired
def popitem(self):
"""Remove and return the `(key, value)` pair least recently used that
has not already expired.
"""
with self.timer as time:
self.expire(time)
try:
key = next(iter(self.__items))
except StopIteration:
raise KeyError("%s is empty" % type(self).__name__) from None
else:
return (key, self.pop(key))
def __getitem(self, key):
value = self.__items[key]
self.__items.move_to_end(key)
return value
_CacheInfo = collections.namedtuple(
"CacheInfo", ["hits", "misses", "maxsize", "currsize"]
)
def cached(cache, key=keys.hashkey, lock=None, condition=None, info=False):
"""Decorator to wrap a function with a memoizing callable that saves
results in a cache.
"""
from ._cached import _wrapper
def decorator(func):
if info:
if isinstance(cache, Cache):
def make_info(hits, misses):
return _CacheInfo(hits, misses, cache.maxsize, cache.currsize)
elif isinstance(cache, collections.abc.Mapping):
def make_info(hits, misses):
return _CacheInfo(hits, misses, None, len(cache))
else:
def make_info(hits, misses):
return _CacheInfo(hits, misses, 0, 0)
return _wrapper(func, cache, key, lock, condition, info=make_info)
else:
return _wrapper(func, cache, key, lock, condition)
return decorator
def cachedmethod(cache, key=keys.methodkey, lock=None, condition=None, info=False):
"""Decorator to wrap a class or instance method with a memoizing
callable that saves results in a cache.
"""
from ._cachedmethod import _wrapper
def decorator(method):
if info:
def make_info(cache, hits, misses):
if isinstance(cache, Cache):
return _CacheInfo(hits, misses, cache.maxsize, cache.currsize)
elif isinstance(cache, collections.abc.Mapping):
return _CacheInfo(hits, misses, None, len(cache))
else:
raise TypeError("cache(self) must return a mutable mapping")
return _wrapper(method, cache, key, lock, condition, info=make_info)
else:
return _wrapper(method, cache, key, lock, condition)
return decorator
tkem-cachetools-78b44a8/src/cachetools/_cached.py 0000664 0000000 0000000 00000015301 15142726461 0022002 0 ustar 00root root 0000000 0000000 """Function decorator helpers."""
__all__ = ()
import functools
# At least for now, the implementation prefers clarity and performance
# over ease of maintenance, thus providing separate wrappers for
# all valid combinations of decorator parameters lock, condition and
# info.
def _condition_info(func, cache, key, lock, cond, info):
hits = misses = 0
pending = set()
def wrapper(*args, **kwargs):
nonlocal hits, misses
k = key(*args, **kwargs)
with lock:
cond.wait_for(lambda: k not in pending)
try:
result = cache[k]
hits += 1
return result
except KeyError:
pending.add(k)
misses += 1
try:
v = func(*args, **kwargs)
with lock:
try:
cache[k] = v
except ValueError:
pass # value too large
return v
finally:
with lock:
pending.remove(k)
cond.notify_all()
def cache_clear():
nonlocal hits, misses
with lock:
cache.clear()
hits = misses = 0
def cache_info():
with lock:
return info(hits, misses)
wrapper.cache_clear = cache_clear
wrapper.cache_info = cache_info
return wrapper
def _locked_info(func, cache, key, lock, info):
hits = misses = 0
def wrapper(*args, **kwargs):
nonlocal hits, misses
k = key(*args, **kwargs)
with lock:
try:
result = cache[k]
hits += 1
return result
except KeyError:
misses += 1
v = func(*args, **kwargs)
with lock:
try:
# In case of a race condition, i.e. if another thread
# stored a value for this key while we were calling
# func(), prefer the cached value.
return cache.setdefault(k, v)
except ValueError:
return v # value too large
def cache_clear():
nonlocal hits, misses
with lock:
cache.clear()
hits = misses = 0
def cache_info():
with lock:
return info(hits, misses)
wrapper.cache_clear = cache_clear
wrapper.cache_info = cache_info
return wrapper
def _unlocked_info(func, cache, key, info):
hits = misses = 0
def wrapper(*args, **kwargs):
nonlocal hits, misses
k = key(*args, **kwargs)
try:
result = cache[k]
hits += 1
return result
except KeyError:
misses += 1
v = func(*args, **kwargs)
try:
cache[k] = v
except ValueError:
pass # value too large
return v
def cache_clear():
nonlocal hits, misses
cache.clear()
hits = misses = 0
def cache_info():
return info(hits, misses)
wrapper.cache_clear = cache_clear
wrapper.cache_info = cache_info
return wrapper
def _uncached_info(func, info):
misses = 0
def wrapper(*args, **kwargs):
nonlocal misses
misses += 1
return func(*args, **kwargs)
def cache_clear():
nonlocal misses
misses = 0
wrapper.cache_clear = cache_clear
wrapper.cache_info = lambda: info(0, misses)
return wrapper
def _condition(func, cache, key, lock, cond):
pending = set()
def wrapper(*args, **kwargs):
k = key(*args, **kwargs)
with lock:
cond.wait_for(lambda: k not in pending)
try:
result = cache[k]
return result
except KeyError:
pending.add(k)
try:
v = func(*args, **kwargs)
with lock:
try:
cache[k] = v
except ValueError:
pass # value too large
return v
finally:
with lock:
pending.remove(k)
cond.notify_all()
def cache_clear():
with lock:
cache.clear()
wrapper.cache_clear = cache_clear
return wrapper
def _locked(func, cache, key, lock):
def wrapper(*args, **kwargs):
k = key(*args, **kwargs)
with lock:
try:
return cache[k]
except KeyError:
pass # key not found
v = func(*args, **kwargs)
with lock:
try:
# In case of a race condition, i.e. if another thread
# stored a value for this key while we were calling
# func(), prefer the cached value.
return cache.setdefault(k, v)
except ValueError:
return v # value too large
def cache_clear():
with lock:
cache.clear()
wrapper.cache_clear = cache_clear
return wrapper
def _unlocked(func, cache, key):
def wrapper(*args, **kwargs):
k = key(*args, **kwargs)
try:
return cache[k]
except KeyError:
pass # key not found
v = func(*args, **kwargs)
try:
cache[k] = v
except ValueError:
pass # value too large
return v
wrapper.cache_clear = lambda: cache.clear()
return wrapper
def _uncached(func):
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
wrapper.cache_clear = lambda: None
return wrapper
def _wrapper(func, cache, key, lock=None, cond=None, info=None):
if info is not None:
if cache is None:
wrapper = _uncached_info(func, info)
elif cond is not None and lock is not None:
wrapper = _condition_info(func, cache, key, lock, cond, info)
elif cond is not None:
wrapper = _condition_info(func, cache, key, cond, cond, info)
elif lock is not None:
wrapper = _locked_info(func, cache, key, lock, info)
else:
wrapper = _unlocked_info(func, cache, key, info)
else:
if cache is None:
wrapper = _uncached(func)
elif cond is not None and lock is not None:
wrapper = _condition(func, cache, key, lock, cond)
elif cond is not None:
wrapper = _condition(func, cache, key, cond, cond)
elif lock is not None:
wrapper = _locked(func, cache, key, lock)
else:
wrapper = _unlocked(func, cache, key)
wrapper.cache_info = None
wrapper.cache = cache
wrapper.cache_key = key
wrapper.cache_lock = lock if lock is not None else cond
wrapper.cache_condition = cond
return functools.update_wrapper(wrapper, func)
tkem-cachetools-78b44a8/src/cachetools/_cachedmethod.py 0000664 0000000 0000000 00000033066 15142726461 0023213 0 ustar 00root root 0000000 0000000 """Method decorator helpers."""
__all__ = ()
import functools
import warnings
import weakref
def _warn_classmethod(stacklevel):
warnings.warn(
"decorating class methods with @cachedmethod is deprecated",
DeprecationWarning,
stacklevel=stacklevel,
)
def _warn_instance_dict(msg, stacklevel):
warnings.warn(
msg,
DeprecationWarning,
stacklevel=stacklevel,
)
class _WrapperBase:
"""Wrapper base class providing default implementations for properties."""
def __init__(self, obj, method, cache, key, lock=None, cond=None):
if isinstance(obj, type):
_warn_classmethod(stacklevel=5)
functools.update_wrapper(self, method)
self._obj = obj # protected
self.__cache = cache
self.__key = key
self.__lock = lock
self.__cond = cond
def __call__(self, *args, **kwargs):
raise NotImplementedError() # pragma: no cover
def cache_clear(self):
raise NotImplementedError() # pragma: no cover
@property
def cache(self):
return self.__cache(self._obj)
@property
def cache_key(self):
return self.__key
@property
def cache_lock(self):
return None if self.__lock is None else self.__lock(self._obj)
@property
def cache_condition(self):
return None if self.__cond is None else self.__cond(self._obj)
class _DescriptorBase:
"""Descriptor base class implementing the basic descriptor protocol."""
def __init__(self, deprecated=False):
self.__attrname = None
self.__deprecated = deprecated
def __set_name__(self, owner, name):
if self.__attrname is None:
self.__attrname = name
elif name != self.__attrname:
raise TypeError(
"Cannot assign the same @cachedmethod to two different names "
f"({self.__attrname!r} and {name!r})."
)
def __get__(self, obj, objtype=None):
wrapper = self.Wrapper(obj)
if self.__attrname is not None:
# replace descriptor instance with wrapper in instance dict
try:
# In case of a race condition where another thread already replaced
# the descriptor, prefer the initial wrapper.
wrapper = obj.__dict__.setdefault(self.__attrname, wrapper)
except AttributeError:
# not all objects have __dict__ (e.g. class defines slots)
msg = (
f"No '__dict__' attribute on {type(obj).__name__!r} "
f"instance to cache {self.__attrname!r} property."
)
if self.__deprecated:
_warn_instance_dict(msg, 3)
else:
raise TypeError(msg) from None
except TypeError:
msg = (
f"The '__dict__' attribute on {type(obj).__name__!r} "
f"instance does not support item assignment for "
f"caching {self.__attrname!r} property."
)
if self.__deprecated:
_warn_instance_dict(msg, 3)
else:
raise TypeError(msg) from None
elif self.__deprecated:
pass # deprecated @classmethod, warning already raised elsewhere
else:
msg = "Cannot use @cachedmethod instance without calling __set_name__ on it"
raise TypeError(msg) from None
return wrapper
class _DeprecatedDescriptorBase(_DescriptorBase):
"""Descriptor base class supporting deprecated @classmethod use."""
def __init__(self, wrapper, cache_clear):
super().__init__(deprecated=True)
self.__wrapper = wrapper
self.__cache_clear = cache_clear
# called for @classmethod with Python >= 3.13
def __call__(self, *args, **kwargs):
_warn_classmethod(stacklevel=3)
return self.__wrapper(*args, **kwargs)
# backward-compatible @classmethod handling with Python >= 3.13
def cache_clear(self, objtype):
_warn_classmethod(stacklevel=3)
return self.__cache_clear(objtype)
# At least for now, the implementation prefers clarity and performance
# over ease of maintenance, thus providing separate descriptors for
# all valid combinations of decorator parameters lock, condition and
# info.
def _condition_info(method, cache, key, lock, cond, info):
class Descriptor(_DescriptorBase):
class Wrapper(_WrapperBase):
def __init__(self, obj):
super().__init__(obj, method, cache, key, lock, cond)
self.__hits = self.__misses = 0
self.__pending = set()
def __call__(self, *args, **kwargs):
cache = self.cache
lock = self.cache_lock
cond = self.cache_condition
key = self.cache_key(self._obj, *args, **kwargs)
with lock:
cond.wait_for(lambda: key not in self.__pending)
try:
result = cache[key]
self.__hits += 1
return result
except KeyError:
self.__pending.add(key)
self.__misses += 1
try:
val = method(self._obj, *args, **kwargs)
with lock:
try:
cache[key] = val
except ValueError:
pass # value too large
return val
finally:
with lock:
self.__pending.remove(key)
cond.notify_all()
def cache_clear(self):
with self.cache_lock:
self.cache.clear()
self.__hits = self.__misses = 0
def cache_info(self):
with self.cache_lock:
return info(self.cache, self.__hits, self.__misses)
return Descriptor()
def _locked_info(method, cache, key, lock, info):
class Descriptor(_DescriptorBase):
class Wrapper(_WrapperBase):
def __init__(self, obj):
super().__init__(obj, method, cache, key, lock)
self.__hits = self.__misses = 0
def __call__(self, *args, **kwargs):
cache = self.cache
lock = self.cache_lock
key = self.cache_key(self._obj, *args, **kwargs)
with lock:
try:
result = cache[key]
self.__hits += 1
return result
except KeyError:
self.__misses += 1
val = method(self._obj, *args, **kwargs)
with lock:
try:
# In case of a race condition, i.e. if another thread
# stored a value for this key while we were calling
# method(), prefer the cached value.
return cache.setdefault(key, val)
except ValueError:
return val # value too large
def cache_clear(self):
with self.cache_lock:
self.cache.clear()
self.__hits = self.__misses = 0
def cache_info(self):
with self.cache_lock:
return info(self.cache, self.__hits, self.__misses)
return Descriptor()
def _unlocked_info(method, cache, key, info):
class Descriptor(_DescriptorBase):
class Wrapper(_WrapperBase):
def __init__(self, obj):
super().__init__(obj, method, cache, key)
self.__hits = self.__misses = 0
def __call__(self, *args, **kwargs):
cache = self.cache
key = self.cache_key(self._obj, *args, **kwargs)
try:
result = cache[key]
self.__hits += 1
return result
except KeyError:
self.__misses += 1
val = method(self._obj, *args, **kwargs)
try:
cache[key] = val
except ValueError:
pass # value too large
return val
def cache_clear(self):
self.cache.clear()
self.__hits = self.__misses = 0
def cache_info(self):
return info(self.cache, self.__hits, self.__misses)
return Descriptor()
def _condition(method, cache, key, lock, cond):
# backward-compatible weakref dictionary for Python >= 3.13
pending = weakref.WeakKeyDictionary()
def wrapper(self, pending, *args, **kwargs):
c = cache(self)
k = key(self, *args, **kwargs)
with lock(self):
cond(self).wait_for(lambda: k not in pending)
try:
return c[k]
except KeyError:
pending.add(k)
try:
v = method(self, *args, **kwargs)
with lock(self):
try:
c[k] = v
except ValueError:
pass # value too large
return v
finally:
with lock(self):
pending.remove(k)
cond(self).notify_all()
def cache_clear(self):
c = cache(self)
with lock(self):
c.clear()
def classmethod_wrapper(self, *args, **kwargs):
p = pending.setdefault(self, set())
return wrapper(self, p, *args, **kwargs)
class Descriptor(_DeprecatedDescriptorBase):
class Wrapper(_WrapperBase):
def __init__(self, obj):
super().__init__(obj, method, cache, key, lock, cond)
self.__pending = set()
def __call__(self, *args, **kwargs):
return wrapper(self._obj, self.__pending, *args, **kwargs)
# objtype: backward-compatible @classmethod handling with Python < 3.13
def cache_clear(self, _objtype=None):
return cache_clear(self._obj)
return Descriptor(classmethod_wrapper, cache_clear)
def _locked(method, cache, key, lock):
def wrapper(self, *args, **kwargs):
c = cache(self)
k = key(self, *args, **kwargs)
with lock(self):
try:
return c[k]
except KeyError:
pass # key not found
v = method(self, *args, **kwargs)
with lock(self):
try:
# In case of a race condition, i.e. if another thread
# stored a value for this key while we were calling
# method(), prefer the cached value.
return c.setdefault(k, v)
except ValueError:
return v # value too large
def cache_clear(self):
c = cache(self)
with lock(self):
c.clear()
class Descriptor(_DeprecatedDescriptorBase):
class Wrapper(_WrapperBase):
def __init__(self, obj):
super().__init__(obj, method, cache, key, lock)
def __call__(self, *args, **kwargs):
return wrapper(self._obj, *args, **kwargs)
# objtype: backward-compatible @classmethod handling with Python < 3.13
def cache_clear(self, _objtype=None):
return cache_clear(self._obj)
return Descriptor(wrapper, cache_clear)
def _unlocked(method, cache, key):
def wrapper(self, *args, **kwargs):
c = cache(self)
k = key(self, *args, **kwargs)
try:
return c[k]
except KeyError:
pass # key not found
v = method(self, *args, **kwargs)
try:
c[k] = v
except ValueError:
pass # value too large
return v
def cache_clear(self):
c = cache(self)
c.clear()
class Descriptor(_DeprecatedDescriptorBase):
class Wrapper(_WrapperBase):
def __init__(self, obj):
super().__init__(obj, method, cache, key)
def __call__(self, *args, **kwargs):
return wrapper(self._obj, *args, **kwargs)
# objtype: backward-compatible @classmethod handling with Python < 3.13
def cache_clear(self, _objtype=None):
return cache_clear(self._obj)
return Descriptor(wrapper, cache_clear)
def _wrapper(method, cache, key, lock=None, cond=None, info=None):
if info is not None:
if cond is not None and lock is not None:
wrapper = _condition_info(method, cache, key, lock, cond, info)
elif cond is not None:
wrapper = _condition_info(method, cache, key, cond, cond, info)
elif lock is not None:
wrapper = _locked_info(method, cache, key, lock, info)
else:
wrapper = _unlocked_info(method, cache, key, info)
else:
if cond is not None and lock is not None:
wrapper = _condition(method, cache, key, lock, cond)
elif cond is not None:
wrapper = _condition(method, cache, key, cond, cond)
elif lock is not None:
wrapper = _locked(method, cache, key, lock)
else:
wrapper = _unlocked(method, cache, key)
# backward-compatible properties for deprecated @classmethod use
wrapper.cache = cache
wrapper.cache_key = key
wrapper.cache_lock = lock if lock is not None else cond
wrapper.cache_condition = cond
return functools.update_wrapper(wrapper, method)
tkem-cachetools-78b44a8/src/cachetools/func.py 0000664 0000000 0000000 00000006326 15142726461 0021376 0 ustar 00root root 0000000 0000000 """`functools.lru_cache` compatible memoizing function decorators."""
__all__ = ("fifo_cache", "lfu_cache", "lru_cache", "rr_cache", "ttl_cache")
import functools
import math
import random
import time
from threading import Condition
from . import FIFOCache, LFUCache, LRUCache, RRCache, TTLCache
from . import cached
from . import keys
class _UnboundTTLCache(TTLCache):
def __init__(self, ttl, timer):
TTLCache.__init__(self, math.inf, ttl, timer)
@property
def maxsize(self):
return None
def _cache(cache, maxsize, typed):
def decorator(func):
# like functools.lru_cache, this has to be thread-safe;
# additionally, this also prevents cache stampede scenarios
# using a condition variable
key = keys.typedkey if typed else keys.hashkey
wrapper = cached(cache=cache, key=key, condition=Condition(), info=True)(func)
wrapper.cache_parameters = lambda: {"maxsize": maxsize, "typed": typed}
return wrapper
return decorator
def fifo_cache(maxsize=128, typed=False):
"""Decorator to wrap a function with a memoizing callable that saves
up to `maxsize` results based on a First In First Out (FIFO)
algorithm.
"""
if maxsize is None:
return _cache({}, None, typed)
elif callable(maxsize):
return _cache(FIFOCache(128), 128, typed)(maxsize)
else:
return _cache(FIFOCache(maxsize), maxsize, typed)
def lfu_cache(maxsize=128, typed=False):
"""Decorator to wrap a function with a memoizing callable that saves
up to `maxsize` results based on a Least Frequently Used (LFU)
algorithm.
"""
if maxsize is None:
return _cache({}, None, typed)
elif callable(maxsize):
return _cache(LFUCache(128), 128, typed)(maxsize)
else:
return _cache(LFUCache(maxsize), maxsize, typed)
def lru_cache(maxsize=128, typed=False):
"""Decorator to wrap a function with a memoizing callable that saves
up to `maxsize` results based on a Least Recently Used (LRU)
algorithm.
"""
if maxsize is None:
return _cache({}, None, typed)
elif callable(maxsize):
return _cache(LRUCache(128), 128, typed)(maxsize)
else:
return _cache(LRUCache(maxsize), maxsize, typed)
def rr_cache(maxsize=128, choice=random.choice, typed=False):
"""Decorator to wrap a function with a memoizing callable that saves
up to `maxsize` results based on a Random Replacement (RR)
algorithm.
"""
if maxsize is None:
return _cache({}, None, typed)
elif callable(maxsize):
return _cache(RRCache(128, choice), 128, typed)(maxsize)
else:
return _cache(RRCache(maxsize, choice), maxsize, typed)
def ttl_cache(maxsize=128, ttl=600, timer=time.monotonic, typed=False):
"""Decorator to wrap a function with a memoizing callable that saves
up to `maxsize` results based on a Least Recently Used (LRU)
algorithm with a per-item time-to-live (TTL) value.
"""
if maxsize is None:
return _cache(_UnboundTTLCache(ttl, timer), None, typed)
elif callable(maxsize):
return _cache(TTLCache(128, ttl, timer), 128, typed)(maxsize)
else:
return _cache(TTLCache(maxsize, ttl, timer), maxsize, typed)
tkem-cachetools-78b44a8/src/cachetools/keys.py 0000664 0000000 0000000 00000003657 15142726461 0021422 0 ustar 00root root 0000000 0000000 """Key functions for memoizing decorators."""
__all__ = ("hashkey", "methodkey", "typedkey", "typedmethodkey")
class _HashedTuple(tuple):
"""A tuple that ensures that hash() will be called no more than once
per element, since cache decorators will hash the key multiple
times on a cache miss. See also _HashedSeq in the standard
library functools implementation.
"""
__hashvalue = None # default value, set in instance on first use
def __hash__(self, hash=tuple.__hash__):
hashvalue = self.__hashvalue
if hashvalue is None:
self.__hashvalue = hashvalue = hash(self)
return hashvalue
def __add__(self, other, add=tuple.__add__):
return _HashedTuple(add(self, other))
def __radd__(self, other, add=tuple.__add__):
return _HashedTuple(add(other, self))
def __getstate__(self):
return {}
# A sentinel for separating args from kwargs. Using the class itself
# ensures uniqueness and preserves identity when pickling/unpickling.
_kwmark = (_HashedTuple,)
def hashkey(*args, **kwargs):
"""Return a cache key for the specified hashable arguments."""
if kwargs:
return _HashedTuple(args + _kwmark + tuple(sorted(kwargs.items())))
else:
return _HashedTuple(args)
def methodkey(self, *args, **kwargs):
"""Return a cache key for use with cached methods."""
return hashkey(*args, **kwargs)
def typedkey(*args, **kwargs):
"""Return a typed cache key for the specified hashable arguments."""
if kwargs:
sorted_kwargs = tuple(sorted(kwargs.items()))
key = _HashedTuple(args + _kwmark + sorted_kwargs)
key += tuple(type(v) for _, v in sorted_kwargs)
else:
key = _HashedTuple(args)
key += tuple(type(v) for v in args)
return key
def typedmethodkey(self, *args, **kwargs):
"""Return a typed cache key for use with cached methods."""
return typedkey(*args, **kwargs)
tkem-cachetools-78b44a8/tests/ 0000775 0000000 0000000 00000000000 15142726461 0016311 5 ustar 00root root 0000000 0000000 tkem-cachetools-78b44a8/tests/__init__.py 0000664 0000000 0000000 00000024206 15142726461 0020426 0 ustar 00root root 0000000 0000000 import unittest
class CacheTestMixin:
Cache = None
def test_defaults(self):
cache = self.Cache(maxsize=1)
self.assertEqual(0, len(cache))
self.assertEqual(1, cache.maxsize)
self.assertEqual(0, cache.currsize)
self.assertEqual(1, cache.getsizeof(None))
self.assertEqual(1, cache.getsizeof(""))
self.assertEqual(1, cache.getsizeof(0))
self.assertTrue(repr(cache).startswith(cache.__class__.__name__))
def test_insert(self):
cache = self.Cache(maxsize=2)
cache.update({1: 1, 2: 2})
self.assertEqual(2, len(cache))
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
cache[3] = 3
self.assertEqual(2, len(cache))
self.assertEqual(3, cache[3])
self.assertTrue(1 in cache or 2 in cache)
cache[4] = 4
self.assertEqual(2, len(cache))
self.assertEqual(4, cache[4])
self.assertTrue(1 in cache or 2 in cache or 3 in cache)
def test_update(self):
cache = self.Cache(maxsize=2)
cache.update({1: 1, 2: 2})
self.assertEqual(2, len(cache))
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
cache.update({1: 1, 2: 2})
self.assertEqual(2, len(cache))
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
cache.update({1: "a", 2: "b"})
self.assertEqual(2, len(cache))
self.assertEqual("a", cache[1])
self.assertEqual("b", cache[2])
def test_delete(self):
cache = self.Cache(maxsize=2)
cache.update({1: 1, 2: 2})
self.assertEqual(2, len(cache))
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
del cache[2]
self.assertEqual(1, len(cache))
self.assertEqual(1, cache[1])
self.assertNotIn(2, cache)
del cache[1]
self.assertEqual(0, len(cache))
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
with self.assertRaises(KeyError):
del cache[1]
self.assertEqual(0, len(cache))
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
def test_pop(self):
cache = self.Cache(maxsize=2)
cache.update({1: 1, 2: 2})
self.assertEqual(2, cache.pop(2))
self.assertEqual(1, len(cache))
self.assertEqual(1, cache.pop(1))
self.assertEqual(0, len(cache))
with self.assertRaises(KeyError):
cache.pop(2)
with self.assertRaises(KeyError):
cache.pop(1)
with self.assertRaises(KeyError):
cache.pop(0)
self.assertEqual(None, cache.pop(2, None))
self.assertEqual(None, cache.pop(1, None))
self.assertEqual(None, cache.pop(0, None))
def test_popitem(self):
cache = self.Cache(maxsize=2)
cache.update({1: 1, 2: 2})
key, _ = cache.popitem()
self.assertIn(key, {1, 2})
self.assertEqual(1, len(cache))
key, _ = cache.popitem()
self.assertIn(key, {1, 2})
self.assertEqual(0, len(cache))
with self.assertRaises(KeyError):
cache.popitem()
def test_popitem_exception_context(self):
# since Python 3.7, MutableMapping.popitem() suppresses
# exception context as implementation detail
exception = None
try:
self.Cache(maxsize=2).popitem()
except Exception as e:
exception = e
self.assertIsNone(exception.__cause__)
self.assertTrue(exception.__suppress_context__)
def test_missing(self):
class DefaultCache(self.Cache):
def __missing__(self, key):
self[key] = key
return key
cache = DefaultCache(maxsize=2)
self.assertEqual(0, cache.currsize)
self.assertEqual(2, cache.maxsize)
self.assertEqual(0, len(cache))
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
self.assertEqual(2, len(cache))
self.assertTrue(1 in cache and 2 in cache)
self.assertEqual(3, cache[3])
self.assertEqual(2, len(cache))
self.assertTrue(3 in cache)
self.assertTrue(1 in cache or 2 in cache)
self.assertTrue(1 not in cache or 2 not in cache)
self.assertEqual(4, cache[4])
self.assertEqual(2, len(cache))
self.assertTrue(4 in cache)
self.assertTrue(1 in cache or 2 in cache or 3 in cache)
# verify __missing__() is *not* called for any operations
# besides __getitem__()
self.assertEqual(4, cache.get(4))
self.assertEqual(None, cache.get(5))
self.assertEqual(5 * 5, cache.get(5, 5 * 5))
self.assertEqual(2, len(cache))
self.assertEqual(4, cache.pop(4))
with self.assertRaises(KeyError):
cache.pop(5)
self.assertEqual(None, cache.pop(5, None))
self.assertEqual(5 * 5, cache.pop(5, 5 * 5))
self.assertEqual(1, len(cache))
cache.clear()
cache[1] = 1 + 1
self.assertEqual(1 + 1, cache.setdefault(1))
self.assertEqual(1 + 1, cache.setdefault(1, 1))
self.assertEqual(1 + 1, cache[1])
self.assertEqual(2 + 2, cache.setdefault(2, 2 + 2))
self.assertEqual(2 + 2, cache.setdefault(2, None))
self.assertEqual(2 + 2, cache.setdefault(2))
self.assertEqual(2 + 2, cache[2])
self.assertEqual(2, len(cache))
self.assertTrue(1 in cache and 2 in cache)
self.assertEqual(None, cache.setdefault(3))
self.assertEqual(2, len(cache))
self.assertTrue(3 in cache)
self.assertTrue(1 in cache or 2 in cache)
self.assertTrue(1 not in cache or 2 not in cache)
def test_missing_getsizeof(self):
class DefaultCache(self.Cache):
def __missing__(self, key):
try:
self[key] = key
except ValueError:
pass # not stored
return key
cache = DefaultCache(maxsize=2, getsizeof=lambda x: x)
self.assertEqual(0, cache.currsize)
self.assertEqual(2, cache.maxsize)
self.assertEqual(1, cache[1])
self.assertEqual(1, len(cache))
self.assertEqual(1, cache.currsize)
self.assertIn(1, cache)
self.assertEqual(2, cache[2])
self.assertEqual(1, len(cache))
self.assertEqual(2, cache.currsize)
self.assertNotIn(1, cache)
self.assertIn(2, cache)
self.assertEqual(3, cache[3]) # not stored
self.assertEqual(1, len(cache))
self.assertEqual(2, cache.currsize)
self.assertEqual(1, cache[1])
self.assertEqual(1, len(cache))
self.assertEqual(1, cache.currsize)
self.assertEqual((1, 1), cache.popitem())
def _test_getsizeof(self, cache):
self.assertEqual(0, cache.currsize)
self.assertEqual(3, cache.maxsize)
self.assertEqual(1, cache.getsizeof(1))
self.assertEqual(2, cache.getsizeof(2))
self.assertEqual(3, cache.getsizeof(3))
cache.update({1: 1, 2: 2})
self.assertEqual(2, len(cache))
self.assertEqual(3, cache.currsize)
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
cache[1] = 2
self.assertEqual(1, len(cache))
self.assertEqual(2, cache.currsize)
self.assertEqual(2, cache[1])
self.assertNotIn(2, cache)
cache.update({1: 1, 2: 2})
self.assertEqual(2, len(cache))
self.assertEqual(3, cache.currsize)
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
cache[3] = 3
self.assertEqual(1, len(cache))
self.assertEqual(3, cache.currsize)
self.assertEqual(3, cache[3])
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
with self.assertRaises(ValueError):
cache[3] = 4
self.assertEqual(1, len(cache))
self.assertEqual(3, cache.currsize)
self.assertEqual(3, cache[3])
with self.assertRaises(ValueError):
cache[4] = 4
self.assertEqual(1, len(cache))
self.assertEqual(3, cache.currsize)
self.assertEqual(3, cache[3])
def test_getsizeof_param(self):
self._test_getsizeof(self.Cache(maxsize=3, getsizeof=lambda x: x))
def test_getsizeof_subclass(self):
class Cache(self.Cache):
def getsizeof(self, value):
return value
self._test_getsizeof(Cache(maxsize=3))
def test_pickle(self):
import pickle
source = self.Cache(maxsize=2)
source.update({1: 1, 2: 2})
cache = pickle.loads(pickle.dumps(source))
self.assertEqual(source, cache)
self.assertEqual(2, len(cache))
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
cache[3] = 3
self.assertEqual(2, len(cache))
self.assertEqual(3, cache[3])
self.assertTrue(1 in cache or 2 in cache)
cache[4] = 4
self.assertEqual(2, len(cache))
self.assertEqual(4, cache[4])
self.assertTrue(1 in cache or 2 in cache or 3 in cache)
self.assertEqual(cache, pickle.loads(pickle.dumps(cache)))
def test_pickle_maxsize(self):
import pickle
import sys
# test empty cache, single element, large cache (recursion limit)
for n in [0, 1, sys.getrecursionlimit() * 2]:
source = self.Cache(maxsize=n)
source.update((i, i) for i in range(n))
cache = pickle.loads(pickle.dumps(source))
self.assertEqual(n, len(cache))
self.assertEqual(source, cache)
class CountedLock:
def __init__(self):
self.count = 0
def __enter__(self):
self.count += 1
def __exit__(self, *exc):
pass
class CountedCondition(CountedLock):
def __init__(self):
super().__init__()
self.wait_count = 0
self.notify_count = 0
# only wait_for() and notify_all() are used in the cache tests,
# calling wait() or notify() will fail intentionally
def wait_for(self, predicate):
assert callable(predicate)
self.wait_count += 1
def notify_all(self):
self.notify_count += 1
tkem-cachetools-78b44a8/tests/test_cache.py 0000664 0000000 0000000 00000000224 15142726461 0020763 0 ustar 00root root 0000000 0000000 import unittest
import cachetools
from . import CacheTestMixin
class CacheTest(unittest.TestCase, CacheTestMixin):
Cache = cachetools.Cache
tkem-cachetools-78b44a8/tests/test_cached.py 0000664 0000000 0000000 00000035000 15142726461 0021127 0 ustar 00root root 0000000 0000000 import unittest
import warnings
import cachetools
import cachetools.keys
from . import CountedCondition, CountedLock
class DecoratorTestMixin:
def cache(self, minsize):
raise NotImplementedError
def func(self, *args, **kwargs):
if hasattr(self, "count"):
self.count += 1
else:
self.count = 0
return self.count
def test_decorator(self):
cache = self.cache(2)
wrapper = cachetools.cached(cache)(self.func)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper(0), 0)
self.assertEqual(len(cache), 1)
self.assertIn(cachetools.keys.hashkey(0), cache)
self.assertNotIn(cachetools.keys.hashkey(1), cache)
self.assertNotIn(cachetools.keys.hashkey(1.0), cache)
self.assertEqual(wrapper(1), 1)
self.assertEqual(len(cache), 2)
self.assertIn(cachetools.keys.hashkey(0), cache)
self.assertIn(cachetools.keys.hashkey(1), cache)
self.assertIn(cachetools.keys.hashkey(1.0), cache)
self.assertEqual(wrapper(1), 1)
self.assertEqual(len(cache), 2)
self.assertEqual(wrapper(1.0), 1)
self.assertEqual(len(cache), 2)
self.assertEqual(wrapper(1.0), 1)
self.assertEqual(len(cache), 2)
def test_decorator_typed(self):
cache = self.cache(3)
key = cachetools.keys.typedkey
wrapper = cachetools.cached(cache, key=key)(self.func)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper(0), 0)
self.assertEqual(len(cache), 1)
self.assertIn(cachetools.keys.typedkey(0), cache)
self.assertNotIn(cachetools.keys.typedkey(1), cache)
self.assertNotIn(cachetools.keys.typedkey(1.0), cache)
self.assertEqual(wrapper(1), 1)
self.assertEqual(len(cache), 2)
self.assertIn(cachetools.keys.typedkey(0), cache)
self.assertIn(cachetools.keys.typedkey(1), cache)
self.assertNotIn(cachetools.keys.typedkey(1.0), cache)
self.assertEqual(wrapper(1), 1)
self.assertEqual(len(cache), 2)
self.assertEqual(wrapper(1.0), 2)
self.assertEqual(len(cache), 3)
self.assertIn(cachetools.keys.typedkey(0), cache)
self.assertIn(cachetools.keys.typedkey(1), cache)
self.assertIn(cachetools.keys.typedkey(1.0), cache)
self.assertEqual(wrapper(1.0), 2)
self.assertEqual(len(cache), 3)
def test_decorator_lock(self):
cache = self.cache(2)
lock = CountedLock()
wrapper = cachetools.cached(cache, lock=lock)(self.func)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper(0), 0)
self.assertEqual(lock.count, 2)
self.assertEqual(wrapper(1), 1)
self.assertEqual(lock.count, 4)
self.assertEqual(wrapper(1), 1)
self.assertEqual(lock.count, 5)
def test_decorator_condition(self):
cache = self.cache(2)
lock = cond = CountedCondition()
wrapper = cachetools.cached(cache, condition=cond)(self.func)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper(0), 0)
self.assertEqual(lock.count, 3)
self.assertEqual(cond.wait_count, 1)
self.assertEqual(cond.notify_count, 1)
self.assertEqual(wrapper(1), 1)
self.assertEqual(lock.count, 6)
self.assertEqual(cond.wait_count, 2)
self.assertEqual(cond.notify_count, 2)
self.assertEqual(wrapper(1), 1)
self.assertEqual(lock.count, 7)
self.assertEqual(cond.wait_count, 3)
self.assertEqual(cond.notify_count, 2)
def test_decorator_lock_condition(self):
cache = self.cache(2)
lock = CountedLock()
cond = CountedCondition()
wrapper = cachetools.cached(cache, lock=lock, condition=cond)(self.func)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper(0), 0)
self.assertEqual(lock.count, 3)
self.assertEqual(cond.wait_count, 1)
self.assertEqual(cond.notify_count, 1)
self.assertEqual(wrapper(1), 1)
self.assertEqual(lock.count, 6)
self.assertEqual(cond.wait_count, 2)
self.assertEqual(cond.notify_count, 2)
self.assertEqual(wrapper(1), 1)
self.assertEqual(lock.count, 7)
self.assertEqual(cond.wait_count, 3)
self.assertEqual(cond.notify_count, 2)
def test_decorator_wrapped(self):
cache = self.cache(2)
wrapper = cachetools.cached(cache)(self.func)
self.assertEqual(wrapper.__wrapped__, self.func)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper.__wrapped__(0), 0)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper(0), 1)
self.assertEqual(len(cache), 1)
self.assertEqual(wrapper(0), 1)
self.assertEqual(len(cache), 1)
def test_decorator_attributes(self):
cache = self.cache(2)
wrapper = cachetools.cached(cache)(self.func)
self.assertIs(wrapper.cache, cache)
self.assertIs(wrapper.cache_key, cachetools.keys.hashkey)
self.assertIs(wrapper.cache_lock, None)
self.assertIs(wrapper.cache_condition, None)
def test_decorator_attributes_lock(self):
cache = self.cache(2)
lock = CountedLock()
wrapper = cachetools.cached(cache, lock=lock)(self.func)
self.assertIs(wrapper.cache, cache)
self.assertIs(wrapper.cache_key, cachetools.keys.hashkey)
self.assertIs(wrapper.cache_lock, lock)
self.assertIs(wrapper.cache_condition, None)
def test_decorator_attributes_condition(self):
cache = self.cache(2)
lock = cond = CountedCondition()
wrapper = cachetools.cached(cache, condition=cond)(self.func)
self.assertIs(wrapper.cache, cache)
self.assertIs(wrapper.cache_key, cachetools.keys.hashkey)
self.assertIs(wrapper.cache_lock, lock)
self.assertIs(wrapper.cache_condition, cond)
def test_decorator_clear(self):
cache = self.cache(2)
wrapper = cachetools.cached(cache)(self.func)
self.assertEqual(wrapper(0), 0)
self.assertEqual(len(cache), 1)
wrapper.cache_clear()
self.assertEqual(len(cache), 0)
def test_decorator_clear_lock(self):
cache = self.cache(2)
lock = CountedLock()
wrapper = cachetools.cached(cache, lock=lock)(self.func)
self.assertEqual(wrapper(0), 0)
self.assertEqual(len(cache), 1)
self.assertEqual(lock.count, 2)
wrapper.cache_clear()
self.assertEqual(len(cache), 0)
self.assertEqual(lock.count, 3)
def test_decorator_clear_condition(self):
cache = self.cache(2)
lock = cond = CountedCondition()
wrapper = cachetools.cached(cache, condition=cond)(self.func)
self.assertEqual(wrapper(0), 0)
self.assertEqual(len(cache), 1)
self.assertEqual(lock.count, 3)
wrapper.cache_clear()
self.assertEqual(len(cache), 0)
self.assertEqual(lock.count, 4)
class CacheWrapperTest(unittest.TestCase, DecoratorTestMixin):
def cache(self, minsize):
return cachetools.Cache(maxsize=minsize)
def test_decorator_info(self):
cache = self.cache(2)
wrapper = cachetools.cached(cache, info=True)(self.func)
self.assertEqual(wrapper.cache_info(), (0, 0, 2, 0))
self.assertEqual(wrapper(0), 0)
self.assertEqual(wrapper.cache_info(), (0, 1, 2, 1))
self.assertEqual(wrapper(1), 1)
self.assertEqual(wrapper.cache_info(), (0, 2, 2, 2))
self.assertEqual(wrapper(0), 0)
self.assertEqual(wrapper.cache_info(), (1, 2, 2, 2))
wrapper.cache_clear()
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper.cache_info(), (0, 0, 2, 0))
def test_decorator_lock_info(self):
cache = self.cache(2)
lock = CountedLock()
wrapper = cachetools.cached(cache, lock=lock, info=True)(self.func)
self.assertEqual(wrapper.cache_info(), (0, 0, 2, 0))
self.assertEqual(lock.count, 1)
self.assertEqual(wrapper(0), 0)
self.assertEqual(lock.count, 3)
self.assertEqual(wrapper.cache_info(), (0, 1, 2, 1))
self.assertEqual(lock.count, 4)
self.assertEqual(wrapper(1), 1)
self.assertEqual(lock.count, 6)
self.assertEqual(wrapper.cache_info(), (0, 2, 2, 2))
self.assertEqual(lock.count, 7)
self.assertEqual(wrapper(0), 0)
self.assertEqual(lock.count, 8)
self.assertEqual(wrapper.cache_info(), (1, 2, 2, 2))
self.assertEqual(lock.count, 9)
wrapper.cache_clear()
self.assertEqual(lock.count, 10)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper.cache_info(), (0, 0, 2, 0))
self.assertEqual(lock.count, 11)
def test_decorator_condition_info(self):
cache = self.cache(2)
lock = cond = CountedCondition()
wrapper = cachetools.cached(cache, condition=cond, info=True)(self.func)
self.assertEqual(wrapper.cache_info(), (0, 0, 2, 0))
self.assertEqual(lock.count, 1)
self.assertEqual(wrapper(0), 0)
self.assertEqual(lock.count, 4)
self.assertEqual(wrapper.cache_info(), (0, 1, 2, 1))
self.assertEqual(lock.count, 5)
self.assertEqual(wrapper(1), 1)
self.assertEqual(lock.count, 8)
self.assertEqual(wrapper.cache_info(), (0, 2, 2, 2))
self.assertEqual(lock.count, 9)
self.assertEqual(wrapper(0), 0)
self.assertEqual(lock.count, 10)
self.assertEqual(wrapper.cache_info(), (1, 2, 2, 2))
self.assertEqual(lock.count, 11)
wrapper.cache_clear()
self.assertEqual(lock.count, 12)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper.cache_info(), (0, 0, 2, 0))
self.assertEqual(lock.count, 13)
def test_decorator_lock_condition_info(self):
cache = self.cache(2)
lock = CountedLock()
cond = CountedCondition()
wrapper = cachetools.cached(cache, lock=lock, condition=cond, info=True)(
self.func
)
self.assertEqual(wrapper.cache_info(), (0, 0, 2, 0))
self.assertEqual(lock.count, 1)
self.assertEqual(wrapper(0), 0)
self.assertEqual(lock.count, 4)
self.assertEqual(wrapper.cache_info(), (0, 1, 2, 1))
self.assertEqual(lock.count, 5)
self.assertEqual(wrapper(1), 1)
self.assertEqual(lock.count, 8)
self.assertEqual(wrapper.cache_info(), (0, 2, 2, 2))
self.assertEqual(lock.count, 9)
self.assertEqual(wrapper(0), 0)
self.assertEqual(lock.count, 10)
self.assertEqual(wrapper.cache_info(), (1, 2, 2, 2))
self.assertEqual(lock.count, 11)
wrapper.cache_clear()
self.assertEqual(lock.count, 12)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper.cache_info(), (0, 0, 2, 0))
self.assertEqual(lock.count, 13)
def test_zero_size_cache_decorator(self):
cache = self.cache(0)
wrapper = cachetools.cached(cache)(self.func)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper(0), 0)
self.assertEqual(len(cache), 0)
def test_zero_size_cache_decorator_lock(self):
cache = self.cache(0)
lock = CountedLock()
wrapper = cachetools.cached(cache, lock=lock)(self.func)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper(0), 0)
self.assertEqual(len(cache), 0)
self.assertEqual(lock.count, 2)
def test_zero_size_cache_decorator_condition(self):
cache = self.cache(0)
lock = cond = CountedCondition()
wrapper = cachetools.cached(cache, condition=cond)(self.func)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper(0), 0)
self.assertEqual(len(cache), 0)
self.assertEqual(lock.count, 3)
def test_zero_size_cache_decorator_info(self):
cache = self.cache(0)
wrapper = cachetools.cached(cache, info=True)(self.func)
self.assertEqual(wrapper.cache_info(), (0, 0, 0, 0))
self.assertEqual(wrapper(0), 0)
self.assertEqual(wrapper.cache_info(), (0, 1, 0, 0))
def test_zero_size_cache_decorator_lock_info(self):
cache = self.cache(0)
lock = CountedLock()
wrapper = cachetools.cached(cache, lock=lock, info=True)(self.func)
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper.cache_info(), (0, 0, 0, 0))
self.assertEqual(lock.count, 1)
self.assertEqual(wrapper(0), 0)
self.assertEqual(len(cache), 0)
self.assertEqual(lock.count, 3)
self.assertEqual(wrapper.cache_info(), (0, 1, 0, 0))
self.assertEqual(lock.count, 4)
class DictWrapperTest(unittest.TestCase, DecoratorTestMixin):
def cache(self, minsize):
return dict()
def test_decorator_info(self):
cache = self.cache(2)
wrapper = cachetools.cached(cache, info=True)(self.func)
self.assertEqual(wrapper.cache_info(), (0, 0, None, 0))
self.assertEqual(wrapper(0), 0)
self.assertEqual(wrapper.cache_info(), (0, 1, None, 1))
self.assertEqual(wrapper(1), 1)
self.assertEqual(wrapper.cache_info(), (0, 2, None, 2))
self.assertEqual(wrapper(0), 0)
self.assertEqual(wrapper.cache_info(), (1, 2, None, 2))
wrapper.cache_clear()
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper.cache_info(), (0, 0, None, 0))
class NoneWrapperTest(unittest.TestCase):
def func(self, *args, **kwargs):
return args + tuple(kwargs.items())
def test_decorator(self):
wrapper = cachetools.cached(None)(self.func)
self.assertEqual(wrapper(0), (0,))
self.assertEqual(wrapper(1), (1,))
self.assertEqual(wrapper(1, foo="bar"), (1, ("foo", "bar")))
def test_decorator_attributes(self):
wrapper = cachetools.cached(None)(self.func)
self.assertIs(wrapper.cache, None)
self.assertIs(wrapper.cache_key, cachetools.keys.hashkey)
self.assertIs(wrapper.cache_lock, None)
def test_decorator_clear(self):
wrapper = cachetools.cached(None)(self.func)
wrapper.cache_clear() # no-op
def test_decorator_info(self):
wrapper = cachetools.cached(None, info=True)(self.func)
self.assertEqual(wrapper.cache_info(), (0, 0, 0, 0))
self.assertEqual(wrapper(0), (0,))
self.assertEqual(wrapper.cache_info(), (0, 1, 0, 0))
self.assertEqual(wrapper(1), (1,))
self.assertEqual(wrapper.cache_info(), (0, 2, 0, 0))
wrapper.cache_clear()
self.assertEqual(wrapper.cache_info(), (0, 0, 0, 0))
tkem-cachetools-78b44a8/tests/test_cachedmethod.py 0000664 0000000 0000000 00000060666 15142726461 0022350 0 ustar 00root root 0000000 0000000 import unittest
import warnings
from cachetools import Cache, cachedmethod, keys
from . import CountedCondition, CountedLock
class Cached:
def __init__(self, cache, count=0):
self.cache = cache
self.count = count
self.lock = CountedLock()
self.cond = CountedCondition()
def __get(self, _value):
result = self.count
self.count += 1
return result
@cachedmethod(lambda self: self.cache)
def get(self, value):
"""docstring"""
return self.__get(value)
@cachedmethod(lambda self: self.cache, key=keys.typedmethodkey)
def get_typed(self, value):
return self.__get(value)
@cachedmethod(lambda self: self.cache, info=True)
def get_info(self, value):
return self.__get(value)
@cachedmethod(lambda self: self.cache, lock=lambda self: self.lock)
def get_lock(self, value):
return self.__get(value)
@cachedmethod(lambda self: self.cache, lock=lambda self: self.lock, info=True)
def get_lock_info(self, value):
return self.__get(value)
@cachedmethod(lambda self: self.cache, condition=lambda self: self.cond)
def get_cond(self, value):
return self.__get(value)
@cachedmethod(lambda self: self.cache, condition=lambda self: self.cond)
def get_cond_error(self, _value):
raise ValueError("test error")
@cachedmethod(lambda self: self.cache, condition=lambda self: self.cond, info=True)
def get_cond_info(self, value):
return self.__get(value)
@cachedmethod(
lambda self: self.cache,
lock=lambda self: self.lock,
condition=lambda self: self.cond,
)
def get_lock_cond(self, value):
return self.__get(value)
@cachedmethod(
lambda self: self.cache,
lock=lambda self: self.lock,
condition=lambda self: self.cond,
info=True,
)
def get_lock_cond_info(self, value):
return self.__get(value)
class Unhashable(Cached):
# https://github.com/tkem/cachetools/issues/107
def __hash__(self):
raise TypeError("unhashable type")
class MethodDecoratorTestMixin:
def cache(self, _minsize):
raise NotImplementedError
def test_decorator(self):
cached = Cached(self.cache(2))
self.assertEqual(cached.get(0), 0)
self.assertEqual(cached.get(1), 1)
self.assertEqual(cached.get(1), 1)
self.assertEqual(cached.get(1.0), 1)
self.assertEqual(cached.get(1.0), 1)
cached.cache.clear()
self.assertEqual(cached.get(1), 2)
def test_decorator_typed(self):
cached = Cached(self.cache(3))
self.assertEqual(cached.get_typed(0), 0)
self.assertEqual(cached.get_typed(1), 1)
self.assertEqual(cached.get_typed(1), 1)
self.assertEqual(cached.get_typed(1.0), 2)
self.assertEqual(cached.get_typed(1.0), 2)
self.assertEqual(cached.get_typed(0.0), 3)
def test_decorator_unhashable(self):
cached = Unhashable(self.cache(2))
self.assertEqual(cached.get(0), 0)
self.assertEqual(cached.get(1), 1)
self.assertEqual(cached.get(1), 1)
self.assertEqual(cached.get(1.0), 1)
self.assertEqual(cached.get(1.0), 1)
cached.cache.clear()
self.assertEqual(cached.get(1), 2)
def test_decorator_lock(self):
cached = Cached(self.cache(2))
self.assertEqual(cached.get_lock(0), 0)
self.assertEqual(cached.lock.count, 2)
self.assertEqual(cached.get_lock(1), 1)
self.assertEqual(cached.lock.count, 4)
self.assertEqual(cached.get_lock(1), 1)
self.assertEqual(cached.lock.count, 5)
self.assertEqual(cached.get_lock(1.0), 1)
self.assertEqual(cached.lock.count, 6)
self.assertEqual(cached.get_lock(1.0), 1)
self.assertEqual(cached.lock.count, 7)
cached.cache.clear()
self.assertEqual(cached.get_lock(1), 2)
self.assertEqual(cached.lock.count, 9)
def test_decorator_cond(self):
cached = Cached(self.cache(2))
self.assertEqual(cached.get_cond(0), 0)
self.assertEqual(cached.cond.count, 3)
self.assertEqual(cached.cond.wait_count, 1)
self.assertEqual(cached.cond.notify_count, 1)
self.assertEqual(cached.get_cond(1), 1)
self.assertEqual(cached.cond.count, 6)
self.assertEqual(cached.cond.wait_count, 2)
self.assertEqual(cached.cond.notify_count, 2)
self.assertEqual(cached.get_cond(1), 1)
self.assertEqual(cached.cond.count, 7)
self.assertEqual(cached.cond.wait_count, 3)
self.assertEqual(cached.cond.notify_count, 2)
self.assertEqual(cached.get_cond(1.0), 1)
self.assertEqual(cached.cond.count, 8)
self.assertEqual(cached.cond.wait_count, 4)
self.assertEqual(cached.cond.notify_count, 2)
self.assertEqual(cached.get_cond(1.0), 1)
self.assertEqual(cached.cond.count, 9)
self.assertEqual(cached.cond.wait_count, 5)
self.assertEqual(cached.cond.notify_count, 2)
cached.cache.clear()
self.assertEqual(cached.get_cond(1), 2)
self.assertEqual(cached.cond.count, 12)
self.assertEqual(cached.cond.wait_count, 6)
self.assertEqual(cached.cond.notify_count, 3)
def test_decorator_info(self):
cache = self.cache(2)
cached = Cached(cache)
wrapper = cached.get_info
maxsize = cache.maxsize if isinstance(cache, Cache) else None
self.assertEqual(wrapper.cache_info(), (0, 0, maxsize, 0))
self.assertEqual(wrapper(0), 0)
self.assertEqual(wrapper.cache_info(), (0, 1, maxsize, 1))
self.assertEqual(wrapper(1), 1)
self.assertEqual(wrapper.cache_info(), (0, 2, maxsize, 2))
self.assertEqual(wrapper(0), 0)
self.assertEqual(wrapper.cache_info(), (1, 2, maxsize, 2))
wrapper.cache_clear()
self.assertEqual(len(cache), 0)
self.assertEqual(wrapper.cache_info(), (0, 0, maxsize, 0))
self.assertEqual(wrapper(0), 2)
self.assertEqual(wrapper.cache_info(), (0, 1, maxsize, 1))
self.assertEqual(wrapper(0), 2)
self.assertEqual(wrapper.cache_info(), (1, 1, maxsize, 1))
# assert hits/misses are counted per instance
cached = Cached(self.cache(2))
self.assertEqual(cached.get_info(0), 0)
self.assertEqual(cached.get_info.cache_info(), (0, 1, maxsize, 1))
def test_decorator_lock_info(self):
cache = self.cache(2)
cached = Cached(cache)
wrapper = cached.get_lock_info
maxsize = cache.maxsize if isinstance(cache, Cache) else None
self.assertEqual(wrapper.cache_info(), (0, 0, maxsize, 0))
self.assertEqual(cached.lock.count, 1)
self.assertEqual(wrapper(0), 0)
self.assertEqual(cached.lock.count, 3)
self.assertEqual(wrapper.cache_info(), (0, 1, maxsize, 1))
self.assertEqual(cached.lock.count, 4)
self.assertEqual(wrapper(1), 1)
self.assertEqual(cached.lock.count, 6)
self.assertEqual(wrapper.cache_info(), (0, 2, maxsize, 2))
self.assertEqual(cached.lock.count, 7)
self.assertEqual(wrapper(0), 0)
self.assertEqual(cached.lock.count, 8)
self.assertEqual(wrapper.cache_info(), (1, 2, maxsize, 2))
self.assertEqual(cached.lock.count, 9)
wrapper.cache_clear()
self.assertEqual(cached.lock.count, 10)
self.assertEqual(len(cache), 0)
self.assertEqual(cached.lock.count, 10)
self.assertEqual(wrapper.cache_info(), (0, 0, maxsize, 0))
self.assertEqual(cached.lock.count, 11)
self.assertEqual(wrapper(0), 2)
self.assertEqual(cached.lock.count, 13)
self.assertEqual(wrapper.cache_info(), (0, 1, maxsize, 1))
self.assertEqual(cached.lock.count, 14)
self.assertEqual(wrapper(0), 2)
self.assertEqual(cached.lock.count, 15)
self.assertEqual(wrapper.cache_info(), (1, 1, maxsize, 1))
self.assertEqual(cached.lock.count, 16)
# assert hits/misses are counted per instance
cached = Cached(self.cache(2))
self.assertEqual(cached.get_lock_info(0), 0)
self.assertEqual(cached.get_lock_info.cache_info(), (0, 1, maxsize, 1))
def test_decorator_cond_info(self):
cache = self.cache(2)
cached = Cached(cache)
wrapper = cached.get_cond_info
maxsize = cache.maxsize if isinstance(cache, Cache) else None
self.assertEqual(wrapper.cache_info(), (0, 0, maxsize, 0))
self.assertEqual(cached.cond.count, 1)
self.assertEqual(wrapper(0), 0)
self.assertEqual(cached.cond.count, 4)
self.assertEqual(cached.cond.wait_count, 1)
self.assertEqual(cached.cond.notify_count, 1)
self.assertEqual(wrapper.cache_info(), (0, 1, maxsize, 1))
self.assertEqual(cached.cond.count, 5)
self.assertEqual(wrapper(1), 1)
self.assertEqual(cached.cond.count, 8)
self.assertEqual(cached.cond.wait_count, 2)
self.assertEqual(cached.cond.notify_count, 2)
self.assertEqual(wrapper.cache_info(), (0, 2, maxsize, 2))
self.assertEqual(cached.cond.count, 9)
self.assertEqual(wrapper(0), 0)
self.assertEqual(cached.cond.count, 10)
self.assertEqual(cached.cond.wait_count, 3)
self.assertEqual(cached.cond.notify_count, 2)
self.assertEqual(wrapper.cache_info(), (1, 2, maxsize, 2))
self.assertEqual(cached.cond.count, 11)
wrapper.cache_clear()
self.assertEqual(cached.cond.count, 12)
self.assertEqual(len(cache), 0)
self.assertEqual(cached.cond.count, 12)
self.assertEqual(wrapper.cache_info(), (0, 0, maxsize, 0))
self.assertEqual(cached.cond.count, 13)
self.assertEqual(wrapper(0), 2)
self.assertEqual(cached.cond.count, 16)
self.assertEqual(cached.cond.wait_count, 4)
self.assertEqual(cached.cond.notify_count, 3)
self.assertEqual(wrapper.cache_info(), (0, 1, maxsize, 1))
self.assertEqual(cached.cond.count, 17)
self.assertEqual(wrapper(0), 2)
self.assertEqual(cached.cond.count, 18)
self.assertEqual(cached.cond.wait_count, 5)
self.assertEqual(cached.cond.notify_count, 3)
self.assertEqual(wrapper.cache_info(), (1, 1, maxsize, 1))
self.assertEqual(cached.cond.count, 19)
# assert hits/misses are counted per instance
cached = Cached(self.cache(2))
self.assertEqual(cached.get_cond_info(0), 0)
self.assertEqual(cached.get_cond_info.cache_info(), (0, 1, maxsize, 1))
def test_decorator_lock_cond_info(self):
cache = self.cache(2)
cached = Cached(cache)
wrapper = cached.get_lock_cond_info
maxsize = cache.maxsize if isinstance(cache, Cache) else None
self.assertEqual(wrapper.cache_info(), (0, 0, maxsize, 0))
self.assertEqual(cached.lock.count, 1)
self.assertEqual(wrapper(0), 0)
self.assertEqual(cached.lock.count, 4)
self.assertEqual(cached.cond.wait_count, 1)
self.assertEqual(cached.cond.notify_count, 1)
self.assertEqual(wrapper.cache_info(), (0, 1, maxsize, 1))
self.assertEqual(cached.lock.count, 5)
self.assertEqual(wrapper(1), 1)
self.assertEqual(cached.lock.count, 8)
self.assertEqual(cached.cond.wait_count, 2)
self.assertEqual(cached.cond.notify_count, 2)
self.assertEqual(wrapper.cache_info(), (0, 2, maxsize, 2))
self.assertEqual(cached.lock.count, 9)
self.assertEqual(wrapper(0), 0)
self.assertEqual(cached.lock.count, 10)
self.assertEqual(cached.cond.wait_count, 3)
self.assertEqual(cached.cond.notify_count, 2)
self.assertEqual(wrapper.cache_info(), (1, 2, maxsize, 2))
self.assertEqual(cached.lock.count, 11)
wrapper.cache_clear()
self.assertEqual(cached.lock.count, 12)
self.assertEqual(len(cache), 0)
self.assertEqual(cached.lock.count, 12)
self.assertEqual(wrapper.cache_info(), (0, 0, maxsize, 0))
self.assertEqual(cached.lock.count, 13)
self.assertEqual(wrapper(0), 2)
self.assertEqual(cached.lock.count, 16)
self.assertEqual(cached.cond.wait_count, 4)
self.assertEqual(cached.cond.notify_count, 3)
self.assertEqual(wrapper.cache_info(), (0, 1, maxsize, 1))
self.assertEqual(cached.lock.count, 17)
self.assertEqual(wrapper(0), 2)
self.assertEqual(cached.lock.count, 18)
self.assertEqual(cached.cond.wait_count, 5)
self.assertEqual(cached.cond.notify_count, 3)
self.assertEqual(wrapper.cache_info(), (1, 1, maxsize, 1))
self.assertEqual(cached.lock.count, 19)
# assert hits/misses are counted per instance
cached = Cached(self.cache(2))
self.assertEqual(cached.get_lock_cond_info(0), 0)
self.assertEqual(cached.get_lock_cond_info.cache_info(), (0, 1, maxsize, 1))
def test_decorator_cond_error(self):
cached = Cached(self.cache(2))
with self.assertRaises(ValueError):
cached.get_cond_error(0)
self.assertEqual(cached.cond.count, 2)
self.assertEqual(cached.cond.wait_count, 1)
self.assertEqual(cached.cond.notify_count, 1)
# verify pending set is cleaned up, otherwise this might deadlock
with self.assertRaises(ValueError):
cached.get_cond_error(0)
self.assertEqual(cached.cond.count, 4)
self.assertEqual(cached.cond.wait_count, 2)
self.assertEqual(cached.cond.notify_count, 2)
def test_decorator_wrapped(self):
cache = self.cache(2)
cached = Cached(cache)
self.assertEqual(len(cache), 0)
self.assertEqual(cached.get.__wrapped__(cached, 0), 0)
self.assertEqual(cached.get.__name__, "get")
self.assertEqual(cached.get.__doc__.strip(), "docstring")
self.assertEqual(len(cache), 0)
self.assertEqual(cached.get(0), 1)
self.assertEqual(len(cache), 1)
self.assertEqual(cached.get(0), 1)
self.assertEqual(len(cache), 1)
def test_decorator_attributes(self):
cache = self.cache(2)
cached = Cached(cache)
self.assertIs(cached.get.cache, cache)
self.assertIs(cached.get.cache_key, keys.methodkey)
self.assertIs(cached.get.cache_lock, None)
self.assertIs(cached.get.cache_condition, None)
self.assertIs(cached.get_lock.cache, cache)
self.assertIs(cached.get_lock.cache_key, keys.methodkey)
self.assertIs(cached.get_lock.cache_lock, cached.lock)
self.assertIs(cached.get_lock.cache_condition, None)
self.assertIs(cached.get_cond.cache, cache)
self.assertIs(cached.get_cond.cache_key, keys.methodkey)
self.assertIs(cached.get_cond.cache_lock, cached.cond)
self.assertIs(cached.get_cond.cache_condition, cached.cond)
self.assertIs(cached.get_lock_cond_info.cache, cache)
self.assertIs(cached.get_lock_cond_info.cache_key, keys.methodkey)
self.assertIs(cached.get_lock_cond_info.cache_lock, cached.lock)
self.assertIs(cached.get_lock_cond_info.cache_condition, cached.cond)
def test_decorator_clear(self):
cache = self.cache(2)
cached = Cached(cache)
self.assertEqual(cached.get(0), 0)
self.assertEqual(len(cache), 1)
cached.get.cache_clear()
self.assertEqual(len(cache), 0)
def test_decorator_lock_clear(self):
cache = self.cache(2)
cached = Cached(cache)
self.assertEqual(cached.get_lock(0), 0)
self.assertEqual(len(cache), 1)
self.assertEqual(cached.lock.count, 2)
cached.get_lock.cache_clear()
self.assertEqual(len(cache), 0)
self.assertEqual(cached.lock.count, 3)
def test_decorator_cond_clear(self):
cache = self.cache(2)
cached = Cached(cache)
self.assertEqual(cached.get_cond(0), 0)
self.assertEqual(len(cache), 1)
self.assertEqual(cached.cond.count, 3)
cached.get_cond.cache_clear()
self.assertEqual(len(cache), 0)
self.assertEqual(cached.cond.count, 4)
def test_decorator_slots(self):
class Slots:
__slots__ = ("cache",)
def __init__(self, cache):
self.cache = cache
@cachedmethod(lambda self: self.cache)
def get(self, value):
return value
@cachedmethod(lambda self: self.cache, info=True)
def get_info(self, value):
return value
cache = self.cache(2)
cached = Slots(cache)
with self.assertRaises(TypeError):
cached.get_info(42)
with warnings.catch_warnings(record=True) as w:
warnings.simplefilter("always")
self.assertEqual(cached.get(0), 0)
self.assertEqual(cached.get(0), 0)
self.assertEqual(len(cache), 1)
self.assertEqual(len(w), 2)
self.assertIs(w[0].category, DeprecationWarning)
self.assertIs(w[1].category, DeprecationWarning)
def test_decorator_immutable_dict(self):
class ReadOnlyDict(dict):
def __setitem__(self, _key, _value):
raise TypeError("Modification not supported")
def setdefault(self, _key, _value):
raise TypeError("Modification not supported")
class Immutable:
def __init__(self, cache):
self.cache = cache
self.__dict__ = ReadOnlyDict(self.__dict__)
@cachedmethod(lambda self: self.cache)
def get(self, value):
return value
@cachedmethod(lambda self: self.cache, info=True)
def get_info(self, value):
return value
cache = self.cache(2)
cached = Immutable(cache)
with self.assertRaises(TypeError):
cached.get_info(42)
with warnings.catch_warnings(record=True) as w:
warnings.simplefilter("always")
self.assertEqual(cached.get(0), 0)
self.assertEqual(cached.get(0), 0)
self.assertEqual(len(cache), 1)
self.assertEqual(len(w), 2)
self.assertIs(w[0].category, DeprecationWarning)
self.assertIs(w[1].category, DeprecationWarning)
def test_decorator_different_names(self):
# RuntimeError for Python < 3.12, TypeError otherwise
with self.assertRaisesRegex(Exception, "bar"):
class Cached:
@cachedmethod(lambda _: self.cache(2))
def foo(_self):
pass
bar = foo
def test_decorator_no_set_name(self):
class Cached:
def foo(_self):
pass
# __set_name__ currently only asserted with info, since this
# may also occur with deprecated @classmethod use
Cached.bar = cachedmethod(lambda _: self.cache(2), info=True)(Cached.foo)
cached = Cached()
with self.assertRaises(TypeError):
cached.bar()
class CacheMethodTest(unittest.TestCase, MethodDecoratorTestMixin):
def cache(self, minsize, **kwargs):
return Cache(maxsize=minsize, **kwargs)
def test_nospace(self):
cached = Cached(self.cache(0))
self.assertEqual(cached.get(0), 0)
self.assertEqual(cached.get(1), 1)
self.assertEqual(cached.get(1), 2)
self.assertEqual(cached.get(1.0), 3)
self.assertEqual(cached.get(1.0), 4)
self.assertEqual(cached.get_typed(0), 5)
self.assertEqual(cached.get_typed(1), 6)
self.assertEqual(cached.get_typed(1.0), 7)
self.assertEqual(cached.get_info(0), 8)
self.assertEqual(cached.get_info(1), 9)
self.assertEqual(cached.get_info(1.0), 10)
def test_lock_nospace(self):
cached = Cached(self.cache(0))
self.assertEqual(cached.get_lock(0), 0)
self.assertEqual(cached.lock.count, 2)
self.assertEqual(cached.get_lock(1), 1)
self.assertEqual(cached.lock.count, 4)
self.assertEqual(cached.get_lock(1), 2)
self.assertEqual(cached.lock.count, 6)
self.assertEqual(cached.get_lock(1.0), 3)
self.assertEqual(cached.lock.count, 8)
self.assertEqual(cached.get_lock(1.0), 4)
self.assertEqual(cached.lock.count, 10)
self.assertEqual(cached.get_lock_info(0), 5)
self.assertEqual(cached.lock.count, 12)
self.assertEqual(cached.get_lock_info(1), 6)
self.assertEqual(cached.lock.count, 14)
self.assertEqual(cached.get_lock_info(1.0), 7)
self.assertEqual(cached.lock.count, 16)
def test_cond_nospace(self):
cached = Cached(self.cache(0))
self.assertEqual(cached.get_cond(0), 0)
self.assertEqual(cached.cond.count, 3)
self.assertEqual(cached.cond.wait_count, 1)
self.assertEqual(cached.cond.notify_count, 1)
self.assertEqual(cached.get_cond(1), 1)
self.assertEqual(cached.cond.count, 6)
self.assertEqual(cached.cond.wait_count, 2)
self.assertEqual(cached.cond.notify_count, 2)
self.assertEqual(cached.get_cond(1), 2)
self.assertEqual(cached.cond.count, 9)
self.assertEqual(cached.cond.wait_count, 3)
self.assertEqual(cached.cond.notify_count, 3)
self.assertEqual(cached.get_cond(1.0), 3)
self.assertEqual(cached.cond.count, 12)
self.assertEqual(cached.cond.wait_count, 4)
self.assertEqual(cached.cond.notify_count, 4)
self.assertEqual(cached.get_cond(1.0), 4)
self.assertEqual(cached.cond.count, 15)
self.assertEqual(cached.cond.wait_count, 5)
self.assertEqual(cached.cond.notify_count, 5)
self.assertEqual(cached.get_cond_info(0), 5)
self.assertEqual(cached.cond.count, 18)
self.assertEqual(cached.cond.wait_count, 6)
self.assertEqual(cached.cond.notify_count, 6)
self.assertEqual(cached.get_cond_info(1), 6)
self.assertEqual(cached.cond.count, 21)
self.assertEqual(cached.cond.wait_count, 7)
self.assertEqual(cached.cond.notify_count, 7)
self.assertEqual(cached.get_cond(1.0), 7)
self.assertEqual(cached.cond.count, 24)
self.assertEqual(cached.cond.wait_count, 8)
self.assertEqual(cached.cond.notify_count, 8)
def test_shared_cache(self):
cache = self.cache(2)
cached1 = Cached(cache)
cached2 = Cached(cache)
self.assertEqual(cached1.get_info.cache_info(), (0, 0, 2, 0))
self.assertEqual(cached2.get_info.cache_info(), (0, 0, 2, 0))
self.assertEqual(cached1.get_info(0), 0)
self.assertEqual(cached1.get_info.cache_info(), (0, 1, 2, 1))
self.assertEqual(cached2.get_info.cache_info(), (0, 0, 2, 1))
self.assertEqual(cached2.get_info(0), 0)
self.assertEqual(cached1.get_info.cache_info(), (0, 1, 2, 1))
self.assertEqual(cached2.get_info.cache_info(), (1, 0, 2, 1))
def test_value_too_large(self):
cache = self.cache(1, getsizeof=lambda x: x)
cached = Cached(cache)
self.assertEqual(cached.get(0), 0)
self.assertIn(0, cache.values())
self.assertEqual(cached.get(1), 1)
self.assertIn(1, cache.values())
self.assertEqual(cached.get(2), 2)
self.assertNotIn(2, cache.values())
class DictMethodTest(unittest.TestCase, MethodDecoratorTestMixin):
def cache(self, _minsize):
return dict()
class WeakRefMethodTest(unittest.TestCase):
def test_weakref(self):
import fractions
import gc
import weakref
# in Python 3.9, `int` does not support weak references even
# when subclassed, but Fraction apparently does...
class Int(fractions.Fraction):
def __add__(self, other):
return Int(fractions.Fraction.__add__(self, other))
cache = weakref.WeakValueDictionary()
cached = Cached(cache, count=Int(0))
self.assertEqual(cached.get(0), 0)
gc.collect()
self.assertEqual(cached.get(0), 1)
ref = cached.get(1)
self.assertEqual(ref, 2)
self.assertEqual(cached.get(1), 2)
self.assertEqual(cached.get(1.0), 2)
ref = cached.get_typed(2)
self.assertEqual(ref, 3)
self.assertEqual(cached.get_typed(1), 4)
self.assertEqual(cached.get_typed(1.0), 5)
cached.cache.clear()
self.assertEqual(cached.get(1), 6)
class NoneMethodTest(unittest.TestCase):
def cache(self, _minsize):
return None
def test_none_info(self):
cached = Cached(self.cache(1))
wrapper = cached.get_info
with self.assertRaises(TypeError):
wrapper.cache_info()
tkem-cachetools-78b44a8/tests/test_classmethod.py 0000664 0000000 0000000 00000011712 15142726461 0022232 0 ustar 00root root 0000000 0000000 import threading
import unittest
import warnings
from cachetools import LRUCache, cachedmethod, keys
class Cached:
cache = LRUCache(2)
count = 0
lock = threading.Lock()
cond = threading.Condition()
@classmethod
@cachedmethod(lambda cls: cls.cache)
def get(cls, value):
cls.count += 1
return cls.count
@classmethod
@cachedmethod(lambda cls: cls.cache, key=keys.typedmethodkey)
def get_typed(cls, value):
cls.count += 1
return cls.count
@classmethod
@cachedmethod(lambda cls: cls.cache, lock=lambda cls: cls.lock)
def get_locked(cls, value):
cls.count += 1
return cls.count
@classmethod
@cachedmethod(lambda cls: cls.cache, condition=lambda cls: cls.cond)
def get_condition(cls, value):
cls.count += 1
return cls.count
class CachedClassMethodTest(unittest.TestCase):
def test(self):
Cached.cache = LRUCache(2)
Cached.count = 0
cached = Cached()
with warnings.catch_warnings(record=True) as w:
warnings.simplefilter("always")
self.assertEqual(cached.get(0), 1)
self.assertEqual(Cached.get(0), 1)
self.assertEqual(len(w), 2)
self.assertIs(w[0].category, DeprecationWarning)
self.assertIs(w[1].category, DeprecationWarning)
with warnings.catch_warnings():
warnings.simplefilter("ignore")
self.assertEqual(cached.get(1), 2)
self.assertEqual(Cached.get(1), 2)
self.assertEqual(cached.get(1), 2)
self.assertEqual(Cached.get(1), 2)
self.assertEqual(cached.get(1.0), 2)
self.assertEqual(Cached.get(1.0), 2)
self.assertEqual(cached.get(1.1), 3)
self.assertEqual(Cached.get(1.1), 3)
Cached.cache.clear()
self.assertEqual(Cached.get(1), 4)
def test_typed(self):
Cached.cache = LRUCache(2)
Cached.count = 0
cached = Cached()
with warnings.catch_warnings(record=True) as w:
warnings.simplefilter("always")
self.assertEqual(cached.get_typed(0), 1)
self.assertEqual(Cached.get_typed(0), 1)
self.assertEqual(len(w), 2)
self.assertIs(w[0].category, DeprecationWarning)
self.assertIs(w[1].category, DeprecationWarning)
with warnings.catch_warnings():
warnings.simplefilter("ignore")
self.assertEqual(cached.get_typed(1), 2)
self.assertEqual(Cached.get_typed(1), 2)
self.assertEqual(cached.get_typed(1.0), 3)
self.assertEqual(Cached.get_typed(1.0), 3)
self.assertEqual(cached.get_typed(0.0), 4)
self.assertEqual(Cached.get_typed(0.0), 4)
self.assertEqual(cached.get_typed(0), 5)
self.assertEqual(Cached.get_typed(0), 5)
def test_locked(self):
Cached.cache = LRUCache(2)
Cached.count = 0
cached = Cached()
with warnings.catch_warnings(record=True) as w:
warnings.simplefilter("always")
self.assertEqual(cached.get_locked(0), 1)
self.assertEqual(Cached.get_locked(0), 1)
self.assertEqual(len(w), 2)
self.assertIs(w[0].category, DeprecationWarning)
self.assertIs(w[1].category, DeprecationWarning)
def test_condition(self):
Cached.cache = LRUCache(2)
Cached.count = 0
cached = Cached()
with warnings.catch_warnings(record=True) as w:
warnings.simplefilter("always")
self.assertEqual(cached.get_condition(0), 1)
self.assertEqual(Cached.get_condition(0), 1)
self.assertEqual(len(w), 2)
self.assertIs(w[0].category, DeprecationWarning)
self.assertIs(w[1].category, DeprecationWarning)
def test_clear(self):
Cached.cache = LRUCache(2)
Cached.count = 0
cached = Cached()
with warnings.catch_warnings():
warnings.simplefilter("ignore")
self.assertEqual(cached.get(0), 1)
self.assertEqual(len(Cached.cache), 1)
Cached.get.cache_clear(cached)
self.assertEqual(len(Cached.cache), 0)
def test_clear_locked(self):
Cached.cache = LRUCache(2)
Cached.count = 0
cached = Cached()
with warnings.catch_warnings():
warnings.simplefilter("ignore")
self.assertEqual(cached.get_locked(0), 1)
self.assertEqual(len(Cached.cache), 1)
Cached.get_locked.cache_clear(cached)
self.assertEqual(len(Cached.cache), 0)
def test_clear_condition(self):
Cached.cache = LRUCache(2)
Cached.count = 0
cached = Cached()
with warnings.catch_warnings():
warnings.simplefilter("ignore")
self.assertEqual(cached.get_condition(0), 1)
self.assertEqual(len(Cached.cache), 1)
Cached.get_condition.cache_clear(cached)
self.assertEqual(len(Cached.cache), 0)
tkem-cachetools-78b44a8/tests/test_fifo.py 0000664 0000000 0000000 00000003177 15142726461 0020655 0 ustar 00root root 0000000 0000000 import unittest
from cachetools import FIFOCache
from . import CacheTestMixin
class FIFOCacheTest(unittest.TestCase, CacheTestMixin):
Cache = FIFOCache
def test_fifo(self):
cache = FIFOCache(maxsize=2)
cache[1] = 1
cache[2] = 2
cache[3] = 3
self.assertEqual(len(cache), 2)
self.assertEqual(cache[2], 2)
self.assertEqual(cache[3], 3)
self.assertNotIn(1, cache)
cache[2]
cache[4] = 4
self.assertEqual(len(cache), 2)
self.assertEqual(cache[3], 3)
self.assertEqual(cache[4], 4)
self.assertNotIn(2, cache)
cache[5] = 5
self.assertEqual(len(cache), 2)
self.assertEqual(cache[4], 4)
self.assertEqual(cache[5], 5)
self.assertNotIn(3, cache)
def test_fifo_getsizeof(self):
cache = FIFOCache(maxsize=3, getsizeof=lambda x: x)
cache[1] = 1
cache[2] = 2
self.assertEqual(len(cache), 2)
self.assertEqual(cache[1], 1)
self.assertEqual(cache[2], 2)
cache[3] = 3
self.assertEqual(len(cache), 1)
self.assertEqual(cache[3], 3)
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
with self.assertRaises(ValueError):
cache[4] = 4
self.assertEqual(len(cache), 1)
self.assertEqual(cache[3], 3)
def test_fifo_update_existing(self):
cache = FIFOCache(maxsize=2)
cache[1] = 1
cache[2] = 2
cache[1] = "updated"
cache[3] = 3
self.assertEqual(cache[1], "updated")
self.assertIn(3, cache)
self.assertNotIn(2, cache)
tkem-cachetools-78b44a8/tests/test_func.py 0000664 0000000 0000000 00000012024 15142726461 0020654 0 ustar 00root root 0000000 0000000 import unittest
import cachetools.func
class DecoratorTestMixin:
def decorator(self, maxsize, **kwargs):
return self.DECORATOR(maxsize, **kwargs)
def test_decorator(self):
cached = self.decorator(maxsize=2)(lambda n: n)
self.assertEqual(cached.cache_parameters(), {"maxsize": 2, "typed": False})
self.assertEqual(cached.cache_info(), (0, 0, 2, 0))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (0, 1, 2, 1))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (1, 1, 2, 1))
self.assertEqual(cached(1.0), 1.0)
self.assertEqual(cached.cache_info(), (2, 1, 2, 1))
def test_decorator_clear(self):
cached = self.decorator(maxsize=2)(lambda n: n)
self.assertEqual(cached.cache_parameters(), {"maxsize": 2, "typed": False})
self.assertEqual(cached.cache_info(), (0, 0, 2, 0))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (0, 1, 2, 1))
cached.cache_clear()
self.assertEqual(cached.cache_info(), (0, 0, 2, 0))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (0, 1, 2, 1))
def test_decorator_nocache(self):
cached = self.decorator(maxsize=0)(lambda n: n)
self.assertEqual(cached.cache_parameters(), {"maxsize": 0, "typed": False})
self.assertEqual(cached.cache_info(), (0, 0, 0, 0))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (0, 1, 0, 0))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (0, 2, 0, 0))
self.assertEqual(cached(1.0), 1.0)
self.assertEqual(cached.cache_info(), (0, 3, 0, 0))
def test_decorator_unbound(self):
cached = self.decorator(maxsize=None)(lambda n: n)
self.assertEqual(cached.cache_parameters(), {"maxsize": None, "typed": False})
self.assertEqual(cached.cache_info(), (0, 0, None, 0))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (0, 1, None, 1))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (1, 1, None, 1))
self.assertEqual(cached(1.0), 1.0)
self.assertEqual(cached.cache_info(), (2, 1, None, 1))
def test_decorator_typed(self):
cached = self.decorator(maxsize=2, typed=True)(lambda n: n)
self.assertEqual(cached.cache_parameters(), {"maxsize": 2, "typed": True})
self.assertEqual(cached.cache_info(), (0, 0, 2, 0))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (0, 1, 2, 1))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (1, 1, 2, 1))
self.assertEqual(cached(1.0), 1.0)
self.assertEqual(cached.cache_info(), (1, 2, 2, 2))
self.assertEqual(cached(1.0), 1.0)
self.assertEqual(cached.cache_info(), (2, 2, 2, 2))
def test_decorator_user_function(self):
cached = self.decorator(lambda n: n)
self.assertEqual(cached.cache_parameters(), {"maxsize": 128, "typed": False})
self.assertEqual(cached.cache_info(), (0, 0, 128, 0))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (0, 1, 128, 1))
self.assertEqual(cached(1), 1)
self.assertEqual(cached.cache_info(), (1, 1, 128, 1))
self.assertEqual(cached(1.0), 1.0)
self.assertEqual(cached.cache_info(), (2, 1, 128, 1))
def test_decorator_needs_rlock(self):
"""This will deadlock on a cache that uses a regular lock.
https://github.com/python/cpython/blob/3.13/Lib/test/test_functools.py#L1791
"""
cached = self.decorator(lambda n: n)
class RecursiveEquals:
def __init__(self, use_cache):
self._use_cache = use_cache
def __hash__(self):
return hash(self._use_cache)
def __eq__(self, other):
if self._use_cache:
# This call will happen while the cache-lock is held,
# requiring a reentrant lock to avoid deadlock.
cached(self)
return self._use_cache == other._use_cache
# Prime the cache.
cached(RecursiveEquals(False))
cached(RecursiveEquals(True))
# Then do a call which will cause a deadlock with a non-reentrant lock.
self.assertEqual(cached(RecursiveEquals(True)), RecursiveEquals(True))
class FIFODecoratorTest(unittest.TestCase, DecoratorTestMixin):
DECORATOR = staticmethod(cachetools.func.fifo_cache)
class LFUDecoratorTest(unittest.TestCase, DecoratorTestMixin):
DECORATOR = staticmethod(cachetools.func.lfu_cache)
class LRUDecoratorTest(unittest.TestCase, DecoratorTestMixin):
DECORATOR = staticmethod(cachetools.func.lru_cache)
class RRDecoratorTest(unittest.TestCase, DecoratorTestMixin):
DECORATOR = staticmethod(cachetools.func.rr_cache)
class TTLDecoratorTest(unittest.TestCase, DecoratorTestMixin):
DECORATOR = staticmethod(cachetools.func.ttl_cache)
tkem-cachetools-78b44a8/tests/test_keys.py 0000664 0000000 0000000 00000011245 15142726461 0020700 0 ustar 00root root 0000000 0000000 import unittest
import cachetools.keys
class CacheKeysTest(unittest.TestCase):
def test_hashkey(self, key=cachetools.keys.hashkey):
self.assertEqual(key(), key())
self.assertEqual(hash(key()), hash(key()))
self.assertEqual(key(1, 2, 3), key(1, 2, 3))
self.assertEqual(hash(key(1, 2, 3)), hash(key(1, 2, 3)))
self.assertEqual(key(1, 2, 3, x=0), key(1, 2, 3, x=0))
self.assertEqual(hash(key(1, 2, 3, x=0)), hash(key(1, 2, 3, x=0)))
self.assertNotEqual(key(1, 2, 3), key(3, 2, 1))
self.assertNotEqual(key(1, 2, 3), key(1, 2, 3, x=None))
self.assertNotEqual(key(1, 2, 3, x=0), key(1, 2, 3, x=None))
self.assertNotEqual(key(1, 2, 3, x=0), key(1, 2, 3, y=0))
with self.assertRaises(TypeError):
hash(key({}))
# untyped keys compare equal
self.assertEqual(key(1, 2, 3), key(1.0, 2.0, 3.0))
self.assertEqual(hash(key(1, 2, 3)), hash(key(1.0, 2.0, 3.0)))
def test_methodkey(self, key=cachetools.keys.methodkey):
# similar to hashkey(), but ignores its first positional argument
self.assertEqual(key("x"), key("y"))
self.assertEqual(hash(key("x")), hash(key("y")))
self.assertEqual(key("x", 1, 2, 3), key("y", 1, 2, 3))
self.assertEqual(hash(key("x", 1, 2, 3)), hash(key("y", 1, 2, 3)))
self.assertEqual(key("x", 1, 2, 3, x=0), key("y", 1, 2, 3, x=0))
self.assertEqual(hash(key("x", 1, 2, 3, x=0)), hash(key("y", 1, 2, 3, x=0)))
self.assertNotEqual(key("x", 1, 2, 3), key("x", 3, 2, 1))
self.assertNotEqual(key("x", 1, 2, 3), key("x", 1, 2, 3, x=None))
self.assertNotEqual(key("x", 1, 2, 3, x=0), key("x", 1, 2, 3, x=None))
self.assertNotEqual(key("x", 1, 2, 3, x=0), key("x", 1, 2, 3, y=0))
with self.assertRaises(TypeError):
hash(key("x", {}))
# untyped keys compare equal
self.assertEqual(key("x", 1, 2, 3), key("y", 1.0, 2.0, 3.0))
self.assertEqual(hash(key("x", 1, 2, 3)), hash(key("y", 1.0, 2.0, 3.0)))
def test_typedkey(self, key=cachetools.keys.typedkey):
self.assertEqual(key(), key())
self.assertEqual(hash(key()), hash(key()))
self.assertEqual(key(1, 2, 3), key(1, 2, 3))
self.assertEqual(hash(key(1, 2, 3)), hash(key(1, 2, 3)))
self.assertEqual(key(1, 2, 3, x=0), key(1, 2, 3, x=0))
self.assertEqual(hash(key(1, 2, 3, x=0)), hash(key(1, 2, 3, x=0)))
self.assertNotEqual(key(1, 2, 3), key(3, 2, 1))
self.assertNotEqual(key(1, 2, 3), key(1, 2, 3, x=None))
self.assertNotEqual(key(1, 2, 3, x=0), key(1, 2, 3, x=None))
self.assertNotEqual(key(1, 2, 3, x=0), key(1, 2, 3, y=0))
with self.assertRaises(TypeError):
hash(key({}))
# typed keys compare unequal
self.assertNotEqual(key(1, 2, 3), key(1.0, 2.0, 3.0))
def test_typedmethodkey(self, key=cachetools.keys.typedmethodkey):
# similar to typedkey(), but ignores its first positional argument
self.assertEqual(key("x"), key("y"))
self.assertEqual(hash(key("x")), hash(key("y")))
self.assertEqual(key("x", 1, 2, 3), key("y", 1, 2, 3))
self.assertEqual(hash(key("x", 1, 2, 3)), hash(key("y", 1, 2, 3)))
self.assertEqual(key("x", 1, 2, 3, x=0), key("y", 1, 2, 3, x=0))
self.assertEqual(hash(key("x", 1, 2, 3, x=0)), hash(key("y", 1, 2, 3, x=0)))
self.assertNotEqual(key("x", 1, 2, 3), key("x", 3, 2, 1))
self.assertNotEqual(key("x", 1, 2, 3), key("x", 1, 2, 3, x=None))
self.assertNotEqual(key("x", 1, 2, 3, x=0), key("x", 1, 2, 3, x=None))
self.assertNotEqual(key("x", 1, 2, 3, x=0), key("x", 1, 2, 3, y=0))
with self.assertRaises(TypeError):
hash(key("x", {}))
# typed keys compare unequal
self.assertNotEqual(key("x", 1, 2, 3), key("x", 1.0, 2.0, 3.0))
def test_addkeys(self, key=cachetools.keys.hashkey):
self.assertIsInstance(key(), tuple)
self.assertIsInstance(key(1, 2, 3) + key(4, 5, 6), type(key()))
self.assertIsInstance(key(1, 2, 3) + (4, 5, 6), type(key()))
self.assertIsInstance((1, 2, 3) + key(4, 5, 6), type(key()))
def test_pickle(self, key=cachetools.keys.hashkey):
import pickle
for k in [key(), key("abc"), key("abc", 123), key("abc", q="abc")]:
# white-box test: assert cached hash value is not pickled
self.assertEqual(len(k.__dict__), 0)
h = hash(k)
self.assertEqual(len(k.__dict__), 1)
pickled = pickle.loads(pickle.dumps(k))
self.assertEqual(len(pickled.__dict__), 0)
self.assertEqual(k, pickled)
self.assertEqual(h, hash(pickled))
tkem-cachetools-78b44a8/tests/test_lfu.py 0000664 0000000 0000000 00000003117 15142726461 0020512 0 ustar 00root root 0000000 0000000 import unittest
from cachetools import LFUCache
from . import CacheTestMixin
class LFUCacheTest(unittest.TestCase, CacheTestMixin):
Cache = LFUCache
def test_lfu(self):
cache = LFUCache(maxsize=2)
cache[1] = 1
cache[1]
cache[2] = 2
cache[3] = 3
self.assertEqual(len(cache), 2)
self.assertEqual(cache[1], 1)
self.assertTrue(2 in cache or 3 in cache)
self.assertTrue(2 not in cache or 3 not in cache)
cache[4] = 4
self.assertEqual(len(cache), 2)
self.assertEqual(cache[4], 4)
self.assertEqual(cache[1], 1)
cache[1]
self.assertEqual(len(cache), 2)
self.assertEqual(cache[1], 1)
self.assertEqual(cache[4], 4)
def test_lfu_getsizeof(self):
cache = LFUCache(maxsize=3, getsizeof=lambda x: x)
cache[1] = 1
cache[2] = 2
self.assertEqual(len(cache), 2)
self.assertEqual(cache[1], 1)
self.assertEqual(cache[2], 2)
cache[3] = 3
self.assertEqual(len(cache), 1)
self.assertEqual(cache[3], 3)
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
with self.assertRaises(ValueError):
cache[4] = 4
self.assertEqual(len(cache), 1)
self.assertEqual(cache[3], 3)
def test_lfu_update_existing(self):
cache = LFUCache(maxsize=2)
cache[1] = 1
cache[2] = 2
cache[1] = "updated"
cache[3] = 3
self.assertEqual(cache[1], "updated")
self.assertIn(3, cache)
self.assertNotIn(2, cache)
tkem-cachetools-78b44a8/tests/test_lru.py 0000664 0000000 0000000 00000003166 15142726461 0020532 0 ustar 00root root 0000000 0000000 import unittest
from cachetools import LRUCache
from . import CacheTestMixin
class LRUCacheTest(unittest.TestCase, CacheTestMixin):
Cache = LRUCache
def test_lru(self):
cache = LRUCache(maxsize=2)
cache[1] = 1
cache[2] = 2
cache[3] = 3
self.assertEqual(len(cache), 2)
self.assertEqual(cache[2], 2)
self.assertEqual(cache[3], 3)
self.assertNotIn(1, cache)
cache[2]
cache[4] = 4
self.assertEqual(len(cache), 2)
self.assertEqual(cache[2], 2)
self.assertEqual(cache[4], 4)
self.assertNotIn(3, cache)
cache[5] = 5
self.assertEqual(len(cache), 2)
self.assertEqual(cache[4], 4)
self.assertEqual(cache[5], 5)
self.assertNotIn(2, cache)
def test_lru_getsizeof(self):
cache = LRUCache(maxsize=3, getsizeof=lambda x: x)
cache[1] = 1
cache[2] = 2
self.assertEqual(len(cache), 2)
self.assertEqual(cache[1], 1)
self.assertEqual(cache[2], 2)
cache[3] = 3
self.assertEqual(len(cache), 1)
self.assertEqual(cache[3], 3)
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
with self.assertRaises(ValueError):
cache[4] = 4
self.assertEqual(len(cache), 1)
self.assertEqual(cache[3], 3)
def test_lru_update_existing(self):
cache = LRUCache(maxsize=2)
cache[1] = 1
cache[2] = 2
cache[1] = "updated"
cache[3] = 3
self.assertEqual(cache[1], "updated")
self.assertIn(3, cache)
self.assertNotIn(2, cache)
tkem-cachetools-78b44a8/tests/test_rr.py 0000775 0000000 0000000 00000004361 15142726461 0020354 0 ustar 00root root 0000000 0000000 import random
import unittest
from cachetools import RRCache
from . import CacheTestMixin
class RRCacheTest(unittest.TestCase, CacheTestMixin):
Cache = RRCache
def test_rr(self):
cache = RRCache(maxsize=2, choice=min)
self.assertEqual(min, cache.choice)
cache[1] = 1
cache[2] = 2
cache[3] = 3
self.assertEqual(2, len(cache))
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
self.assertNotIn(1, cache)
cache[0] = 0
self.assertEqual(2, len(cache))
self.assertEqual(0, cache[0])
self.assertEqual(3, cache[3])
self.assertNotIn(2, cache)
cache[4] = 4
self.assertEqual(2, len(cache))
self.assertEqual(3, cache[3])
self.assertEqual(4, cache[4])
self.assertNotIn(0, cache)
def test_rr_getsizeof(self):
cache = RRCache(maxsize=3, choice=min, getsizeof=lambda x: x)
cache[1] = 1
cache[2] = 2
self.assertEqual(len(cache), 2)
self.assertEqual(cache[1], 1)
self.assertEqual(cache[2], 2)
cache[3] = 3
self.assertEqual(len(cache), 1)
self.assertEqual(cache[3], 3)
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
with self.assertRaises(ValueError):
cache[4] = 4
self.assertEqual(len(cache), 1)
self.assertEqual(cache[3], 3)
def test_rr_update_existing(self):
cache = RRCache(maxsize=2, choice=min)
cache[1] = 1
cache[2] = 2
cache[1] = "updated"
cache[3] = 3
self.assertIn(2, cache)
self.assertIn(3, cache)
self.assertNotIn(1, cache)
def test_rr_bad_choice(self):
def bad_choice(seq):
raise ValueError("test error")
cache = RRCache(maxsize=2, choice=bad_choice)
cache[1] = 1
cache[2] = 2
with self.assertRaises(ValueError):
cache[3] = 3
self.assertEqual(len(cache), 2)
self.assertEqual(cache.currsize, 2)
self.assertIn(1, cache)
self.assertIn(2, cache)
self.assertNotIn(3, cache)
def test_rr_default_choice(self):
cache = RRCache(maxsize=2)
self.assertIs(cache.choice, random.choice)
tkem-cachetools-78b44a8/tests/test_threading.py 0000664 0000000 0000000 00000003007 15142726461 0021667 0 ustar 00root root 0000000 0000000 import threading
import time
import unittest
from os import environ
from cachetools import LRUCache, cached, cachedmethod
@cached(cache=LRUCache(1), condition=threading.Condition(), info=True)
def func():
time.sleep(1.0)
if hasattr(func, "count"):
func.count += 1
else:
func.count = 1
return func.count
@unittest.skipUnless(environ.get("THREADING_TESTS", False), "THREADING_TESTS not set")
class ThreadingTest(unittest.TestCase):
NTHREADS = 10
cache = LRUCache(1)
cond = threading.Condition()
count = 0
@cachedmethod(
cache=lambda self: self.cache, condition=lambda self: self.cond, info=True
)
def meth(self):
time.sleep(1.0)
self.count += 1
return 42
def test_cached_stampede(self):
threads = [threading.Thread(target=func) for i in range(0, self.NTHREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
self.assertEqual(func.count, 1)
info = func.cache_info()
self.assertEqual(info.hits, self.NTHREADS - 1)
self.assertEqual(info.misses, 1)
def test_cachedmethod_stampede(self):
threads = [threading.Thread(target=self.meth) for i in range(0, self.NTHREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
self.assertEqual(self.count, 1)
info = self.meth.cache_info()
self.assertEqual(info.hits, self.NTHREADS - 1)
self.assertEqual(info.misses, 1)
tkem-cachetools-78b44a8/tests/test_tlru.py 0000664 0000000 0000000 00000021504 15142726461 0020712 0 ustar 00root root 0000000 0000000 import math
import unittest
from cachetools import TLRUCache
from . import CacheTestMixin
def default_ttu(_key, _value, _time):
return math.inf
class Timer:
def __init__(self, auto=False):
self.auto = auto
self.time = 0
def __call__(self):
if self.auto:
self.time += 1
return self.time
def tick(self):
self.time += 1
class TLRUTestCache(TLRUCache):
def __init__(self, maxsize, ttu=default_ttu, **kwargs):
TLRUCache.__init__(self, maxsize, ttu, timer=Timer(), **kwargs)
class TLRUCacheTest(unittest.TestCase, CacheTestMixin):
Cache = TLRUTestCache
def test_ttu(self):
cache = TLRUCache(maxsize=6, ttu=lambda _, v, t: t + v + 1, timer=Timer())
self.assertEqual(0, cache.timer())
self.assertEqual(3, cache.ttu(None, 1, 1))
cache[1] = 1
self.assertEqual(1, cache[1])
self.assertEqual(1, len(cache))
self.assertEqual({1}, set(cache))
cache.timer.tick()
self.assertEqual(1, cache[1])
self.assertEqual(1, len(cache))
self.assertEqual({1}, set(cache))
cache[2] = 2
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
self.assertEqual(2, len(cache))
self.assertEqual({1, 2}, set(cache))
cache.timer.tick()
self.assertNotIn(1, cache)
self.assertEqual(2, cache[2])
self.assertEqual(1, len(cache))
self.assertEqual({2}, set(cache))
cache[3] = 3
self.assertNotIn(1, cache)
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
self.assertEqual(2, len(cache))
self.assertEqual({2, 3}, set(cache))
cache.timer.tick()
self.assertNotIn(1, cache)
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
self.assertEqual(2, len(cache))
self.assertEqual({2, 3}, set(cache))
cache[1] = 1
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
self.assertEqual(3, len(cache))
self.assertEqual({1, 2, 3}, set(cache))
cache.timer.tick()
self.assertEqual(1, cache[1])
self.assertNotIn(2, cache)
self.assertEqual(3, cache[3])
self.assertEqual(2, len(cache))
self.assertEqual({1, 3}, set(cache))
cache.timer.tick()
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertEqual(3, cache[3])
self.assertEqual(1, len(cache))
self.assertEqual({3}, set(cache))
cache.timer.tick()
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertNotIn(3, cache)
with self.assertRaises(KeyError):
del cache[1]
with self.assertRaises(KeyError):
cache.pop(2)
with self.assertRaises(KeyError):
del cache[3]
self.assertEqual(0, len(cache))
self.assertEqual(set(), set(cache))
def test_ttu_lru(self):
cache = TLRUCache(maxsize=2, ttu=lambda k, v, t: t + 1, timer=Timer())
self.assertEqual(0, cache.timer())
self.assertEqual(2, cache.ttu(None, None, 1))
cache[1] = 1
cache[2] = 2
cache[3] = 3
self.assertEqual(len(cache), 2)
self.assertNotIn(1, cache)
self.assertEqual(cache[2], 2)
self.assertEqual(cache[3], 3)
cache[2]
cache[4] = 4
self.assertEqual(len(cache), 2)
self.assertNotIn(1, cache)
self.assertEqual(cache[2], 2)
self.assertNotIn(3, cache)
self.assertEqual(cache[4], 4)
cache[5] = 5
self.assertEqual(len(cache), 2)
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertNotIn(3, cache)
self.assertEqual(cache[4], 4)
self.assertEqual(cache[5], 5)
def test_ttu_expire(self):
cache = TLRUCache(maxsize=3, ttu=lambda k, v, t: t + 3, timer=Timer())
with cache.timer as time:
self.assertEqual(time, cache.timer())
cache[1] = 1
cache.timer.tick()
cache[2] = 2
cache.timer.tick()
cache[3] = 3
self.assertEqual(2, cache.timer())
self.assertEqual({1, 2, 3}, set(cache))
self.assertEqual(3, len(cache))
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
items = cache.expire()
self.assertEqual(set(), set(items))
self.assertEqual({1, 2, 3}, set(cache))
self.assertEqual(3, len(cache))
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
items = cache.expire(3)
self.assertEqual({(1, 1)}, set(items))
self.assertEqual({2, 3}, set(cache))
self.assertEqual(2, len(cache))
self.assertNotIn(1, cache)
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
items = cache.expire(4)
self.assertEqual({(2, 2)}, set(items))
self.assertEqual({3}, set(cache))
self.assertEqual(1, len(cache))
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertEqual(3, cache[3])
items = cache.expire(5)
self.assertEqual({(3, 3)}, set(items))
self.assertEqual(set(), set(cache))
self.assertEqual(0, len(cache))
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertNotIn(3, cache)
def test_ttu_expired(self):
cache = TLRUCache(maxsize=1, ttu=lambda k, _, t: t + k, timer=Timer())
cache[1] = None
self.assertEqual(cache[1], None)
self.assertEqual(1, len(cache))
cache[0] = None
self.assertNotIn(0, cache)
self.assertEqual(cache[1], None)
self.assertEqual(1, len(cache))
cache[-1] = None
self.assertNotIn(-1, cache)
self.assertNotIn(0, cache)
self.assertEqual(cache[1], None)
self.assertEqual(1, len(cache))
def test_ttu_atomic(self):
cache = TLRUCache(maxsize=1, ttu=lambda k, v, t: t + 2, timer=Timer(auto=True))
cache[1] = 1
self.assertEqual(1, cache[1])
cache[1] = 1
self.assertEqual(1, cache.get(1))
cache[1] = 1
self.assertEqual(1, cache.pop(1))
cache[1] = 1
self.assertEqual(1, cache.setdefault(1))
cache[1] = 1
cache.clear()
self.assertEqual(0, len(cache))
def test_ttu_tuple_key(self):
cache = TLRUCache(maxsize=1, ttu=lambda k, v, t: t + 1, timer=Timer())
cache[(1, 2, 3)] = 42
self.assertEqual(42, cache[(1, 2, 3)])
cache.timer.tick()
with self.assertRaises(KeyError):
cache[(1, 2, 3)]
self.assertNotIn((1, 2, 3), cache)
def test_ttu_reverse_insert(self):
cache = TLRUCache(maxsize=4, ttu=lambda k, v, t: t + v, timer=Timer())
self.assertEqual(0, cache.timer())
cache[3] = 3
cache[2] = 2
cache[1] = 1
cache[0] = 0
self.assertEqual({1, 2, 3}, set(cache))
self.assertEqual(3, len(cache))
self.assertNotIn(0, cache)
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
cache.timer.tick()
self.assertEqual({2, 3}, set(cache))
self.assertEqual(2, len(cache))
self.assertNotIn(0, cache)
self.assertNotIn(1, cache)
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
cache.timer.tick()
self.assertEqual({3}, set(cache))
self.assertEqual(1, len(cache))
self.assertNotIn(0, cache)
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertEqual(3, cache[3])
cache.timer.tick()
self.assertEqual(set(), set(cache))
self.assertEqual(0, len(cache))
self.assertNotIn(0, cache)
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertNotIn(3, cache)
def test_ttu_heap_cleanup(self):
cache = TLRUCache(maxsize=4, ttu=lambda k, v, t: t + 1, timer=Timer())
self.assertEqual(0, cache.timer())
cache[1] = 1
cache[2] = 2
# replace items to accumulate removed entries in the internal heap
for i in range(5):
cache[1] = 10 + i
cache[2] = 20 + i
# this should compact the internal heap
expired = cache.expire()
self.assertEqual([], expired)
# verify cache is functional after cleanup
self.assertEqual(2, len(cache))
self.assertEqual(14, cache[1])
self.assertEqual(24, cache[2])
cache[3] = 3
cache[4] = 4
self.assertEqual(4, len(cache))
cache.timer.tick()
expired = cache.expire()
self.assertEqual(4, len(expired))
self.assertEqual(0, len(cache))
tkem-cachetools-78b44a8/tests/test_ttl.py 0000664 0000000 0000000 00000014411 15142726461 0020526 0 ustar 00root root 0000000 0000000 import math
import unittest
from cachetools import TTLCache
from . import CacheTestMixin
class Timer:
def __init__(self, auto=False):
self.auto = auto
self.time = 0
def __call__(self):
if self.auto:
self.time += 1
return self.time
def tick(self):
self.time += 1
class TTLTestCache(TTLCache):
def __init__(self, maxsize, ttl=math.inf, **kwargs):
TTLCache.__init__(self, maxsize, ttl=ttl, timer=Timer(), **kwargs)
class TTLCacheTest(unittest.TestCase, CacheTestMixin):
Cache = TTLTestCache
def test_ttl(self):
cache = TTLCache(maxsize=2, ttl=2, timer=Timer())
self.assertEqual(0, cache.timer())
self.assertEqual(2, cache.ttl)
cache[1] = 1
self.assertEqual(1, cache[1])
self.assertEqual(1, len(cache))
self.assertEqual({1}, set(cache))
cache.timer.tick()
self.assertEqual(1, cache[1])
self.assertEqual(1, len(cache))
self.assertEqual({1}, set(cache))
cache[2] = 2
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
self.assertEqual(2, len(cache))
self.assertEqual({1, 2}, set(cache))
cache.timer.tick()
self.assertNotIn(1, cache)
self.assertEqual(2, cache[2])
self.assertEqual(1, len(cache))
self.assertEqual({2}, set(cache))
cache[3] = 3
self.assertNotIn(1, cache)
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
self.assertEqual(2, len(cache))
self.assertEqual({2, 3}, set(cache))
cache.timer.tick()
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertEqual(3, cache[3])
self.assertEqual(1, len(cache))
self.assertEqual({3}, set(cache))
cache.timer.tick()
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertNotIn(3, cache)
with self.assertRaises(KeyError):
del cache[1]
with self.assertRaises(KeyError):
cache.pop(2)
with self.assertRaises(KeyError):
del cache[3]
self.assertEqual(0, len(cache))
self.assertEqual(set(), set(cache))
def test_ttl_timer(self):
cache = TTLCache(maxsize=2, ttl=2, timer=Timer())
self.assertEqual(cache.timer.time, 0)
self.assertFalse(cache.timer.auto)
cache[1] = 1
cache.timer.tick()
self.assertEqual(cache.timer.time, 1)
self.assertEqual(1, cache[1])
cache.timer.tick()
self.assertEqual(cache.timer.time, 2)
self.assertNotIn(1, cache)
def test_ttl_lru(self):
cache = TTLCache(maxsize=2, ttl=1, timer=Timer())
cache[1] = 1
cache[2] = 2
cache[3] = 3
self.assertEqual(len(cache), 2)
self.assertNotIn(1, cache)
self.assertEqual(cache[2], 2)
self.assertEqual(cache[3], 3)
cache[2]
cache[4] = 4
self.assertEqual(len(cache), 2)
self.assertNotIn(1, cache)
self.assertEqual(cache[2], 2)
self.assertNotIn(3, cache)
self.assertEqual(cache[4], 4)
cache[5] = 5
self.assertEqual(len(cache), 2)
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertNotIn(3, cache)
self.assertEqual(cache[4], 4)
self.assertEqual(cache[5], 5)
def test_ttl_expire(self):
cache = TTLCache(maxsize=3, ttl=3, timer=Timer())
with cache.timer as time:
self.assertEqual(time, cache.timer())
self.assertEqual(3, cache.ttl)
cache[1] = 1
cache.timer.tick()
cache[2] = 2
cache.timer.tick()
cache[3] = 3
self.assertEqual(2, cache.timer())
self.assertEqual({1, 2, 3}, set(cache))
self.assertEqual(3, len(cache))
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
items = cache.expire()
self.assertEqual(set(), set(items))
self.assertEqual({1, 2, 3}, set(cache))
self.assertEqual(3, len(cache))
self.assertEqual(1, cache[1])
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
items = cache.expire(3)
self.assertEqual({(1, 1)}, set(items))
self.assertEqual({2, 3}, set(cache))
self.assertEqual(2, len(cache))
self.assertNotIn(1, cache)
self.assertEqual(2, cache[2])
self.assertEqual(3, cache[3])
items = cache.expire(4)
self.assertEqual({(2, 2)}, set(items))
self.assertEqual({3}, set(cache))
self.assertEqual(1, len(cache))
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertEqual(3, cache[3])
items = cache.expire(5)
self.assertEqual({(3, 3)}, set(items))
self.assertEqual(set(), set(cache))
self.assertEqual(0, len(cache))
self.assertNotIn(1, cache)
self.assertNotIn(2, cache)
self.assertNotIn(3, cache)
def test_ttl_atomic(self):
cache = TTLCache(maxsize=1, ttl=2, timer=Timer(auto=True))
cache[1] = 1
self.assertEqual(1, cache[1])
cache[1] = 1
self.assertEqual(1, cache.get(1))
cache[1] = 1
self.assertEqual(1, cache.pop(1))
cache[1] = 1
self.assertEqual(1, cache.setdefault(1))
cache[1] = 1
cache.clear()
self.assertEqual(0, len(cache))
def test_ttl_tuple_key(self):
cache = TTLCache(maxsize=1, ttl=1, timer=Timer())
self.assertEqual(1, cache.ttl)
cache[(1, 2, 3)] = 42
self.assertEqual(42, cache[(1, 2, 3)])
cache.timer.tick()
with self.assertRaises(KeyError):
cache[(1, 2, 3)]
self.assertNotIn((1, 2, 3), cache)
def test_ttl_datetime(self):
from datetime import datetime, timedelta
cache = TTLCache(maxsize=1, ttl=timedelta(days=1), timer=datetime.now)
cache[1] = 1
self.assertEqual(1, len(cache))
items = cache.expire(datetime.now())
self.assertEqual([], list(items))
self.assertEqual(1, len(cache))
items = cache.expire(datetime.now() + timedelta(days=1))
self.assertEqual([(1, 1)], list(items))
self.assertEqual(0, len(cache))
tkem-cachetools-78b44a8/tox.ini 0000664 0000000 0000000 00000001346 15142726461 0016466 0 ustar 00root root 0000000 0000000 [tox]
envlist = check-manifest,docs,doctest,flake8,py
[testenv]
deps =
pytest
pytest-cov
setenv =
THREADING_TESTS = 1
commands =
py.test --basetemp={envtmpdir} --cov=cachetools --cov-report term-missing {posargs}
[testenv:check-manifest]
deps =
check-manifest
commands =
check-manifest
skip_install = true
[testenv:docs]
deps =
sphinx
commands =
sphinx-build -W -b html -d {envtmpdir}/doctrees docs {envtmpdir}/html
[testenv:doctest]
deps =
sphinx
commands =
sphinx-build -W -b doctest -d {envtmpdir}/doctrees docs {envtmpdir}/doctest
[testenv:flake8]
deps =
flake8
flake8-black
flake8-bugbear
flake8-import-order
flake8-pyproject
commands =
flake8
skip_install = true