pax_global_header 0000666 0000000 0000000 00000000064 14733557262 0014530 g ustar 00root root 0000000 0000000 52 comment=dd965dce22e5278d4935bea923441ecde31b5325
python-msgspec-0.19.0/ 0000775 0000000 0000000 00000000000 14733557262 0014577 5 ustar 00root root 0000000 0000000 python-msgspec-0.19.0/.codecov.yml 0000664 0000000 0000000 00000000172 14733557262 0017022 0 ustar 00root root 0000000 0000000 comment: false
coverage:
status:
project:
default:
target: auto
threshold: 1%
patch: off
python-msgspec-0.19.0/.gitattributes 0000664 0000000 0000000 00000000041 14733557262 0017465 0 ustar 00root root 0000000 0000000 msgspec/_version.py export-subst
python-msgspec-0.19.0/.github/ 0000775 0000000 0000000 00000000000 14733557262 0016137 5 ustar 00root root 0000000 0000000 python-msgspec-0.19.0/.github/CODE_OF_CONDUCT.md 0000664 0000000 0000000 00000006212 14733557262 0020737 0 ustar 00root root 0000000 0000000 # Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, gender identity and expression, level of experience,
nationality, personal appearance, race, religion, or sexual identity and
orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at jcristharif@gmail.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at [http://contributor-covenant.org/version/1/4][version]
[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/4/
python-msgspec-0.19.0/.github/CONTRIBUTING.md 0000664 0000000 0000000 00000006045 14733557262 0020375 0 ustar 00root root 0000000 0000000 # How to Contribute
Thank you for taking the time to contribute to `msgspec`!
Here we document some contribution guidelines to help you ensure that your
contribution is at its best.
## Setting up your Development Environment
Before getting started, you will need to already have installed:
- Python (3.8+ only), with development headers installed
- A C compiler (`gcc`, `clang`, and `msvc` are all tested)
- `git`
Once you have those installed, you're ready to:
- Clone the repository
- Install all development dependencies
- Build a development version of `msgspec`
- Install the `pre-commit` hooks
```bash
# Clone the repository
git clone https://github.com/jcrist/msgspec.git
# cd into the repo root directory
cd msgspec/
# Build and install msgspec & all dev dependencies
pip install -e ".[dev]"
# Install the pre-commit hooks
pre-commit install
```
## Editing and Rebuilding
You now have a "development" build of `msgspec` installed. This means that you
can make changes to the `.py` files and test them without requiring a rebuild
of msgspec's C extension. Edit away!
If you do make changes to a `.c` file, you'll need to recompile. You can do
this by running
```bash
pip install -e .
```
By default `msgspec` is built in release mode, with optimizations enabled. To
build a debug build instead (for use with e.g. `gdb` or `lldb`) define the
`MSGSPEC_DEBUG` environment variable before building.
```bash
MSGSPEC_DEBUG=1 pip install -e .
```
## Testing
Tests are located in the `tests/` directory. Any code changes should include
additional tests to ensure correctness. The tests are broken into various
`test_*.py` files specific to the functionality that they're testing.
The tests can be run using `pytest` as follows:
```bash
pytest
```
If you want to run a specific test file, you may specify that file explicitly:
```bash
pytest tests/test_json.py
```
## Linting
We use `pre-commit` to automatically run a few code linters before every
commit. If you followed the development setup above, you should already have
`pre-commit` and all the commit hooks installed.
These hooks will run whenever you try to commit changes.
```bash
git commit # linters will run automatically here
```
If you wish to run the linters manually without committing, you can run:
```bash
pre-commit run
```
## Documentation
The source of the documentation can be found under `docs/source/`. They are
built using `Sphinx` and can be built locally by running the following steps:
```bash
cd docs/ # Make sure we are in the docs/ folder
make html # Build the html
# Output can now be found under docs/build/html and can be viewed in the browser
```
## Continuous Integration (CI)
We use GitHub Actions to provide "continuous integration" testing for all Pull
Requests (PRs). When submitting a PR, please check to see that all tests pass,
and fix any issues that come up.
## Code of Conduct
``msgspec`` has a code of conduct that must be followed by all contributors to
the project. You may read the code of conduct
[here](https://github.com/jcrist/msgspec/blob/main/CODE_OF_CONDUCT.md).
python-msgspec-0.19.0/.github/ISSUE_TEMPLATE/ 0000775 0000000 0000000 00000000000 14733557262 0020322 5 ustar 00root root 0000000 0000000 python-msgspec-0.19.0/.github/ISSUE_TEMPLATE/bug.yml 0000664 0000000 0000000 00000001523 14733557262 0021623 0 ustar 00root root 0000000 0000000 name: 🪲 Bug Report
description: Report a bug or unexpected behavior in msgspec
body:
- type: markdown
attributes:
value: Thanks for taking the time to fill out a bug report!
- type: textarea
id: description
attributes:
label: Description
description: >
Describe the bug. What happened? What did you expect to happen?
When possible, please also include a [minimal, complete, verifiable
example](https://stackoverflow.com/help/minimal-reproducible-example).
Ideally this should be code that can be run without modification to
demonstrate the problem.
When including errors and tracebacks, please include the _full
traceback_ as well as the code that generated the error (or at least
the line that caused it).
validations:
required: true
python-msgspec-0.19.0/.github/ISSUE_TEMPLATE/config.yml 0000664 0000000 0000000 00000000033 14733557262 0022306 0 ustar 00root root 0000000 0000000 blank_issues_enabled: true
python-msgspec-0.19.0/.github/ISSUE_TEMPLATE/feature.yml 0000664 0000000 0000000 00000001404 14733557262 0022477 0 ustar 00root root 0000000 0000000 name: 🙌 Feature Request
description: Suggest a new feature or change to msgspec
body:
- type: markdown
attributes:
value: Thanks for taking the time to fill out a feature request!
- type: textarea
id: description
attributes:
label: Description
description: >
Describe the feature. What problems does it solve?
If the feature is to related to a problem, please describe in detail
your use case. What would this new feature help you do that you
couldn't do before? Why is this useful?
When relevant, please also include example code making use of your
proposed feature. How would you use this feature? What would code using
it look like?
validations:
required: true
python-msgspec-0.19.0/.github/ISSUE_TEMPLATE/question.yml 0000664 0000000 0000000 00000000703 14733557262 0022714 0 ustar 00root root 0000000 0000000 name: ❓ Question
description: Ask a question
body:
- type: markdown
attributes:
value: Thanks for taking the time to ask a question!
- type: textarea
id: description
attributes:
label: Question
description: >
Ask your question here. Please search through existing and closed
issues first to ensure your question hasn't already been answered
elsewhere.
validations:
required: true
python-msgspec-0.19.0/.github/SECURITY.md 0000664 0000000 0000000 00000001233 14733557262 0017727 0 ustar 00root root 0000000 0000000 # Security Policy
If you believe you have found a security-related bug with `msgspec`, **do not
open a public GitHub issue**. Instead, please email jcristharif@gmail.com.
Please include as much detail as you would for a normal issue in your report.
In particular, including a minimal reproducible example will help the
maintainers diagnose and resolve the issue quickly and efficiently.
After the issue is resolved, we will make a release and announce the security
fix through our normal communication channels. When it makes sense we may also
obtain a CVE ID. If you would like to be credited with the report, please
include your name and any links in the email.
python-msgspec-0.19.0/.github/workflows/ 0000775 0000000 0000000 00000000000 14733557262 0020174 5 ustar 00root root 0000000 0000000 python-msgspec-0.19.0/.github/workflows/ci.yml 0000664 0000000 0000000 00000007661 14733557262 0021324 0 ustar 00root root 0000000 0000000 name: Build and Test
on:
push:
branches: [main]
pull_request:
branches: [main]
paths-ignore:
- "docs/**"
- "benchmarks/**"
- "examples/**"
- ".github/**"
- "README.rst"
release:
types: [published]
jobs:
lint:
name: Lint and ruff code
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Build msgspec and install dependencies
run: |
pip install -e ".[dev]"
- name: Run pre-commit hooks
uses: pre-commit/action@v3.0.0
- name: mypy
run: pytest tests/test_mypy.py
- name: pyright
run: pytest tests/test_pyright.py
- name: doctests
run: pytest --doctest-modules msgspec
- name: Rebuild with sanitizers & coverage
env:
MSGSPEC_SANITIZE: "true"
MSGSPEC_COVERAGE: "true"
run: |
python setup.py clean --all
# I know this is deprecated, but I can't find a way to keep the build
# directory around anymore on new versions of setuptools
python setup.py develop
- name: Run tests with sanitizers
env:
PYTHONMALLOC: "malloc"
ASAN_OPTIONS: "detect_leaks=0"
run: |
LD_PRELOAD=`gcc -print-file-name=libasan.so` coverage run -m pytest -s -m "not mypy and not pyright"
- name: Generate coverage files
run: |
coverage xml
gcov -abcu `find build/ -name *.o`
- name: Upload Codecov
uses: codecov/codecov-action@v3
with:
files: coverage.xml,_core.c.gcov,atof.h.gcov,ryu.h.gcov
build_wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-13, windows-latest]
env:
CIBW_TEST_EXTRAS: "test"
CIBW_TEST_COMMAND: "pytest {project}/tests"
CIBW_BUILD: "cp39-* cp310-* cp311-* cp312-* cp313-*"
CIBW_SKIP: "*-win32 *_i686 *_s390x *_ppc64le"
CIBW_ARCHS_MACOS: "x86_64 arm64"
CIBW_ARCHS_LINUX: "x86_64 aarch64"
CIBW_TEST_SKIP: "*_arm64 *-musllinux_*"
CIBW_ENVIRONMENT: "CFLAGS=-g0"
steps:
- uses: actions/checkout@v4
- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v1
with:
platforms: all
- name: Set up Environment
if: github.event_name != 'release'
run: |
echo "CIBW_SKIP=${CIBW_SKIP} *-musllinux_* cp39-*_aarch64 cp311-*_aarch64 cp312-*_aarch64 cp313-*_aarch64" >> $GITHUB_ENV
- name: Build & Test Wheels
uses: pypa/cibuildwheel@v2.22.0
- name: Upload artifact
uses: actions/upload-artifact@v4
if: github.event_name == 'release' && github.event.action == 'published'
with:
name: artifact-wheels-${{ matrix.os }}
path: ./wheelhouse/*.whl
build_sdist:
name: Build Source Distribution
runs-on: ubuntu-latest
if: github.event_name == 'release' && github.event.action == 'published'
steps:
- uses: actions/checkout@v4
- name: Install Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Build source distribution
run: python setup.py sdist
- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: artifact-sdist
path: dist/*.tar.gz
upload_pypi:
needs: [build_wheels, build_sdist]
runs-on: ubuntu-latest
permissions:
id-token: write
if: github.event_name == 'release' && github.event.action == 'published'
steps:
- uses: actions/download-artifact@v4
with:
merge-multiple: true
path: dist
pattern: artifact-*
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
python-msgspec-0.19.0/.github/workflows/docs.yml 0000664 0000000 0000000 00000001266 14733557262 0021654 0 ustar 00root root 0000000 0000000 name: documentation
on:
push:
branches: [main]
pull_request: null
jobs:
build-docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install msgspec and dependencies
run: |
pip install -e ".[doc]"
- name: Build Docs
run: |
pushd docs
make html
popd
- name: Deploy
uses: peaceiris/actions-gh-pages@v3
if: github.ref == 'refs/heads/main'
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./docs/build/html
python-msgspec-0.19.0/.gitignore 0000664 0000000 0000000 00000001020 14733557262 0016560 0 ustar 00root root 0000000 0000000 # Editor config folders
## Vscode
.settings/
.project
.vscode/
.vs/
## PyCharm/IntelliJ-generated files
*.iml
.idea/
# Python cached sources
__pycache__/
*.pyc
# Virtual environments
.venv*/
venv*/
# Pytest and coverage
.coverage
.pytest/
.pytest_cache/
htmlcov/
# Mypy Cache
.mypy_cache/
# Docs build
docs/build/
# Benchmark outputs
benchmarks/*.html
benchmarks/*.json
# Setuptools/twine-generated files, compiled sources.
build/
dist/
*.egg-info/
pip-wheel-metadata/
*.so
*.o
*.pyd
# Misc
*.pem
out/
.cache/
.DS_Store
python-msgspec-0.19.0/.pre-commit-config.yaml 0000664 0000000 0000000 00000000447 14733557262 0021065 0 ustar 00root root 0000000 0000000 repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.7.1
hooks:
- id: ruff
args: [ --fix ]
- id: ruff-format
- repo: https://github.com/codespell-project/codespell
rev: v2.2.2
hooks:
- id: codespell
language_version: python3
python-msgspec-0.19.0/LICENSE 0000664 0000000 0000000 00000002732 14733557262 0015610 0 ustar 00root root 0000000 0000000 Copyright (c) 2021, Jim Crist-Harif
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
python-msgspec-0.19.0/MANIFEST.in 0000664 0000000 0000000 00000000311 14733557262 0016330 0 ustar 00root root 0000000 0000000 include msgspec/*.c
include msgspec/*.h
include msgspec/*.py
include msgspec/*.pyi
include msgspec/py.typed
include setup.py
include versioneer.py
include README.md
include LICENSE
include MANIFEST.in
python-msgspec-0.19.0/README.md 0000664 0000000 0000000 00000010511 14733557262 0016054 0 ustar 00root root 0000000 0000000
`msgspec` is a *fast* serialization and validation library, with builtin
support for [JSON](https://json.org), [MessagePack](https://msgpack.org),
[YAML](https://yaml.org), and [TOML](https://toml.io). It features:
- 🚀 **High performance encoders/decoders** for common protocols. The JSON and
MessagePack implementations regularly
[benchmark](https://jcristharif.com/msgspec/benchmarks.html) as the fastest
options for Python.
- 🎉 **Support for a wide variety of Python types**. Additional types may be
supported through
[extensions](https://jcristharif.com/msgspec/extending.html).
- 🔍 **Zero-cost schema validation** using familiar Python type annotations. In
[benchmarks](https://jcristharif.com/msgspec/benchmarks.html) `msgspec`
decodes *and* validates JSON faster than
[orjson](https://github.com/ijl/orjson) can decode it alone.
- ✨ **A speedy Struct type** for representing structured data. If you already
use [dataclasses](https://docs.python.org/3/library/dataclasses.html) or
[attrs](https://www.attrs.org),
[structs](https://jcristharif.com/msgspec/structs.html) should feel familiar.
However, they're
[5-60x faster](https://jcristharif.com/msgspec/benchmarks.html#benchmark-structs>)
for common operations.
All of this is included in a
[lightweight library](https://jcristharif.com/msgspec/benchmarks.html#benchmark-library-size)
with no required dependencies.
---
`msgspec` may be used for serialization alone, as a faster JSON or
MessagePack library. For the greatest benefit though, we recommend using
`msgspec` to handle the full serialization & validation workflow:
**Define** your message schemas using standard Python type annotations.
```python
>>> import msgspec
>>> class User(msgspec.Struct):
... """A new type describing a User"""
... name: str
... groups: set[str] = set()
... email: str | None = None
```
**Encode** messages as JSON, or one of the many other supported protocols.
```python
>>> alice = User("alice", groups={"admin", "engineering"})
>>> alice
User(name='alice', groups={"admin", "engineering"}, email=None)
>>> msg = msgspec.json.encode(alice)
>>> msg
b'{"name":"alice","groups":["admin","engineering"],"email":null}'
```
**Decode** messages back into Python objects, with optional schema validation.
```python
>>> msgspec.json.decode(msg, type=User)
User(name='alice', groups={"admin", "engineering"}, email=None)
>>> msgspec.json.decode(b'{"name":"bob","groups":[123]}', type=User)
Traceback (most recent call last):
File "", line 1, in
msgspec.ValidationError: Expected `str`, got `int` - at `$.groups[0]`
```
`msgspec` is designed to be as performant as possible, while retaining some of
the nicities of validation libraries like
[pydantic](https://pydantic-docs.helpmanual.io/). For supported types,
encoding/decoding a message with `msgspec` can be
[~10-80x faster than alternative libraries](https://jcristharif.com/msgspec/benchmarks.html).
See [the documentation](https://jcristharif.com/msgspec/) for more information.
## LICENSE
New BSD. See the
[License File](https://github.com/jcrist/msgspec/blob/main/LICENSE).
python-msgspec-0.19.0/benchmarks/ 0000775 0000000 0000000 00000000000 14733557262 0016714 5 ustar 00root root 0000000 0000000 python-msgspec-0.19.0/benchmarks/__init__.py 0000664 0000000 0000000 00000000000 14733557262 0021013 0 ustar 00root root 0000000 0000000 python-msgspec-0.19.0/benchmarks/bench_encodings.py 0000664 0000000 0000000 00000013035 14733557262 0022400 0 ustar 00root root 0000000 0000000 from __future__ import annotations
import sys
import dataclasses
import json
import timeit
import importlib.metadata
from typing import Any, Literal, Callable
from .generate_data import make_filesystem_data
import msgspec
class File(msgspec.Struct, kw_only=True, omit_defaults=True, tag="file"):
name: str
created_by: str
created_at: str
updated_by: str | None = None
updated_at: str | None = None
nbytes: int
permissions: Literal["READ", "WRITE", "READ_WRITE"]
class Directory(msgspec.Struct, kw_only=True, omit_defaults=True, tag="directory"):
name: str
created_by: str
created_at: str
updated_by: str | None = None
updated_at: str | None = None
contents: list[File | Directory]
@dataclasses.dataclass
class Benchmark:
label: str
version: str
encode: Callable
decode: Callable
schema: Any = None
def run(self, data: bytes) -> dict:
if self.schema is not None:
data = msgspec.convert(data, self.schema)
timer = timeit.Timer("func(data)", globals={"func": self.encode, "data": data})
n, t = timer.autorange()
encode_time = t / n
data = self.encode(data)
timer = timeit.Timer("func(data)", globals={"func": self.decode, "data": data})
n, t = timer.autorange()
decode_time = t / n
return {
"label": self.label,
"encode": encode_time,
"decode": decode_time,
}
def json_benchmarks():
import orjson
import ujson
import rapidjson
import simdjson
simdjson_ver = importlib.metadata.version("pysimdjson")
rj_dumps = rapidjson.Encoder()
rj_loads = rapidjson.Decoder()
def uj_dumps(obj):
return ujson.dumps(obj)
enc = msgspec.json.Encoder()
dec = msgspec.json.Decoder(Directory)
dec2 = msgspec.json.Decoder()
return [
Benchmark("msgspec structs", None, enc.encode, dec.decode, Directory),
Benchmark("msgspec", msgspec.__version__, enc.encode, dec2.decode),
Benchmark("json", None, json.dumps, json.loads),
Benchmark("orjson", orjson.__version__, orjson.dumps, orjson.loads),
Benchmark("ujson", ujson.__version__, uj_dumps, ujson.loads),
Benchmark("rapidjson", rapidjson.__version__, rj_dumps, rj_loads),
Benchmark("simdjson", simdjson_ver, simdjson.dumps, simdjson.loads),
]
def msgpack_benchmarks():
import msgpack
import ormsgpack
enc = msgspec.msgpack.Encoder()
dec = msgspec.msgpack.Decoder(Directory)
dec2 = msgspec.msgpack.Decoder()
return [
Benchmark("msgspec structs", None, enc.encode, dec.decode, Directory),
Benchmark("msgspec", msgspec.__version__, enc.encode, dec2.decode),
Benchmark("msgpack", msgpack.__version__, msgpack.dumps, msgpack.loads),
Benchmark(
"ormsgpack", ormsgpack.__version__, ormsgpack.packb, ormsgpack.unpackb
),
]
def main():
import argparse
parser = argparse.ArgumentParser(
description="Benchmark different python serialization libraries"
)
parser.add_argument(
"--versions",
action="store_true",
help="Output library version info, and exit immediately",
)
parser.add_argument(
"-n",
type=int,
help="The number of objects in the generated data, defaults to 1000",
default=1000,
)
parser.add_argument(
"-p",
"--protocol",
choices=["json", "msgpack"],
default="json",
help="The protocol to benchmark, defaults to JSON",
)
parser.add_argument(
"--json",
action="store_true",
help="whether to output the results as json",
)
args = parser.parse_args()
benchmarks = json_benchmarks() if args.protocol == "json" else msgpack_benchmarks()
if args.versions:
for bench in benchmarks:
if bench.version is not None:
print(f"- {bench.label}: {bench.version}")
sys.exit(0)
data = make_filesystem_data(args.n)
results = [benchmark.run(data) for benchmark in benchmarks]
if args.json:
for line in results:
print(json.dumps(line))
else:
# Compose the results table
results.sort(key=lambda row: row["encode"] + row["decode"])
best_et = results[0]["encode"]
best_dt = results[0]["decode"]
best_tt = best_et + best_dt
columns = (
"",
"encode (μs)",
"vs.",
"decode (μs)",
"vs.",
"total (μs)",
"vs.",
)
rows = [
(
r["label"],
f"{1_000_000 * r['encode']:.1f}",
f"{r['encode'] / best_et:.1f}",
f"{1_000_000 * r['decode']:.1f}",
f"{r['decode'] / best_dt:.1f}",
f"{1_000_000 * (r['encode'] + r['decode']):.1f}",
f"{(r['encode'] + r['decode']) / best_tt:.1f}",
)
for r in results
]
widths = tuple(
max(max(map(len, x)), len(c)) for x, c in zip(zip(*rows), columns)
)
row_template = ("|" + (" %%-%ds |" * len(columns))) % widths
header = row_template % tuple(columns)
bar_underline = "+%s+" % "+".join("=" * (w + 2) for w in widths)
bar = "+%s+" % "+".join("-" * (w + 2) for w in widths)
parts = [bar, header, bar_underline]
for r in rows:
parts.append(row_template % r)
parts.append(bar)
print("\n".join(parts))
if __name__ == "__main__":
main()
python-msgspec-0.19.0/benchmarks/bench_gc.py 0000664 0000000 0000000 00000005375 14733557262 0021030 0 ustar 00root root 0000000 0000000 """This file benchmarks GC collection time for a large number of tiny
dataclass-like instances.
For each type, the following is measured:
- Time for a single full GC pass over all the data.
- Amount of memory used to hold all the data
"""
import gc
import sys
import time
import msgspec
def sizeof(x, _seen=None):
"""Get the recursive sizeof for an object (memoized).
Not generic, works on types used in this benchmark.
"""
if _seen is None:
_seen = set()
_id = id(x)
if _id in _seen:
return 0
_seen.add(_id)
size = sys.getsizeof(x)
if isinstance(x, dict):
for k, v in x.items():
size += sizeof(k, _seen)
size += sizeof(v, _seen)
if hasattr(x, "__dict__"):
size += sizeof(x.__dict__, _seen)
if hasattr(x, "__slots__"):
for k in x.__slots__:
size += sizeof(k, _seen)
size += sizeof(getattr(x, k), _seen)
return size
class Point(msgspec.Struct):
x: int
y: int
z: int
class PointGCFalse(msgspec.Struct, gc=False):
x: int
y: int
z: int
class PointClass:
def __init__(self, x, y, z):
self.x = x
self.y = y
self.z = z
class PointClassSlots:
__slots__ = ("x", "y", "z")
def __init__(self, x, y, z):
self.x = x
self.y = y
self.z = z
def bench_gc(cls):
# Allocate a dict of structs
data = {i: cls(i, i, i) for i in range(1_000_000)}
# Run a full collection
start = time.perf_counter()
gc.collect()
stop = time.perf_counter()
gc_time = (stop - start) * 1e3
mibytes = sizeof(data) / (2**20)
return gc_time, mibytes
def format_table(results):
columns = ("", "GC time (ms)", "Memory Used (MiB)")
rows = []
for name, t, mem in results:
rows.append((f"**{name}**", f"{t:.2f}", f"{mem:.2f}"))
widths = tuple(max(max(map(len, x)), len(c)) for x, c in zip(zip(*rows), columns))
row_template = ("|" + (" %%-%ds |" * len(columns))) % widths
header = row_template % tuple(columns)
bar_underline = "+%s+" % "+".join("=" * (w + 2) for w in widths)
bar = "+%s+" % "+".join("-" * (w + 2) for w in widths)
parts = [bar, header, bar_underline]
for r in rows:
parts.append(row_template % r)
parts.append(bar)
return "\n".join(parts)
def main():
results = []
for name, cls in [
("standard class", PointClass),
("standard class with __slots__", PointClassSlots),
("msgspec struct", Point),
("msgspec struct with gc=False", PointGCFalse),
]:
print(f"Benchmarking {name}...")
gc_time, mibytes = bench_gc(cls)
results.append((name, gc_time, mibytes))
print(format_table(results))
if __name__ == "__main__":
main()
python-msgspec-0.19.0/benchmarks/bench_large_json.py 0000664 0000000 0000000 00000010114 14733557262 0022545 0 ustar 00root root 0000000 0000000 import subprocess
import sys
import tempfile
import requests
TEMPLATE = """
import resource
import time
with open({path!r}, "rb") as f:
data = f.read()
initial_rss = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
{setup}
start = time.perf_counter()
for _ in range(5):
decode(data)
stop = time.perf_counter()
max_rss = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
# mem_mib = (max_rss * 1024 - len(data)) / (1024 * 1024)
mem_mib = (max_rss - initial_rss) / 1024
time_ms = ((stop - start) / 5) * 1000
print([mem_mib, time_ms])
"""
JSON = """
import json
decode = json.loads
"""
UJSON = """
import ujson
decode = ujson.loads
"""
ORJSON = """
import orjson
decode = orjson.loads
"""
RAPIDJSON = """
import rapidjson
decode = rapidjson.loads
"""
SIMDJSON = """
import simdjson
decode = simdjson.loads
"""
MSGSPEC = """
import msgspec
decode = msgspec.json.decode
"""
MSGSPEC_STRUCTS = """
import msgspec
from typing import Union
class Package(msgspec.Struct, gc=False):
build: str
build_number: int
depends: tuple[str, ...]
md5: str
name: str
sha256: str
subdir: str
version: str
license: str = ""
noarch: Union[str, bool, None] = None
size: int = 0
timestamp: int = 0
class RepoData(msgspec.Struct, gc=False):
repodata_version: int
info: dict
packages: dict[str, Package]
removed: tuple[str, ...]
decode = msgspec.json.Decoder(RepoData).decode
"""
def main():
import argparse
parser = argparse.ArgumentParser(
description="Benchmark decoding a large JSON message using various JSON libraries"
)
parser.add_argument(
"--versions",
action="store_true",
help="Output library version info, and exit immediately",
)
args = parser.parse_args()
benchmarks = [
("json", None, JSON),
("ujson", "ujson", UJSON),
("orjson", "orjson", ORJSON),
("rapidjson", "python-rapidjson", RAPIDJSON),
("simdjson", "pysimdjson", SIMDJSON),
("msgspec", "msgspec", MSGSPEC),
("msgspec structs", None, MSGSPEC_STRUCTS),
]
if args.versions:
import importlib.metadata
for _, lib, _ in benchmarks:
if lib is not None:
version = importlib.metadata.version(lib)
print(f"- {lib}: {version}")
sys.exit(0)
with tempfile.NamedTemporaryFile() as f:
# Download the repodata.json
resp = requests.get(
"https://conda.anaconda.org/conda-forge/noarch/repodata.json"
)
resp.raise_for_status()
f.write(resp.content)
# Run the benchmark for each library
results = {}
import ast
for lib, _, setup in benchmarks:
script = TEMPLATE.format(path=f.name, setup=setup)
# We execute each script in a subprocess to isolate their memory usage
output = subprocess.check_output([sys.executable, "-c", script])
results[lib] = ast.literal_eval(output.decode())
# Compose the results table
best_mem, best_time = results["msgspec structs"]
columns = (
"",
"memory (MiB)",
"vs.",
"time (ms)",
"vs.",
)
rows = [
(
f"**{lib}**",
f"{mem:.1f}",
f"{mem / best_mem:.1f}x",
f"{time:.1f}",
f"{time / best_time:.1f}x",
)
for lib, (mem, time) in results.items()
]
rows.sort(key=lambda x: float(x[1]))
widths = tuple(
max(max(map(len, x)), len(c)) for x, c in zip(zip(*rows), columns)
)
row_template = ("|" + (" %%-%ds |" * len(columns))) % widths
header = row_template % tuple(columns)
bar_underline = "+%s+" % "+".join("=" * (w + 2) for w in widths)
bar = "+%s+" % "+".join("-" * (w + 2) for w in widths)
parts = [bar, header, bar_underline]
for r in rows:
parts.append(row_template % r)
parts.append(bar)
print("\n".join(parts))
if __name__ == "__main__":
main()
python-msgspec-0.19.0/benchmarks/bench_library_size.py 0000664 0000000 0000000 00000006466 14733557262 0023137 0 ustar 00root root 0000000 0000000 """
This benchmark compares the installed library size between msgspec and pydantic
in a Python 3.10 x86 environment.
"""
import io
import zipfile
import requests
def get_latest_noarch_wheel_size(library):
"""Get the total uncompressed size of the latest noarch wheel"""
resp = requests.get(f"https://pypi.org/pypi/{library}/json").json()
version = resp["info"]["version"]
files = {}
for file_info in resp["releases"][version]:
name = file_info["filename"]
url = file_info["url"]
if name.endswith(".whl"):
files[name] = url
if len(files) != 1:
raise ValueError(
f"Expected to find only 1 matching file for {library}, got {list(files)}"
)
url = list(files.values())[0]
resp = requests.get(url)
fil = io.BytesIO(resp.content)
zfil = zipfile.ZipFile(fil)
size = sum(f.file_size for f in zfil.filelist)
return version, size
def get_latest_manylinux_wheel_size(library):
"""Get the total uncompressed size of the latest Python 3.10 manylinux
x86_64 wheel for the library"""
resp = requests.get(f"https://pypi.org/pypi/{library}/json").json()
version = resp["info"]["version"]
files = {}
for file_info in resp["releases"][version]:
name = file_info["filename"]
url = file_info["url"]
if "310" in name and "manylinux_2_17_x86_64" in name and "pp73" not in name:
files[name] = url
if len(files) != 1:
raise ValueError(
f"Expected to find only 1 matching file for {library}, got {list(files)}"
)
url = list(files.values())[0]
resp = requests.get(url)
fil = io.BytesIO(resp.content)
zfil = zipfile.ZipFile(fil)
size = sum(f.file_size for f in zfil.filelist)
return version, size
def main():
msgspec_version, msgspec_size = get_latest_manylinux_wheel_size("msgspec")
pydantic_version, pydantic_size = get_latest_noarch_wheel_size("pydantic")
_, pydantic_core_size = get_latest_manylinux_wheel_size("pydantic-core")
_, typing_extensions_size = get_latest_noarch_wheel_size("typing-extensions")
_, annotated_types_size = get_latest_noarch_wheel_size("annotated-types")
data = [
("msgspec", msgspec_version, msgspec_size),
(
"pydantic",
pydantic_version,
pydantic_size
+ pydantic_core_size
+ typing_extensions_size
+ annotated_types_size,
),
]
data.sort(key=lambda x: x[2])
msgspec_size = next(s for l, _, s in data if l == "msgspec")
columns = ("", "version", "size (MiB)", "vs. msgspec")
rows = [
(
f"**{lib}**",
version,
f"{size / (1024 * 1024):.2f}",
f"{size / msgspec_size:.2f}x",
)
for lib, version, size in data
]
widths = tuple(max(max(map(len, x)), len(c)) for x, c in zip(zip(*rows), columns))
row_template = ("|" + (" %%-%ds |" * len(columns))) % widths
header = row_template % tuple(columns)
bar_underline = "+%s+" % "+".join("=" * (w + 2) for w in widths)
bar = "+%s+" % "+".join("-" * (w + 2) for w in widths)
parts = [bar, header, bar_underline]
for r in rows:
parts.append(row_template % r)
parts.append(bar)
print("\n".join(parts))
if __name__ == "__main__":
main()
python-msgspec-0.19.0/benchmarks/bench_structs.py 0000664 0000000 0000000 00000012260 14733557262 0022135 0 ustar 00root root 0000000 0000000 """This file benchmarks dataclass-like libraries. It measures the following
operations:
- Time to import a new class definition
- Time to create an instance of that class
- Time to compare an instance of that class with another instance.
"""
from time import perf_counter
order_template = """
def __{method}__(self, other):
if type(self) is not type(other):
return NotImplemented
return (
(self.a, self.b, self.c, self.d, self.e) {op}
(other.a, other.b, other.c, other.d, other.e)
)
"""
classes_template = """
import reprlib
class C{n}:
def __init__(self, a, b, c, d, e):
self.a = a
self.b = b
self.c = c
self.d = d
self.e = e
@reprlib.recursive_repr()
def __repr__(self):
return (
f"{{type(self).__name__}}(a={{self.a!r}}, b={{self.b!r}}, "
f"c={{self.c!r}}, d={{self.d!r}}, e={{self.e!r}})"
)
def __eq__(self, other):
if type(self) is not type(other):
return NotImplemented
return (
self.a == other.a and
self.b == other.b and
self.c == other.c and
self.d == other.d and
self.e == other.e
)
""" + "".join(
[
order_template.format(method="lt", op="<"),
order_template.format(method="le", op="<="),
order_template.format(method="gt", op=">"),
order_template.format(method="ge", op=">="),
]
)
attrs_template = """
from attr import define
@define(order=True)
class C{n}:
a: int
b: int
c: int
d: int
e: int
"""
dataclasses_template = """
from dataclasses import dataclass
@dataclass(order=True)
class C{n}:
a: int
b: int
c: int
d: int
e: int
"""
pydantic_template = """
from pydantic import BaseModel
class C{n}(BaseModel):
a: int
b: int
c: int
d: int
e: int
"""
msgspec_template = """
from msgspec import Struct
class C{n}(Struct, order=True):
a: int
b: int
c: int
d: int
e: int
"""
BENCHMARKS = [
("msgspec", "msgspec", msgspec_template),
("standard classes", None, classes_template),
("attrs", "attrs", attrs_template),
("dataclasses", None, dataclasses_template),
("pydantic", "pydantic", pydantic_template),
]
def bench(name, template):
N_classes = 100
source = "\n".join(template.format(n=i) for i in range(N_classes))
code_obj = compile(source, "__main__", "exec")
# Benchmark defining new types
N = 200
start = perf_counter()
for _ in range(N):
ns = {}
exec(code_obj, ns)
end = perf_counter()
define_time = ((end - start) / (N * N_classes)) * 1e6
C = ns["C0"]
# Benchmark creating new instances
N = 1000
M = 1000
start = perf_counter()
for _ in range(N):
[C(a=i, b=i, c=i, d=i, e=i) for i in range(M)]
end = perf_counter()
init_time = ((end - start) / (N * M)) * 1e6
# Benchmark equality
N = 1000
M = 1000
val = M - 1
needle = C(a=val, b=val, c=val, d=val, e=val)
haystack = [C(a=i, b=i, c=i, d=i, e=i) for i in range(M)]
start = perf_counter()
for _ in range(N):
haystack.index(needle)
end = perf_counter()
equality_time = ((end - start) / (N * M)) * 1e6
# Benchmark order
try:
needle < needle
except TypeError:
order_time = None
else:
start = perf_counter()
for _ in range(N):
for obj in haystack:
if obj >= needle:
break
end = perf_counter()
order_time = ((end - start) / (N * M)) * 1e6
return (name, define_time, init_time, equality_time, order_time)
def format_table(results):
columns = (
"",
"import (μs)",
"create (μs)",
"equality (μs)",
"order (μs)",
)
def f(n):
return "N/A" if n is None else f"{n:.2f}"
rows = []
for name, *times in results:
rows.append((f"**{name}**", *(f(t) for t in times)))
widths = tuple(max(max(map(len, x)), len(c)) for x, c in zip(zip(*rows), columns))
row_template = ("|" + (" %%-%ds |" * len(columns))) % widths
header = row_template % tuple(columns)
bar_underline = "+%s+" % "+".join("=" * (w + 2) for w in widths)
bar = "+%s+" % "+".join("-" * (w + 2) for w in widths)
parts = [bar, header, bar_underline]
for r in rows:
parts.append(row_template % r)
parts.append(bar)
return "\n".join(parts)
def main():
import argparse
parser = argparse.ArgumentParser(description="Benchmark msgspec Struct operations")
parser.add_argument(
"--versions",
action="store_true",
help="Output library version info, and exit immediately",
)
args = parser.parse_args()
if args.versions:
import sys
import importlib.metadata
for _, lib, _ in BENCHMARKS:
if lib is not None:
version = importlib.metadata.version(lib)
print(f"- {lib}: {version}")
sys.exit(0)
results = []
for name, _, source in BENCHMARKS:
results.append(bench(name, source))
print(format_table(results))
if __name__ == "__main__":
main()
python-msgspec-0.19.0/benchmarks/bench_validation/ 0000775 0000000 0000000 00000000000 14733557262 0022205 5 ustar 00root root 0000000 0000000 python-msgspec-0.19.0/benchmarks/bench_validation/__init__.py 0000664 0000000 0000000 00000000000 14733557262 0024304 0 ustar 00root root 0000000 0000000 python-msgspec-0.19.0/benchmarks/bench_validation/__main__.py 0000664 0000000 0000000 00000006020 14733557262 0024275 0 ustar 00root root 0000000 0000000 import argparse
import json
import tempfile
from ..generate_data import make_filesystem_data
import sys
import subprocess
LIBRARIES = ["msgspec", "mashumaro", "cattrs", "pydantic"]
def parse_list(value):
libs = [lib.strip() for lib in value.split(",")]
for lib in libs:
if lib not in LIBRARIES:
print(f"{lib!r} is not a supported library, choose from {LIBRARIES}")
sys.exit(1)
return libs
parser = argparse.ArgumentParser(
description="Benchmark different python validation libraries"
)
parser.add_argument(
"--json",
action="store_true",
help="Whether to output the results as json",
)
parser.add_argument(
"-n",
type=int,
help="The number of objects in the generated data, defaults to 1000",
default=1000,
)
parser.add_argument(
"--libs",
type=parse_list,
help="A comma-separated list of libraries to benchmark. Defaults to all.",
default=LIBRARIES,
)
parser.add_argument(
"--versions",
action="store_true",
help="Output library version info, and exit immediately",
)
args = parser.parse_args()
if args.versions:
import importlib.metadata
for lib in args.libs:
version = importlib.metadata.version(lib)
print(f"- {lib}: {version}")
sys.exit(0)
data = json.dumps(make_filesystem_data(args.n)).encode("utf-8")
results = []
with tempfile.NamedTemporaryFile() as f:
f.write(data)
f.flush()
for lib in args.libs:
res = subprocess.check_output(
[sys.executable, "-m", "benchmarks.bench_validation.runner", lib, f.name]
)
results.append(json.loads(res))
if args.json:
for line in results:
print(json.dumps(line))
else:
# Compose the results table
results.sort(key=lambda row: row["encode"] + row["decode"])
best_et = results[0]["encode"]
best_dt = results[0]["decode"]
best_tt = best_et + best_dt
best_mem = results[0]["memory"]
columns = (
"",
"encode (μs)",
"vs.",
"decode (μs)",
"vs.",
"total (μs)",
"vs.",
"memory (MiB)",
"vs.",
)
rows = [
(
r["label"],
f"{1_000_000 * r['encode']:.1f}",
f"{r['encode'] / best_et:.1f}",
f"{1_000_000 * r['decode']:.1f}",
f"{r['decode'] / best_dt:.1f}",
f"{1_000_000 * (r['encode'] + r['decode']):.1f}",
f"{(r['encode'] + r['decode']) / best_tt:.1f}",
f"{r['memory']:.1f}",
f"{r['memory'] / best_mem:.1f}",
)
for r in results
]
widths = tuple(max(max(map(len, x)), len(c)) for x, c in zip(zip(*rows), columns))
row_template = ("|" + (" %%-%ds |" * len(columns))) % widths
header = row_template % tuple(columns)
bar_underline = "+%s+" % "+".join("=" * (w + 2) for w in widths)
bar = "+%s+" % "+".join("-" * (w + 2) for w in widths)
parts = [bar, header, bar_underline]
for r in rows:
parts.append(row_template % r)
parts.append(bar)
print("\n".join(parts))
python-msgspec-0.19.0/benchmarks/bench_validation/bench_cattrs.py 0000664 0000000 0000000 00000001736 14733557262 0025225 0 ustar 00root root 0000000 0000000 from __future__ import annotations
import enum
import datetime
from typing import Literal
import attrs
import cattrs.preconf.orjson
class Permissions(enum.Enum):
READ = "READ"
WRITE = "WRITE"
READ_WRITE = "READ_WRITE"
@attrs.define(kw_only=True)
class File:
name: str
created_by: str
created_at: datetime.datetime
updated_by: str | None = None
updated_at: datetime.datetime | None = None
nbytes: int
permissions: Permissions
type: Literal["file"] = "file"
@attrs.define(kw_only=True)
class Directory:
name: str
created_by: str
created_at: datetime.datetime
updated_by: str | None = None
updated_at: datetime.datetime | None = None
contents: list[File | Directory]
type: Literal["directory"] = "directory"
converter = cattrs.preconf.orjson.make_converter(omit_if_default=True)
def encode(obj):
return converter.dumps(obj)
def decode(msg):
return converter.loads(msg, Directory)
label = "cattrs"
python-msgspec-0.19.0/benchmarks/bench_validation/bench_mashumaro.py 0000664 0000000 0000000 00000002204 14733557262 0025710 0 ustar 00root root 0000000 0000000 from __future__ import annotations
import enum
import dataclasses
import datetime
from typing import Literal
from mashumaro.mixins.orjson import DataClassORJSONMixin
class Permissions(enum.Enum):
READ = "READ"
WRITE = "WRITE"
READ_WRITE = "READ_WRITE"
@dataclasses.dataclass(kw_only=True)
class File(DataClassORJSONMixin):
name: str
created_by: str
created_at: datetime.datetime
updated_by: str | None = None
updated_at: datetime.datetime | None = None
nbytes: int
permissions: Permissions
type: Literal["file"] = "file"
class Config:
omit_default = True
lazy_compilation = True
@dataclasses.dataclass(kw_only=True)
class Directory(DataClassORJSONMixin):
name: str
created_by: str
created_at: datetime.datetime
updated_by: str | None = None
updated_at: datetime.datetime | None = None
contents: list[File | Directory]
type: Literal["directory"] = "directory"
class Config:
omit_default = True
lazy_compilation = True
label = "mashumaro"
def encode(x):
return x.to_json()
def decode(msg):
return Directory.from_json(msg)
python-msgspec-0.19.0/benchmarks/bench_validation/bench_msgspec.py 0000664 0000000 0000000 00000001524 14733557262 0025361 0 ustar 00root root 0000000 0000000 from __future__ import annotations
import enum
import datetime
import msgspec
class Permissions(enum.Enum):
READ = "READ"
WRITE = "WRITE"
READ_WRITE = "READ_WRITE"
class File(msgspec.Struct, kw_only=True, omit_defaults=True, tag="file"):
name: str
created_by: str
created_at: datetime.datetime
updated_by: str | None = None
updated_at: datetime.datetime | None = None
nbytes: int
permissions: Permissions
class Directory(msgspec.Struct, kw_only=True, omit_defaults=True, tag="directory"):
name: str
created_by: str
created_at: datetime.datetime
updated_by: str | None = None
updated_at: datetime.datetime | None = None
contents: list[File | Directory]
enc = msgspec.json.Encoder()
dec = msgspec.json.Decoder(Directory)
label = "msgspec"
encode = enc.encode
decode = dec.decode
python-msgspec-0.19.0/benchmarks/bench_validation/bench_pydantic.py 0000664 0000000 0000000 00000002245 14733557262 0025534 0 ustar 00root root 0000000 0000000 from __future__ import annotations
import enum
import datetime
from typing import Literal, Annotated
import pydantic
class Permissions(enum.Enum):
READ = "READ"
WRITE = "WRITE"
READ_WRITE = "READ_WRITE"
class File(pydantic.BaseModel):
type: Literal["file"] = "file"
name: str
created_by: str
created_at: datetime.datetime
updated_by: str | None = None
updated_at: datetime.datetime | None = None
nbytes: int
permissions: Permissions
class Directory(pydantic.BaseModel):
type: Literal["directory"] = "directory"
name: str
created_by: str
created_at: datetime.datetime
updated_by: str | None = None
updated_at: datetime.datetime | None = None
contents: list[Annotated[File | Directory, pydantic.Field(discriminator="type")]]
if pydantic.__version__.startswith("2."):
label = "pydantic v2"
def encode(obj):
return obj.model_dump_json(exclude_defaults=True)
def decode(msg):
return Directory.model_validate_json(msg)
else:
label = "pydantic v1"
def encode(obj):
return obj.json(exclude_defaults=True)
def decode(msg):
return Directory.parse_raw(msg)
python-msgspec-0.19.0/benchmarks/bench_validation/runner.py 0000664 0000000 0000000 00000001613 14733557262 0024071 0 ustar 00root root 0000000 0000000 import importlib
import json
import timeit
import resource
import sys
import gc
library, path = sys.argv[1:3]
with open(path, "rb") as f:
json_data = f.read()
initial_rss = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
mod = importlib.import_module(f"benchmarks.bench_validation.bench_{library}")
msg = mod.decode(json_data)
gc.collect()
timer = timeit.Timer("func(data)", setup="", globals={"func": mod.encode, "data": msg})
n, t = timer.autorange()
encode_time = t / n
del msg
gc.collect()
timer = timeit.Timer(
"func(data)", setup="", globals={"func": mod.decode, "data": json_data}
)
n, t = timer.autorange()
decode_time = t / n
max_rss = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
report = json.dumps(
{
"label": mod.label,
"encode": encode_time,
"decode": decode_time,
"memory": (max_rss - initial_rss) / 1024,
}
)
print(report)
python-msgspec-0.19.0/benchmarks/generate_data.py 0000664 0000000 0000000 00000005220 14733557262 0022050 0 ustar 00root root 0000000 0000000 import datetime
import random
import string
class Generator:
UTC = datetime.timezone.utc
DATE_2018 = datetime.datetime(2018, 1, 1, tzinfo=UTC)
DATE_2023 = datetime.datetime(2023, 1, 1, tzinfo=UTC)
PERMISSIONS = ["READ", "WRITE", "READ_WRITE"]
NAMES = [
"alice",
"ben",
"carol",
"daniel",
"esther",
"franklin",
"genevieve",
"harold",
"ilana",
"jerome",
"katelyn",
"leonard",
"monique",
"nathan",
"ora",
"patrick",
"quinn",
"ronald",
"stephanie",
"thomas",
"uma",
"vince",
"wendy",
"xavier",
"yitzchak",
"zahra",
]
def __init__(self, capacity, seed=42):
self.capacity = capacity
self.random = random.Random(seed)
def randdt(self, min, max):
ts = self.random.randint(min.timestamp(), max.timestamp())
return datetime.datetime.fromtimestamp(ts).replace(tzinfo=self.UTC)
def randstr(self, min=None, max=None):
if max is not None:
min = self.random.randint(min, max)
return "".join(self.random.choices(string.ascii_letters, k=min))
def make(self, is_dir):
name = self.randstr(4, 30)
created_by = self.random.choice(self.NAMES)
created_at = self.randdt(self.DATE_2018, self.DATE_2023)
data = {
"type": "directory" if is_dir else "file",
"name": name,
"created_by": created_by,
"created_at": created_at.isoformat(),
}
if self.random.random() > 0.75:
updated_by = self.random.choice(self.NAMES)
updated_at = self.randdt(created_at, self.DATE_2023)
data.update(
updated_by=updated_by,
updated_at=updated_at.isoformat(),
)
if is_dir:
n = min(self.random.randint(0, 30), self.capacity)
self.capacity -= n
data["contents"] = [self.make_node() for _ in range(n)]
else:
data["nbytes"] = self.random.randint(0, 1000000)
data["permissions"] = self.random.choice(self.PERMISSIONS)
return data
def make_node(self):
return self.make(self.random.random() > 0.8)
def generate(self):
self.capacity -= 1
if self.capacity == 0:
out = self.make(False)
else:
out = self.make(True)
while self.capacity:
self.capacity -= 1
out["contents"].append(self.make_node())
return out
def make_filesystem_data(n):
return Generator(n).generate()
python-msgspec-0.19.0/docs/ 0000775 0000000 0000000 00000000000 14733557262 0015527 5 ustar 00root root 0000000 0000000 python-msgspec-0.19.0/docs/Makefile 0000664 0000000 0000000 00000001216 14733557262 0017167 0 ustar 00root root 0000000 0000000 # Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?= -W --keep-going
SPHINXBUILD ?= sphinx-build
SOURCEDIR = source
BUILDDIR = build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
python-msgspec-0.19.0/docs/make.bat 0000664 0000000 0000000 00000001437 14733557262 0017141 0 ustar 00root root 0000000 0000000 @ECHO OFF
pushd %~dp0
REM Command file for Sphinx documentation
if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=source
set BUILDDIR=build
if "%1" == "" goto help
%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
goto end
:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
:end
popd
python-msgspec-0.19.0/docs/source/ 0000775 0000000 0000000 00000000000 14733557262 0017027 5 ustar 00root root 0000000 0000000 python-msgspec-0.19.0/docs/source/_static/ 0000775 0000000 0000000 00000000000 14733557262 0020455 5 ustar 00root root 0000000 0000000 python-msgspec-0.19.0/docs/source/_static/anywidget.png 0000664 0000000 0000000 00000035136 14733557262 0023166 0 ustar 00root root 0000000 0000000 PNG
IHDR DH iCCPICC profile (}=HPOS";8dNq*BZu0y4iHZ\ׂ?Ug]\AIEJ/)}w Qf4jq1]"&3˘$'0ď\W\~\pX!3'V:Mx8j:U[r_+\5 "PBUDhIKRU#* ;~?=[+tqŶ?ƀ.Ьm7O 3p0Izm⺭){0dȦH~ZB>7e[o͝[@fe{snr? bKGD pHYs
B(x tIME7,N| IDATxy@ṴA@%P8)=NiCfNijGRxHM͉qYAQAQA6ck~*EֺneUU -z w ; p @ w p @ w ; p @ ; p w ; p @ w ; @ w ; p @ w ; p w ; p @ DI@,*uⲲ²²Ҋ2"LWUP(K++TF*R044774472252250014410076VT;P**rrs2
czulbfNnV[Y(JOYUUEЀ23Srs\ڞ'ѺT}5VV&GҴ~!-
65mkkjccoiikanwH*=7^ffLz=5i]SsccJ.%++>#ZZZxJrٖbիNN]7`;'NFƵv%$-- O52ť)A_ȸ|sr(H
kkր'/AC *ލ"%ȈpG(jo>xp] YOo˫m܆z22N%$V*?hͭCf<wԉԜn[/k]]h4yv;^^VeiiTeK-\mmwܼ kk;Օ Q#Z.:1חs.xmߺ Ꭷ+.+xkܱׄO5iXv͚+w.3?͛ƍhjx+wqs320 eAvvxt 41yS
(%+koL;qqB|
o߾;( w]Fʕ(ן;P;,ѣ/.HJr;vlRUP˵koR
14ӳ>k&ׯϺv-EjnA.]7IbU\VvƍWVVR
L;]j4;DEݽYVAB5puZM)w@bf7g~N)LjyN۷g"pprJ;P
¢N5y:8ⅭnbtVffp de}qĚLJi`[^[,
`qq&&p˛lJA`je>-i*I$'O:~v&Rk !QS11(nF eeߜ81=JQfk8 ܜRK)YY= *VjkK)w